code: void removeRobots() {
  		// Nothing to remove
  		if (pathQueries.isEmpty()) return;
  
  		// We test the first path
  		final byte[] firstPath = pathQueries.dequeue();
  		if (firstPath == VisitState.ROBOTS_PATH) {
  			// It's robots.txt: we set the last robots fetch time and don't put it back
  			lastRobotsFetch = Long.MAX_VALUE;
  			return;
  		}
  
  		// It's not robots.txt
  		if (isEmpty()) {
  			// If there are no more paths, we put it back and return
  			pathQueries.enqueueFirst(firstPath);
  			return;
  		}
  
  		// We test the second path
  		final byte[] secondPath = pathQueries.dequeue();
  		if (secondPath == VisitState.ROBOTS_PATH) {
  			// It's robots.txt: we put back the first path and set the last robots fetch time (and don't put back the second path)
  			pathQueries.enqueueFirst(firstPath);
  			lastRobotsFetch = Long.MAX_VALUE;
  			return;
  		}
  
  		// No robots.txt found: we put back everything
  		pathQueries.enqueueFirst(secondPath);
  		pathQueries.enqueueFirst(firstPath);
  	}
docs: Remove the <code>/robots.txt</code> path, if present, and in this case sets the last robots fetch
  to {@link Long#MAX_VALUE}. Note that <strong>this method is not synchronized</strong> and
  should be called only after having stopped all threads accessing this visit state.