code: public FileStatus next() throws IOException {
      if (finished) {
        LOG.warn("Should not call next() after FINISH_TOKEN is obtained.");
        return FINISH_TOKEN;
      }
      FileStatus f = null;
      try {
        f = output.take();
      } catch (InterruptedException e) {
        finished = true;
        interruptProcessors();
        throw new IOException(e);
      }
      if (f == FINISH_TOKEN) {
        LOG.info("traversal is done. Returning FINISH_TOKEN");
        finished = true;
      }
      return f;
    }
docs: Retrieves the next filtered element.
  Returns {@link FINISH_TOKEN} when traversal is done. Calling this after
  {@link FINISH_TOKEN} is returned is not allowed.