code: private void refreshFileList()
        throws IOException {
      FileStatus[] files = fileSystem.globStatus(inputDirectoryAndPattern);
      for (FileStatus file : files) {
        String fileName = file.getPath().getName();
        if (!knownFileSet.contains(fileName)) {
          if (endFileNumber != -1) {
            // Sanity check against endFileNumber.
            int newFileIndex = parseFileIndex(fileName);
            Preconditions.checkState(newFileIndex < endFileNumber,
                "Found new file '%s' with index %s, which isn't less than endFileNumber %s!",
                fileName, newFileIndex, endFileNumber);
          }
  
          logger.atInfo().log(
              "Adding new file '%s' of size %s to knownFileSet.", fileName, file.getLen());
          knownFileSet.add(fileName);
          fileQueue.add(file);
        }
      }
    }
docs: Lists files, and sifts through the results for any new files we haven't found before.
  If a file of size 0 is found, we mark the 'endFileNumber' from it.