code: private Token simpleTokenLexer(Token tkn, int c) throws IOException {
          for (;;) {
              if (isEndOfLine(c)) {
                  // end of record
                  tkn.type = TT_EORECORD;
                  tkn.isReady = true;
                  break;
              } else if (isEndOfFile(c)) {
                  // end of file
                  tkn.type = TT_EOF;
                  tkn.isReady = true;
                  break;
              } else if (c == strategy.getDelimiter()) {
                  // end of token
                  tkn.type = TT_TOKEN;
                  tkn.isReady = true;
                  break;
              } else if (c == '\\' && strategy.getUnicodeEscapeInterpretation() && in.lookAhead() == 'u') {
                  // interpret unicode escaped chars (like \u0070 -> p)
                  tkn.content.append((char) unicodeEscapeLexer(c));
              } else if (c == strategy.getEscape()) {
                  tkn.content.append((char) readEscape(c));
              } else {
                  tkn.content.append((char) c);
              }
  
              c = in.read();
          }
  
          if (strategy.getIgnoreTrailingWhitespaces()) {
              tkn.content.trimTrailingWhitespace();
          }
  
          return tkn;
      }
docs: A simple token lexer
  
  Simple token are tokens which are not surrounded by encapsulators.
  A simple token might contain escaped delimiters (as \, or \;). The
  token is finished when one of the following conditions become true:
  <ul>
  <li>end of line has been reached (TT_EORECORD)</li>
  <li>end of stream has been reached (TT_EOF)</li>
  <li>an unescaped delimiter has been reached (TT_TOKEN)</li>
  </ul>
  
  @param tkn  the current token
  @param c    the current character
  @return the filled token
  
  @throws IOException on stream access error