code: private void getNextToken(StreamTokenizer tokenizer) throws IOException, ParseException {
          if (tokenizer.nextToken() == StreamTokenizer.TT_EOL) {
              throw new ParseException("premature end of line", tokenizer.lineno());
          }
          if (tokenizer.ttype == StreamTokenizer.TT_EOF) {
              throw new ParseException(PREMATURE_END_OF_FILE, tokenizer.lineno());
          } else if ((tokenizer.ttype == '\'') || (tokenizer.ttype == '"')) {
              tokenizer.ttype = StreamTokenizer.TT_WORD;
          } else if ((tokenizer.ttype == StreamTokenizer.TT_WORD) && (tokenizer.sval.equals("?"))) {
              tokenizer.ttype = '?';
          }
      }
docs: Gets next token, checking for a premature and of line.
  
  @throws IllegalStateException if it finds a premature end of line