code: def preprocess_dataset(data, min_freq=5, max_vocab_size=None):
      
      with print_time('count and construct vocabulary'):
          counter = nlp.data.count_tokens(itertools.chain.from_iterable(data))
          vocab = nlp.Vocab(counter, unknown_token=None, padding_token=None,
                            bos_token=None, eos_token=None, min_freq=min_freq,
                            max_size=max_vocab_size)
          idx_to_counts = [counter[w] for w in vocab.idx_to_token]
  
      def code(sentence):
          return [vocab[token] for token in sentence if token in vocab]
  
      with print_time('code data'):
          data = data.transform(code, lazy=False)
      data = nlp.data.SimpleDataStream([data])
      return data, vocab, idx_to_counts
docs: Dataset preprocessing helper.
  
      Parameters
      ----------
      data : mx.data.Dataset
          Input Dataset. For example gluonnlp.data.Text8 or gluonnlp.data.Fil9
      min_freq : int, default 5
          Minimum token frequency for a token to be included in the vocabulary
          and returned DataStream.
      max_vocab_size : int, optional
          Specifies a maximum size for the vocabulary.
  
      Returns
      -------
      gluonnlp.data.DataStream
          Each sample is a valid input to
          gluonnlp.data.EmbeddingCenterContextBatchify.
      gluonnlp.Vocab
          Vocabulary of all tokens in Text8 that occur at least min_freq times of
          maximum size max_vocab_size.
      idx_to_counts : list of int
          Mapping from token indices to their occurrence-counts in the Text8
          dataset.