code: def encode(self, s):
      
      sentence = s
      tokens = sentence.strip().split()
      ids = []
      ids_extend = []
      oovs = {}
      for t in tokens:
        if t in self._token_to_id:
          ids.append(self._token_to_id[t])
          ids_extend.append(self._token_to_id[t])
        else:
          next_oov_id = len(oovs)
          oov_num = oovs.get(t, next_oov_id)
          if oov_num == next_oov_id:
            oovs[t] = oov_num
          ids_extend.append(self.vocab_size + oov_num)
          ids.append(self._token_to_id[self._replace_oov])
      source_oov_id_to_token = [""] * len(oovs)
      for oov in oovs:
        source_oov_id_to_token[oovs[oov]] = oov
      if self._reverse:
        return ids[::-1], ids_extend[::-1], oovs, source_oov_id_to_token
      else:
        return ids, ids_extend, oovs, source_oov_id_to_token
docs: Converts a space-separated string of tokens to lists of ids.
  
      Also store temporary vocabulary IDs for source OOV tokens. OOVs are
      represented by their temporary OOV number. E.g., if the vocabulary size
      is 50k and the source has 3 OOVs, then these temporary OOV numbers will
      be 50000, 50001, 50002.
  
      Args:
        s: human-readable string to be converted.
  
      Returns:
        ids: list of integers
        ids_extend: list of integers including extended temporary vocab IDs for
        source OOVs.
        oovs: A dict storing source OOV words, used for the decoder to copy. The
        key is OOV word, and the value is the order they appear in the source,
        starting from 0.
        source_oov_id_to_token: a list of source OOV tokens, in the same order as
        they appear in the source.