code: def encode(self, raw_string, add_eos=False):
      
      ret = []
      tokens = _split_string_to_tokens(_native_to_unicode(raw_string))
      for token in tokens:
        ret.extend(self._token_to_subtoken_ids(token))
      if add_eos:
        ret.append(EOS_ID)
      return ret
docs: Encodes a string into a list of int subtoken ids.