code: def tokens2ids(tokens: Iterable[str], vocab: Dict[str, int]) -> List[int]:
      
      return [vocab.get(w, vocab[C.UNK_SYMBOL]) for w in tokens]
docs: Returns sequence of integer ids given a sequence of tokens and vocab.
  
      :param tokens: List of string tokens.
      :param vocab: Vocabulary (containing UNK symbol).
      :return: List of word ids.