code: def tokenize(self, s, pattern=None, active=None):
          
          if pattern is None:
              if self.tokenize_pattern is None:
                  pattern = r'[ \t]+'
              else:
                  pattern = self.tokenize_pattern
          if active is None:
              active = self.active
          return self.group.tokenize(s, pattern=pattern, active=active)
docs: Rewrite and tokenize the input string *s*.
  
          Args:
              s (str): the input string to process
              pattern (str, optional): the regular expression pattern on
                  which to split tokens; defaults to `[ \t]+`
              active (optional): a collection of external module names
                  that may be applied if called
          Returns:
              a :class:`~delphin.tokens.YyTokenLattice` containing the
              tokens and their characterization information