code: def process_tokendef(cls, name, tokendefs=None):
          
          processed = cls._all_tokens[name] = {}
          tokendefs = tokendefs or cls.tokens[name]
          for state in list(tokendefs):
              cls._process_state(tokendefs, processed, state)
          return processed
docs: Preprocess a dictionary of token definitions.