code: def _compress_tokens(tokens):
      
      recorder = None
  
      def _edge_case_stray_end_quoted(tokens, index):
          """Convert stray end_quoted_literals to unquoted_literals."""
          # In this case, "tokenize" the matched token into what it would
          # have looked like had the last quote not been there. Put the
          # last quote on the end of the final token and call it an
          # unquoted_literal
          tokens[index] = Token(type=TokenType.UnquotedLiteral,
                                content=tokens[index].content,
                                line=tokens[index].line,
                                col=tokens[index].col)
  
      tokens_len = len(tokens)
      index = 0
  
      with _EdgeCaseStrayParens() as edge_case_stray_parens:
          edge_cases = [
              (_is_paren_type, edge_case_stray_parens),
              (_is_end_quoted_type, _edge_case_stray_end_quoted),
          ]
  
          while index < tokens_len:
  
              recorder = _find_recorder(recorder, tokens, index)
  
              if recorder is not None:
                  # Do recording
                  result = recorder.consume_token(tokens, index, tokens_len)
                  if result is not None:
                      (index, tokens_len, tokens) = result
                      recorder = None
  
              else:
                  # Handle edge cases
                  for matcher, handler in edge_cases:
                      if matcher(tokens[index].type):
                          handler(tokens, index)
  
              index += 1
  
      return tokens
docs: Paste multi-line strings, comments, RST etc together.
  
      This function works by iterating over each over the _RECORDERS to determine
      if we should start recording a token sequence for pasting together. If
      it finds one, then we keep recording until that recorder is done and
      returns a pasted together token sequence. Keep going until we reach
      the end of the sequence.
  
      The sequence is modified in place, so any function that modifies it
      must return its new length. This is also why we use a while loop here.