parameters: (tokens, lines, callback)
return_statement: return ret
docstring: Traverses a list of tokens to identify functions.
  
    Then uses a callback to perform some work on the functions.  Each function
    seen gets a new State object created from the given callback method; there is
    a single State for global code which is given None in the constructor.  Then,
    each token seen is passed to the 'add' method of the State.  This is used by
    the State to either calculate sizes, print tokens, or detect dependencies.
    The 'build' method is called at the end of the function to create a result
    object that is returned as an array at the end.
  
    Args:
      tokens: An array of Tokens.
      lines: An array of compiled code lines.
      callback: A constructor that returns a state object.  It takes a start
          token or None if outside a function.  It has two member functions
            add - accepts the current token and the token's index.
            build - returns an object to be added to the results.
  
    Returns:
      an array of State objects in a format controlled by the callback.
docstring_summary: Traverses a list of tokens to identify functions.
docstring_tokens: Traverses a list of tokens to identify functions .
function: def traverse_tokens(tokens, lines, callback):
    """Traverses a list of tokens to identify functions.
  
    Then uses a callback to perform some work on the functions.  Each function
    seen gets a new State object created from the given callback method; there is
    a single State for global code which is given None in the constructor.  Then,
    each token seen is passed to the 'add' method of the State.  This is used by
    the State to either calculate sizes, print tokens, or detect dependencies.
    The 'build' method is called at the end of the function to create a result
    object that is returned as an array at the end.
  
    Args:
      tokens: An array of Tokens.
      lines: An array of compiled code lines.
      callback: A constructor that returns a state object.  It takes a start
          token or None if outside a function.  It has two member functions
            add - accepts the current token and the token's index.
            build - returns an object to be added to the results.
  
    Returns:
      an array of State objects in a format controlled by the callback.
    """
  
    ret = []
    state = callback(None, None)
    # Create a token iterator.  This is used to read tokens from the array.  We
    # cannot use a for loop because the iterator is passed to readFunction.
    token_iter = enumerate(tokens)
    try:
      while True:
        index, token = next(token_iter)
  
        if is_function(token, lines):
          ret += read_function(token_iter, token, index, lines, callback)
        else:
          state.add(token, index)
    except StopIteration:
      pass
  
    temp = state.build()
    if temp:
      ret.append(temp)
    return ret
function_tokens: def traverse_tokens ( tokens , lines , callback ) : ret = [ ] state = callback ( None , None ) # Create a token iterator.  This is used to read tokens from the array.  We # cannot use a for loop because the iterator is passed to readFunction. token_iter = enumerate ( tokens ) try : while True : index , token = next ( token_iter ) if is_function ( token , lines ) : ret += read_function ( token_iter , token , index , lines , callback ) else : state . add ( token , index ) except StopIteration : pass temp = state . build ( ) if temp : ret . append ( temp ) return ret