code: def tokenize(self, data_source, callback=None):
          
  
          self._reinitialize()
  
          if callback is not None:
              self._deliver = callback
  
          while True:
              frame = data_source.read()
              if frame is None:
                  break
              self._current_frame += 1
              self._process(frame)
  
          self._post_process()
  
          if callback is None:
              _ret = self._tokens
              self._tokens = None
              return _ret
docs: Read data from `data_source`, one frame a time, and process the read frames in
          order to detect sequences of frames that make up valid tokens.
  
          :Parameters:
             `data_source` : instance of the :class:`DataSource` class that implements a `read` method.
                 'read' should return a slice of signal, i.e. frame (of whatever \
                 type as long as it can be processed by validator) and None if \
                 there is no more signal.
  
             `callback` : an optional 3-argument function.
                 If a `callback` function is given, it will be called each time a valid token
                 is found.
  
  
          :Returns:
             A list of tokens if `callback` is None. Each token is tuple with the following elements:
  
              .. code python
  
                  (data, start, end)
  
             where `data` is a list of read frames, `start`: index of the first frame in the
             original data and `end` : index of the last frame.