code: def _remove_existing_jobs(data):
      
      new_data = []
  
      guids = [datum['job']['job_guid'] for datum in data]
      state_map = {
          guid: state for (guid, state) in Job.objects.filter(
              guid__in=guids).values_list('guid', 'state')
      }
  
      for datum in data:
          job = datum['job']
          if not state_map.get(job['job_guid']):
              new_data.append(datum)
          else:
              # should not transition from running to pending,
              # or completed to any other state
              current_state = state_map[job['job_guid']]
              if current_state == 'completed' or (
                      job['state'] == 'pending' and
                      current_state == 'running'):
                  continue
              new_data.append(datum)
  
      return new_data
docs: Remove jobs from data where we already have them in the same state.
  
      1. split the incoming jobs into pending, running and complete.
      2. fetch the ``job_guids`` from the db that are in the same state as they
         are in ``data``.
      3. build a new list of jobs in ``new_data`` that are not already in
         the db and pass that back.  It could end up empty at that point.