code: def _get_url_hashes(path):
    
    urls = _read_text_file(path)
    def url_hash(u):
      h = hashlib.sha1()
      try:
        u = u.encode('utf-8')
      except UnicodeDecodeError:
        logging.error('Cannot hash url: %s', u)
      h.update(u)
      return h.hexdigest()
    return {url_hash(u): True for u in urls}
docs: Get hashes of urls in file.