code: def _chunk(self, response, size=4096):
          
          method = response.headers.get("content-encoding")
          if method == "gzip":
              d = zlib.decompressobj(16+zlib.MAX_WBITS)
              b = response.read(size)
              while b:
                  data = d.decompress(b)
                  yield data
                  b = response.read(size)
                  del data
          else:
              while True:
                  chunk = response.read(size)
                  if not chunk: break
                  yield chunk
docs: downloads a web response in pieces