However, deflate only cares about the last 32 KiB or so. The whole point of the algorithm is that it's able to repeat something that it has already seen before, therefore, all data is required. The problem is that compress expects a complete file, you can't just provide a chunk to it, because the deflate algorithm relies on previous data during decompression. Here is a rework of your code that should help you going on: import httpĭecompressor = compressobj(16 + zlib.MAX_WBITS)Ĭonnection = (api, context = ssl._create_unverified_context())Ĭonnection.request('GET', api_url, headers = auth)ĭata = compress(BytesIO(chunk).read()) I see you have very strong constraints on libraries, but zlib and io are part of the python default, so hopefully you have them available. You will also need to convert your chunk to a file-like object, you can do that with io.BytesIO.
#Daa decompressor how to#
I don't know if you can manage that directly with the gzip library, but I know how to do it with zlib. Trying to decompress a part of a gzip with compress will most likely give you an EOFError saying something like Compressed file ended before the end-of-stream marker was reached. compress will expect a complete file, so you would need to reconstruct it and load it entirely in memory before doing that, which would undermine the whole point of chunking the response. Going forward, you will face another problem:īasically, I am chunking so that I don't have to load the entire response in memory. First, make sure you have 'Accept-Encoding': 'gzip' in your headers. If there is only auth-related keys in your headers like your variable name suggests, you won't receive a gzipped response. Typically, a server won't send you a compressed response if you don't specify in the request headers that you can accept it. I notice that in your code, you pass a variable called auth. The most probable reason for that is, the response your received is indeed not a gzipped file.
0 kommentar(er)
