Export `zlib.{adler,crc}32_combine`

[thatch]

Feature or enhancement

Proposal:

This is already provided by zlib, and is useful for computing a crc32 in multiple chunks (not linearly).

What I currently do is use ctypes, but that doesn't work on all environments (like Windows). I don't think we'll need a pure python fallback, just a wrapper of what's in zlib, equivalent of https://github.com/fastzip/fastzip/blob/9019107a6732fab9004e625b075faab96f29265a/fastzip/_crc32_combine.py#L63-L67

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-134635: add zlib.{adler32,crc32}_combine to combine checksums #134650

[picnixz]

cc @emmatyping (or is zstd not supporting this one? I don't know if you now want to take care of compression issues as well in general)

[emmatyping]

@picnixz this is zlib related, not zstandard, but I am hoping to take care of more compression issues going forward!

[emmatyping]

I think this request is reasonable, we probably would want to also add adler32_combine for completeness/symmetry.

I don't think we should add crc32_combine_gen/crc32_combine_op without additional motivation.

[picnixz]

this is zlib related

Yeah, I actually misread the issue first :')

I am hoping to take care of more compression issues going forward!

Great to hear!

adler32_combine

Yes, sure. Note that we could also think about whether to add binascii.crc32_combine as well (we have a minimal support for CRC32 even without zlib in binascii but I don't know if it would also make sense to expose crc32_combine in this case). I wouldn't recommend it as a first iteration (and we anyway don't have binascii.adler32)

[emmatyping]

I should note that if one is merely iterating over chunks of data, using the value argument to zlib.crc32 (i.e. setting an initial checksum) should work to compute a running checksum.

However, looking at the fastzip code, they are concurrently calculating the checksums and assembling them afterwards, so they cannot chain calls to zlib.crc32.

[picnixz]

Actually, there was #82868 and #17048 for adler32_combine but it was not tested. I'll close the older issue and will add both as part of this specific issue.