6 ms·
> I don't see how that's possible if the cache no longer stores the uncompressed data. Zstd has a seekable format for frames, similar to pigz --independent wor
by gopalv 14d ago
> I don't see how that's possible if the cache no longer stores the uncompressed data.
Zstd has a seekable format for frames, similar to pigz --independent works.
[1] - https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/README.md https://github.com/facebook/zstd/blob/dev/contrib/seekable_f...
- tehlike 14d agoYes. This.
- NostraDavid 13d agoSpeaking of pigz, I've run into pigzpp[1], or rather its paper: "pigzpp: Fast, Parallel, Portable Compression for the Whole Stack"[2]. Turns out we can squeeze quite a bit more compression performance out of DEFLATE - ~10x in certain instances, 2x as a base minimum (read the paper for details). [1]: https://github.com/thammegowda/pigzpp https://github.com/thammegowda/pigzpp [2]: https://arxiv.org/abs/2608.24153 https://arxiv.org/abs/2608.24153