5 ms·Common Crawl delivers the data as bz2. Indeed I store intermediate data in zstd with ZFS.by rwaksmunski 1y agoCommon Crawl delivers the data as bz2. Indeed I store intermediate data in zstd with ZFS.