6 ms·
What about using zip bombs? https://idiallo.com/blog/zipbomb-protection https://idiallo.com/blog/zipbomb-protection
by s0meON3 10mo ago
What about using zip bombs?
https://idiallo.com/blog/zipbomb-protection https://idiallo.com/blog/zipbomb-protection
- lavela 10mo ago"Gzip only provides a compression ratio of a little over 1000: If I want a file that expands to 100 GB, I’ve got to serve a 100 MB asset. Worse, when I tried it, the bots just shrugged it off, with some even coming back for more." https://maurycyz.com/misc/the_cost_of_trash/#:~:text=throw%20them%20some%20bombs https://maurycyz.com/misc/the_cost_of_trash/#:~:text=throw%2...
- LunaSea 10mo agoYou could try different compression methods supported by browsers like brotli. Otherwise you can also chain compression methods like: "Content-Encoding: gzip gzip".
- deleted 10mo ago[deleted]
- kalkin 10mo agoAh cool that site's robots.txt is still broken, just like it was when it first came up on HN...
- jeroenhd 10mo agoModern browsers support brotli or zstd, which is a lot better in terms of compression. Perhaps not as good for on-the-fly compression, but static assets can get a nice compression benefit out of it. With toxic AI scrapers like Perplexity moving more and more to headless web browsers to bypass bot blocks, I think a brotli bomb (100GB of \0 can be compressed to about 78KiB with Brotli) would be quite effective.
- renegat0x0 10mo agoEven I, who does not know much, implemented a workaround. I have a web crawler and I have both scraping byte limit and timeout, so zip bombs dont bother me much. https://github.com/rumca-js/crawler-buddy https://github.com/rumca-js/crawler-buddy I think garbage blabber would be more effective.