8 ms·
It's two separate issues: 1) I do a lot of scraping, and Go concurrency + Colly has better performance 2) My DB size is exploding and I have limited budget, a
by a7fort 1y ago
It's two separate issues:
1) I do a lot of scraping, and Go concurrency + Colly has better performance
2) My DB size is exploding and I have limited budget, and it looks like CH is so much better at compressing data. I recently did a test and for the same table with same exact data, MySQL was using 11GB, ClickHouse 500MB
- rco8786 1y ago> I recently did a test and for the same table with same exact data, MySQL was using 11GB, ClickHouse 500MB That's pretty impressive
- reconnecting 1y agoThe actual makes sense. I thought you had a PHP application that you just decided to rewrite to Go.
- hodgesrm 1y ago> MySQL was using 11GB, ClickHouse 500MB That's pretty typical best case size for weblogs and other time ordered data where column data correlate with time values. You do have to tweak the schema a bit to get there. (Specifically a "good" sort order, codecs, ZSTD instead of LZ4 compression, etc.)