7 ms·
> We built chproxy specifically because ClickHouse doesn't like thousands of tiny inserts. It's a Go service that buffers events and sends them in large batches
by jmg_ 11mo ago
> We built chproxy specifically because ClickHouse doesn't like thousands of tiny inserts. It's a Go service that buffers events and sends them in large batches. Each Cloudflare Worker would send individual analytics events to chproxy, which would then aggregate and send them to ClickHouse.
While I understand how this isn't the only thing that needed to be buffered, for Clickhouse data specifically I'd be curious why they built a separate service rather than use asynchronous inserts:
https://clickhouse.com/docs/optimize/asynchronous-inserts https://clickhouse.com/docs/optimize/asynchronous-inserts