5 ms·

I've operated petabyte-scale ClickHouse clusters for 5 years

243 points by adastral 11d ago

lucrbvi 7d ago
That's a lot of ®, curious how ClickHouse® Inc. is treating the use of its name by others ... Hopes it's not like Oracle with JavaScript
- doe88 7d ago
  It's defensive language for sure, i don't know how much it adds of protection in reality, but i nonethelesss sympathize with the author if he feels the need to protect himself that way or signaling the risk he takes.
- HatchedLake721 7d ago
  They sell managed ClickHouse so I suspect it’s a precaution
  - gcharbonnier 7d ago
    From the article: > Let me clarify something: we created Tinybird ClickHouse because we wanted to build an analytics application without all the pain I'm describing. We do not offer ClickHouse® hosting; we solve the analytics problem EDIT: I'm wrong, their home page clearly states: "Ship fast over a Managed ClickHouse®", though I don't really understand the difference between a managed clickhouse and clickhouse hosting...
    - beberlei 6d ago
      Maybe you bring the hardware from any hoster, and they manage the service.
- mritchie712 7d ago
  > Tinybird is not affiliated with, associated with, or sponsored by ClickHouse, Inc. ClickHouse® is a registered trademark of ClickHouse, Inc. Yeah, it sucks they need to do this. If I was a visitor to their website, I'd immediately want to know what ClickHouse, Inc. is and you'd realize ---> it's managed clickhouse, direct competitor... why would I use the one that needs all the ®'s
- fusl 6d ago
  It's in fact two ®'s too many: even the "they don't like it" link to the GitHub issue has them.
walthamstow 7d ago
As an aside, I was stuck when turning on the Fulham v Crystal Palace game last week to find that Fulham have ClickHouse on their shirts this year, and Palace have Temporal AI. Talk about my worlds colliding.
bradleyy 7d ago
I just wish Amazon would offer it as an RDS DB; it'd make my life so much easier.
- andriy_koval 7d ago
  You can use actual CH Cloud on AWS?..
  - bradleyy 7d ago
    I'm aware; unfortunately that then leads to "must have a vendor approval" and a lot more process. If it were RDS, then it'd just be provision and done.
    - sdairs 7d ago
      Doesn't aws marketplace help with that?
      - kentm 7d ago
        No, because the issue is navigating legal agreements with your vendors, not the ease of deploying. With AWS Marketplace you are making a new legal agreement with the vendor of the product, not with Amazon.
    - bijowo1676 7d ago
      AWS acquired DuckLabs recently, there is a chance they will release a product competing with CH/Snowflake as a new RDS offering
      - viccis 6d ago
        They won't be able to resist the temptation of ruining it before launch by deciding to tie it into AWS Glue
    - justincormack 6d ago
      So how is clickhouse supposed to make money and continue to develop the product if you will only buy from AWS?
zbentley 7d ago
> For loads with over 20k rows/s and people pushing changes, you may need a full-time person to handle the cluster and take a look at the crazy queries people are going to write. I think this was a benefit of DBA culture in previous eras. Not that the DBAs were specifically necessary to write good queries (often they'd need to work with application teams to guide them towards schemas/behavior that worked well) or to maintain the database (managed DB offerings obsolete a lot of this work), but because they functioned as gatekeepers and rate-limiters of what queries and schemas could exist. In that mode, DBAs functioned a bit like a human/process version of a thin microservice wrapping database access functionality. A big benefit was that the rate of change of queries/schema changes/access patterns was controlled and had a higher probability of being reviewed and thought about by humans before it went live. This also resulted in an increased end-database-user culture of trying to make existing schemas/query patterns work before jumping straight to bespoke access patterns. That culture's not what you want as e.g. a startup or pro-rapid-big-refactors shop, but it is what you want when your DB reliability needs or query rate/dataset size are high. I don't think it's a given that a gatekeeper team is worth the overhead and cost; that's situational. I do think that the code version of that team (aforementioned microservice that wraps DB accesses/schema changes and nothing else) is usually not worth the cost. In my experience, that pretty much always reduces reliability and free performance gains that come from using direct DB clients from user code.
- bushbaba 7d ago
  Most startups can just scale your traditional separation of compute & storage here though. You’d be shocked how well duckdb against s3 scales for 99.9% of use cases
  - zbentley 7d ago
    It's one thing to be able to scale database compute/storage; it's another thing to be able to partition it. It's extremely common for bad queries/access patterns to cause noisy-neighbor effects on other simultaneous accesses to the database, to the extreme of knocking the whole database over with timeouts/OOMs/etc. Scaling out DB compute can only help with that to a (expensive) point; eventually, you end up wanting to either prevent the bad queries from being added to the system (DBA culture) or ensure that the bad query runs on database infrastructure that doesn't affect other queries. That's why partitioning DB compute (and storage: noisy-neighbor effects from a bad query at the storage layer don't require storage to be running e.g. a BookKeeper or whatever on a server; they can manifest as hot S3 keys or cloud object/block store rate limiting) is a necessary capability if your plan for dealing with a culture of "anyone can add any access pattern they want" is to scale the DB.
trynotsober 7d ago
When replaying customer queries against the next version, how do you compare results for queries using now() or approximate aggregates? Curious how you separate expected differences from actual regressions.
nicechianti 7d ago
[dead]
Lucasoato 7d ago
Where does ClickHouse fit between ElasticSearch, Pinot, TrinoDB, or just plain Spark? These are very different tools but I’m curious to know if any one has already compared them and can share some thoughts regarding their maintainability, QPS, latency, etc...
- antoniojtorres 7d ago
  I would put it closest to Pinot though pinot makes different architectural choices about storage as a difference that stands out to me. Then i’d rank trino as a second closest in only some ways. Spark the furthest for sure.
- hodgesrm 7d ago
  The relationship depends on the use case and is not linear. For web analytics and log management ClickHouse is a great replacement for ElasticSearch, for example.
- AtNightWeCode 7d ago
  ClickHouse I would say is more for analytics and data warehouse types of loads. So not a direct competitor to those tools except maybe for Pinot. It is very easy to ingest data into CH. We connected exchange topics and it just worked with zero code. But the thing with CH is that it is pretty much a Russian product so you should not use it for production anymore.
  - AntonFriberg 7d ago
    I would like to see some sources on your claim about it being Russian. It is incorporated in the US with most developers in Amsterdam. After the invasion of Ukraine they stayed silent for a while but that was because they needed to allow there developers to get out of Russia. Many of them are Ukrainians including the CTO and founder. As soon as it was safe for there team they took a very firm stance against Russia with Ukrainian flags on the website and written statement from the team. I am not aware of any Russian influence currently.
    - 7d ago
mrngm 7d ago
For those interested, there's also a second part [2025]: https://www.tinybird.co/blog/what-i-learned-operating-clickhouse-part-ii https://www.tinybird.co/blog/what-i-learned-operating-clickh... (note: the first part was originally published April 2025 according to the date tooltip)
yakkomajuri 7d ago
> "Every single company handling ClickHouse® struggles with ingestion." Very true. Reading about "too many parts" gave me flashbacks. (previously owned ingestion into CH at PostHog, no longer)
- fuziontech 7d ago
  Scaling during that era was fun. Slack was non-stop: :oof-1: CH needs more disk to keep up with merges. We ran CH way too lean in those days.
  - yakkomajuri 7d ago
    ofc we meet in the ClickHouse thread
8943gG4f 7d ago
[dead]
threecheese 7d ago
I’m operating a terabyte-scale Clickhouse - but only because I left Langfuse running for a few months on a MacBook :) But seriously Clickhouse does love disk space.
- cnkk 7d ago
  But on the same time it is super efficient with it compare to other solutions. Kind of efficient for application logs for example.
will_pseudonym 7d ago
At first when I read this, I was like "Why does The Onion's ClickHole site need so many servers?"
solatic 7d ago
I cocked an eyebrow more than once reading this. > A quick note about HTTP: ClickHouse® offers a TCP connector with a native protocol, but we don't use it. It does not offer many advantages for the type of application we build This needs more elaboration. One of the major goals of running a ClickHouse cluster is to provide low-latency queries; a persistent TCP connection removes the need to re-establish a new connection for each query and thus reduces overall latency in line with CH goals. So I really didn't understand this. > ClickHouse® open source faces a significant challenge: limited support for cloud storage. Modern OLAP databases and data systems should leverage cloud storage for cost efficiency and independent scaling of compute and storage resources. Snowflake established this standard over a decade ago, and ClickHouse® (open source) lags behind ClickHouse writing to NVMEs is exactly how they provide their latency and performance advtanges. Writing and reading to object buckets is fundamentally slower with multiple network hops to reach what is, in this architectural context, a storage server for your storage server. If you really need far more storage, and are willing to sacrifice query latency to get it... why not architect for one of the OLAP databases, like Snowflake, where that was part of their architecture from day one? > Because you are testing your analytics queries, right? No? Half the point of an OLAP database is to let users write their own queries. If we knew the queries ahead of time, we probably wouldn't need an OLAP database, and instead use a less-flexible streaming architecture storing intermediate calculations so as not to need to pay for petabyte-scale storage. The expected value from paying for all of that storage is to support not knowing which queries will be written by users. > Every single company handling ClickHouse® struggles with ingestion... Backpressure mechanism: Some people put Kafka before ClickHouse®. This does the job The whole trade-off that you make with column-store databases like ClickHouse (instead of row-store databases like Postgres) is that inserts are slow for column stores (whereas they are fast for row stores). Inserts happen slowly, asynchronously, in the background. It is the price you pay for fast analytics queries. This is why OLAP databases have a latency lag and do not show real-time results. This is why stores like Kafka are usually a good fit, you let Kafka hold onto new data until batch insertions can catch up. If you do need real-time queries, you don't write to an OLAP directly; you write to a stateful frontend that answers the query itself, then streams out historical data from the OLAP that was successfully written there. And the first thing you do in a "I want to have my cake, and eat it too, and yes I'm willing to pay for the privilege" architecture like that is... to keep the persistent TCP connections, because that's really low-hanging fruit.
anguss 7d ago
A couple weeks ago I was scanning Github for repos with frequent commits to identify so-called "software factories" and was surprised to see clickhouse. I wouldn't touch it with a 10ft pole considering how quickly they're merging code into main. I'm talking 50+ commits per day and thousands of AI generated issues and triages. Check it out for yourself https://github.com/clickhouse/clickhouse https://github.com/clickhouse/clickhouse
- ryan_lane 7d ago
  Just wait till you find out how many commits per day to a product are occurring to most of the SaaS you rely on. Clickhouse has a managed SaaS and it's their primary product. They have a lot of engineers. They're going to do a lot of commits.
- ggcr 6d ago
  The CTO himself reported 600 commits and 300 PRs each day. And they can do so because they have a massive CI [1]: > Every day CI runs about 20..80 million tests in 600 commits and 300 pull requests > Last year, ClickHouse spent 360 years of machine time for CI I am no user of CH so I can't talk about their product. But we are talking about a company with 686 employees as per their LinkedIn, where ClickHouse is clearly the core of their business. Considering all of this, is 50+ commits a day that much? [1] https://presentations.clickhouse.com/2026-openhouse-sf/great/?full#31 https://presentations.clickhouse.com/2026-openhouse-sf/great...
dev_l1x_be 7d ago
Complex systems have complex operations and problems. I think isolating the read and write workloads and separating the read queries by departments or even users is a distinct possibility especially with technologies like DuckDB.
- a34729t 7d ago
  That is exactly where all the high scale stuff like Trino and StarRocks have gone (and ClickHouse can use S3 as storage too). However, the problem you run into is when you want to index stuff- secondary indices are tightly coupled to the query engine, so in practice you arent going to use StarRocks to write and index data to the object store, and then use ClickHouse or Trino to query it. I think it would be a useful development to decouple writes and reads, but this requires a common indexing scheme, and then once you have this standard, in principle lots more indexing plugins can be written and all compatible query engines can use them. Apache DataFusion is probably the right framework to build off of.
  - dev_l1x_be 6d ago
    Valid. We need to innovate on this and re-think secondary indices. Storage is cheap.
lee_ars 7d ago
My dude, you only need to put the registered trademark symbol after the first occurrence of the mark. You® don't® need® to® put® it® after® every® mention® of® Click®House®™™®™®™®™®™.
- javisantana 6d ago
  it's kind of an internal joke, ClickHouse Inc lawyers are a pain in the ass with their letters and every time you forget a (r) somewhere, they write.
Perepiska 7d ago
Some entity spoiled the link "they don't like it" by adding (R), correct link is [1] 1. https://github.com/ClickHouse/ClickHouse/issues/49383#issuecomment-1532379446 https://github.com/ClickHouse/ClickHouse/issues/49383#issuec...
alexnewman 7d ago
About 20 years ago the titans of the database fields told us they wanted to put ML in the brain of databases. I think agentic databases are the future.
shin_lao 7d ago
[flagged]
pjot 7d ago
I don’t think it’s necessarily the use case changed but maybe the constraints. Streaming directly to Clickhouse is a great solution. Recent approaches I’ve seen though buffer data as parquet in object storage first, and read data from there. Im not sure if CH would be the preference
wackget 6d ago
Yeah that's cool but your blog won't load on my hardened Firefox: - Uncaught TypeError: navigator.sendBeacon is not a function - Uncaught (in promise) ReferenceError: WebAssembly is not defined Not sure why a blog entry needs WebAssembly but I'll give it a pass.
- phoghed 6d ago
  Sometimes I think you guys run your browser like this just to make these performative comments
wackget 6d ago
What kind of systems are generating enough useful analytics data to ever scale to petabytes in the first place? It's difficult (for my bird brain, at least) to imagine a scenario where such volume of analytics data would ever be necessary.
- x0x0 6d ago
  It's multiple records per user over time. See something like posthog. One interaction with a form can generate 10-15 events. eg enter page, click link with text "settings", change input, clicked input, etc. So active users can generate 15-150 events per session, each ~1kb to 2kb in size. The reason to store that data is to be able to build analytics and funnels you didn't anticipate and construct them retrospectively. You can of course save massive amounts of storage by building rollups and aggregating, but that prevents you from being able to eg change the funnels and see the funnel in the past. 10M users × 100 events/day × 1KB ≈ 1TB/day. A couple years gets you into PB range. The other obvious thing is logs. Being able to have a vulnerability then go back in time and see if you were exploited is super valuable. See eg log4shell: if you kept records, you probably could go back and see if you'd been exploited.
- razes 6d ago
  they are working with CDNs and others companies like Vercel, localstack or canva and those companies generate a lot of (really a lot) data. In fact, they have solutions teams to check with the clients what they are storing and what they really need to store, how to consume it, etc.
swyx 6d ago
note to author - whatever you do to add the registered symbol also is screwing up your link to https://github.com/ClickHouse%C2%AE/ClickHouse%C2%AE/issues/49383#issuecomment-1532379446 https://github.com/ClickHouse%C2%AE/ClickHouse%C2%AE/issues/... grep for "they don't like it" and click on that link
boringstack 6d ago
[dead]
halilBB 6d ago
[flagged]
AaronNewcomer 6d ago
I have used tinybird for production fraud monitoring and customer facing analytics. Javi and the team are great to work with. I have also used regular clickhouse cloud. I initially tried tinybird out just to rule out clickhouse as an option for my workload when I was planning on going with Apache Druid but then found out for my workloads they blew it out of the water. So I definitely recommend them and the whole clickhouse ecosystem in general for any kind of high throughput read and write analytics work.

243 points by adastral 11d ago

That's a lot of ®, curious how ClickHouse® Inc. is treating the use of its name by others ... Hopes it's not like Oracle with JavaScript

doe88 7d ago
It's defensive language for sure, i don't know how much it adds of protection in reality, but i nonethelesss sympathize with the author if he feels the need to protect himself that way or signaling the risk he takes.
HatchedLake721 7d ago
They sell managed ClickHouse so I suspect it’s a precaution
- gcharbonnier 7d ago
  From the article: > Let me clarify something: we created Tinybird ClickHouse because we wanted to build an analytics application without all the pain I'm describing. We do not offer ClickHouse® hosting; we solve the analytics problem EDIT: I'm wrong, their home page clearly states: "Ship fast over a Managed ClickHouse®", though I don't really understand the difference between a managed clickhouse and clickhouse hosting...
  - beberlei 6d ago
    Maybe you bring the hardware from any hoster, and they manage the service.
mritchie712 7d ago
> Tinybird is not affiliated with, associated with, or sponsored by ClickHouse, Inc. ClickHouse® is a registered trademark of ClickHouse, Inc. Yeah, it sucks they need to do this. If I was a visitor to their website, I'd immediately want to know what ClickHouse, Inc. is and you'd realize ---> it's managed clickhouse, direct competitor... why would I use the one that needs all the ®'s
fusl 6d ago
It's in fact two ®'s too many: even the "they don't like it" link to the GitHub issue has them.

walthamstow 7d ago

As an aside, I was stuck when turning on the Fulham v Crystal Palace game last week to find that Fulham have ClickHouse on their shirts this year, and Palace have Temporal AI. Talk about my worlds colliding.

bradleyy 7d ago

I just wish Amazon would offer it as an RDS DB; it'd make my life so much easier.

andriy_koval 7d ago
You can use actual CH Cloud on AWS?..
- bradleyy 7d ago
  I'm aware; unfortunately that then leads to "must have a vendor approval" and a lot more process. If it were RDS, then it'd just be provision and done.
  - sdairs 7d ago
    Doesn't aws marketplace help with that?
    - kentm 7d ago
      No, because the issue is navigating legal agreements with your vendors, not the ease of deploying. With AWS Marketplace you are making a new legal agreement with the vendor of the product, not with Amazon.
  - bijowo1676 7d ago
    AWS acquired DuckLabs recently, there is a chance they will release a product competing with CH/Snowflake as a new RDS offering
    - viccis 6d ago
      They won't be able to resist the temptation of ruining it before launch by deciding to tie it into AWS Glue
  - justincormack 6d ago
    So how is clickhouse supposed to make money and continue to develop the product if you will only buy from AWS?

zbentley 7d ago

> For loads with over 20k rows/s and people pushing changes, you may need a full-time person to handle the cluster and take a look at the crazy queries people are going to write. I think this was a benefit of DBA culture in previous eras. Not that the DBAs were specifically necessary to write good queries (often they'd need to work with application teams to guide them towards schemas/behavior that worked well) or to maintain the database (managed DB offerings obsolete a lot of this work), but because they functioned as gatekeepers and rate-limiters of what queries and schemas could exist. In that mode, DBAs functioned a bit like a human/process version of a thin microservice wrapping database access functionality. A big benefit was that the rate of change of queries/schema changes/access patterns was controlled and had a higher probability of being reviewed and thought about by humans before it went live. This also resulted in an increased end-database-user culture of trying to make existing schemas/query patterns work before jumping straight to bespoke access patterns. That culture's not what you want as e.g. a startup or pro-rapid-big-refactors shop, but it is what you want when your DB reliability needs or query rate/dataset size are high. I don't think it's a given that a gatekeeper team is worth the overhead and cost; that's situational. I do think that the code version of that team (aforementioned microservice that wraps DB accesses/schema changes and nothing else) is usually not worth the cost. In my experience, that pretty much always reduces reliability and free performance gains that come from using direct DB clients from user code.

bushbaba 7d ago
Most startups can just scale your traditional separation of compute & storage here though. You’d be shocked how well duckdb against s3 scales for 99.9% of use cases
- zbentley 7d ago
  It's one thing to be able to scale database compute/storage; it's another thing to be able to partition it. It's extremely common for bad queries/access patterns to cause noisy-neighbor effects on other simultaneous accesses to the database, to the extreme of knocking the whole database over with timeouts/OOMs/etc. Scaling out DB compute can only help with that to a (expensive) point; eventually, you end up wanting to either prevent the bad queries from being added to the system (DBA culture) or ensure that the bad query runs on database infrastructure that doesn't affect other queries. That's why partitioning DB compute (and storage: noisy-neighbor effects from a bad query at the storage layer don't require storage to be running e.g. a BookKeeper or whatever on a server; they can manifest as hot S3 keys or cloud object/block store rate limiting) is a necessary capability if your plan for dealing with a culture of "anyone can add any access pattern they want" is to scale the DB.

trynotsober 7d ago

When replaying customer queries against the next version, how do you compare results for queries using now() or approximate aggregates? Curious how you separate expected differences from actual regressions.

nicechianti 7d ago

[dead]

Lucasoato 7d ago

Where does ClickHouse fit between ElasticSearch, Pinot, TrinoDB, or just plain Spark? These are very different tools but I’m curious to know if any one has already compared them and can share some thoughts regarding their maintainability, QPS, latency, etc...

antoniojtorres 7d ago
I would put it closest to Pinot though pinot makes different architectural choices about storage as a difference that stands out to me. Then i’d rank trino as a second closest in only some ways. Spark the furthest for sure.
hodgesrm 7d ago
The relationship depends on the use case and is not linear. For web analytics and log management ClickHouse is a great replacement for ElasticSearch, for example.
AtNightWeCode 7d ago
ClickHouse I would say is more for analytics and data warehouse types of loads. So not a direct competitor to those tools except maybe for Pinot. It is very easy to ingest data into CH. We connected exchange topics and it just worked with zero code. But the thing with CH is that it is pretty much a Russian product so you should not use it for production anymore.
- AntonFriberg 7d ago
  I would like to see some sources on your claim about it being Russian. It is incorporated in the US with most developers in Amsterdam. After the invasion of Ukraine they stayed silent for a while but that was because they needed to allow there developers to get out of Russia. Many of them are Ukrainians including the CTO and founder. As soon as it was safe for there team they took a very firm stance against Russia with Ukrainian flags on the website and written statement from the team. I am not aware of any Russian influence currently.
  - 7d ago

mrngm 7d ago

For those interested, there's also a second part [2025]: https://www.tinybird.co/blog/what-i-learned-operating-clickhouse-part-ii https://www.tinybird.co/blog/what-i-learned-operating-clickh... (note: the first part was originally published April 2025 according to the date tooltip)

yakkomajuri 7d ago

> "Every single company handling ClickHouse® struggles with ingestion." Very true. Reading about "too many parts" gave me flashbacks. (previously owned ingestion into CH at PostHog, no longer)

fuziontech 7d ago
Scaling during that era was fun. Slack was non-stop: :oof-1: CH needs more disk to keep up with merges. We ran CH way too lean in those days.
- yakkomajuri 7d ago
  ofc we meet in the ClickHouse thread

8943gG4f 7d ago

[dead]

threecheese 7d ago

I’m operating a terabyte-scale Clickhouse - but only because I left Langfuse running for a few months on a MacBook :) But seriously Clickhouse does love disk space.

cnkk 7d ago
But on the same time it is super efficient with it compare to other solutions. Kind of efficient for application logs for example.

will_pseudonym 7d ago

At first when I read this, I was like "Why does The Onion's ClickHole site need so many servers?"

solatic 7d ago

I cocked an eyebrow more than once reading this. > A quick note about HTTP: ClickHouse® offers a TCP connector with a native protocol, but we don't use it. It does not offer many advantages for the type of application we build This needs more elaboration. One of the major goals of running a ClickHouse cluster is to provide low-latency queries; a persistent TCP connection removes the need to re-establish a new connection for each query and thus reduces overall latency in line with CH goals. So I really didn't understand this. > ClickHouse® open source faces a significant challenge: limited support for cloud storage. Modern OLAP databases and data systems should leverage cloud storage for cost efficiency and independent scaling of compute and storage resources. Snowflake established this standard over a decade ago, and ClickHouse® (open source) lags behind ClickHouse writing to NVMEs is exactly how they provide their latency and performance advtanges. Writing and reading to object buckets is fundamentally slower with multiple network hops to reach what is, in this architectural context, a storage server for your storage server. If you really need far more storage, and are willing to sacrifice query latency to get it... why not architect for one of the OLAP databases, like Snowflake, where that was part of their architecture from day one? > Because you are testing your analytics queries, right? No? Half the point of an OLAP database is to let users write their own queries. If we knew the queries ahead of time, we probably wouldn't need an OLAP database, and instead use a less-flexible streaming architecture storing intermediate calculations so as not to need to pay for petabyte-scale storage. The expected value from paying for all of that storage is to support not knowing which queries will be written by users. > Every single company handling ClickHouse® struggles with ingestion... Backpressure mechanism: Some people put Kafka before ClickHouse®. This does the job The whole trade-off that you make with column-store databases like ClickHouse (instead of row-store databases like Postgres) is that inserts are slow for column stores (whereas they are fast for row stores). Inserts happen slowly, asynchronously, in the background. It is the price you pay for fast analytics queries. This is why OLAP databases have a latency lag and do not show real-time results. This is why stores like Kafka are usually a good fit, you let Kafka hold onto new data until batch insertions can catch up. If you do need real-time queries, you don't write to an OLAP directly; you write to a stateful frontend that answers the query itself, then streams out historical data from the OLAP that was successfully written there. And the first thing you do in a "I want to have my cake, and eat it too, and yes I'm willing to pay for the privilege" architecture like that is... to keep the persistent TCP connections, because that's really low-hanging fruit.

anguss 7d ago

A couple weeks ago I was scanning Github for repos with frequent commits to identify so-called "software factories" and was surprised to see clickhouse. I wouldn't touch it with a 10ft pole considering how quickly they're merging code into main. I'm talking 50+ commits per day and thousands of AI generated issues and triages. Check it out for yourself https://github.com/clickhouse/clickhouse https://github.com/clickhouse/clickhouse

ryan_lane 7d ago
Just wait till you find out how many commits per day to a product are occurring to most of the SaaS you rely on. Clickhouse has a managed SaaS and it's their primary product. They have a lot of engineers. They're going to do a lot of commits.
ggcr 6d ago
The CTO himself reported 600 commits and 300 PRs each day. And they can do so because they have a massive CI [1]: > Every day CI runs about 20..80 million tests in 600 commits and 300 pull requests > Last year, ClickHouse spent 360 years of machine time for CI I am no user of CH so I can't talk about their product. But we are talking about a company with 686 employees as per their LinkedIn, where ClickHouse is clearly the core of their business. Considering all of this, is 50+ commits a day that much? [1] https://presentations.clickhouse.com/2026-openhouse-sf/great/?full#31 https://presentations.clickhouse.com/2026-openhouse-sf/great...

dev_l1x_be 7d ago

Complex systems have complex operations and problems. I think isolating the read and write workloads and separating the read queries by departments or even users is a distinct possibility especially with technologies like DuckDB.

a34729t 7d ago
That is exactly where all the high scale stuff like Trino and StarRocks have gone (and ClickHouse can use S3 as storage too). However, the problem you run into is when you want to index stuff- secondary indices are tightly coupled to the query engine, so in practice you arent going to use StarRocks to write and index data to the object store, and then use ClickHouse or Trino to query it. I think it would be a useful development to decouple writes and reads, but this requires a common indexing scheme, and then once you have this standard, in principle lots more indexing plugins can be written and all compatible query engines can use them. Apache DataFusion is probably the right framework to build off of.
- dev_l1x_be 6d ago
  Valid. We need to innovate on this and re-think secondary indices. Storage is cheap.

lee_ars 7d ago

My dude, you only need to put the registered trademark symbol after the first occurrence of the mark. You® don't® need® to® put® it® after® every® mention® of® Click®House®™™®™®™®™®™.

javisantana 6d ago
it's kind of an internal joke, ClickHouse Inc lawyers are a pain in the ass with their letters and every time you forget a (r) somewhere, they write.

Perepiska 7d ago

Some entity spoiled the link "they don't like it" by adding (R), correct link is [1] 1. https://github.com/ClickHouse/ClickHouse/issues/49383#issuecomment-1532379446 https://github.com/ClickHouse/ClickHouse/issues/49383#issuec...

alexnewman 7d ago

About 20 years ago the titans of the database fields told us they wanted to put ML in the brain of databases. I think agentic databases are the future.

shin_lao 7d ago

[flagged]

pjot 7d ago

I don’t think it’s necessarily the use case changed but maybe the constraints. Streaming directly to Clickhouse is a great solution. Recent approaches I’ve seen though buffer data as parquet in object storage first, and read data from there. Im not sure if CH would be the preference

wackget 6d ago

Yeah that's cool but your blog won't load on my hardened Firefox: - Uncaught TypeError: navigator.sendBeacon is not a function - Uncaught (in promise) ReferenceError: WebAssembly is not defined Not sure why a blog entry needs WebAssembly but I'll give it a pass.

phoghed 6d ago
Sometimes I think you guys run your browser like this just to make these performative comments

wackget 6d ago

What kind of systems are generating enough useful analytics data to ever scale to petabytes in the first place? It's difficult (for my bird brain, at least) to imagine a scenario where such volume of analytics data would ever be necessary.

x0x0 6d ago
It's multiple records per user over time. See something like posthog. One interaction with a form can generate 10-15 events. eg enter page, click link with text "settings", change input, clicked input, etc. So active users can generate 15-150 events per session, each ~1kb to 2kb in size. The reason to store that data is to be able to build analytics and funnels you didn't anticipate and construct them retrospectively. You can of course save massive amounts of storage by building rollups and aggregating, but that prevents you from being able to eg change the funnels and see the funnel in the past. 10M users × 100 events/day × 1KB ≈ 1TB/day. A couple years gets you into PB range. The other obvious thing is logs. Being able to have a vulnerability then go back in time and see if you were exploited is super valuable. See eg log4shell: if you kept records, you probably could go back and see if you'd been exploited.
razes 6d ago
they are working with CDNs and others companies like Vercel, localstack or canva and those companies generate a lot of (really a lot) data. In fact, they have solutions teams to check with the clients what they are storing and what they really need to store, how to consume it, etc.

swyx 6d ago

note to author - whatever you do to add the registered symbol also is screwing up your link to https://github.com/ClickHouse%C2%AE/ClickHouse%C2%AE/issues/49383#issuecomment-1532379446 https://github.com/ClickHouse%C2%AE/ClickHouse%C2%AE/issues/... grep for "they don't like it" and click on that link

boringstack 6d ago

[dead]

halilBB 6d ago

[flagged]

AaronNewcomer 6d ago

I have used tinybird for production fraud monitoring and customer facing analytics. Javi and the team are great to work with. I have also used regular clickhouse cloud. I initially tried tinybird out just to rule out clickhouse as an option for my workload when I was planning on going with Apache Druid but then found out for my workloads they blew it out of the water. So I definitely recommend them and the whole clickhouse ecosystem in general for any kind of high throughput read and write analytics work.