6 ms·

You'll find many case studies about using Clickhouse for this purpose.

• hipadev23 2y ago
  Do you know any specific case studies for unstructured logs on clickhouse? I think achieving sub-second read latency of adhoc text searching over ~150B rows of unstructured data is going to be quite challenging without a high cost. Clickhouse’s inverted indices are still experimental. If the data can be organized in a way that is conducive to the searching itself, or structured it into columns, that’s definitely possible. Otherwise I suppose a large number of CPUs (150-300) to split the job and just brute force each search?
  • buzer 2y ago
    There is at least https://news.ycombinator.com/item?id=40936947 https://news.ycombinator.com/item?id=40936947 though it's a bit of mixed in terms how they handle schema.
    • SSLy 2y ago
      not sure if an excellent joke or a honest mistake
      • buzer 2y ago
        Let's go with former, I definitely didn't mean to link https://www.uber.com/en-FI/blog/logging/ https://www.uber.com/en-FI/blog/logging/ :)