ople

5 ms·

1.
by ople 9mo ago
"Point at any ClickHouse table – no schema migrations/no OTEL requirement" I haven't tried it yet but I definitely will because of this. Pretty much every other CH observability product is opinionated about their schemas
2.
by ople 1y ago
This is a cool and underrated new feature: It basically enables simple joins of large and dynamically updated tables efficiently. This is something that has been a bit of an Achilles heel of ClickHouse, at least for me so it's excitin
3.
by ople 1y ago
Why not just use SQL? With LLMs evolving to do sophisticated text-to-SQL, the case for a custom language for the sake of simplicity is diminishing. I think that expressiveness, performance and level of fluency by base language models (i.e.
4.
by ople 2y ago
This is one of the things I appreciate about ClickHouse: Contributors willing and able to dive in to performance tune even low-level core stuff like this and getting merged quickly and efficiently, even if they are not affiliated with Click
5.
by ople 2y ago
True. My bet is that arbitrary text queries converted to SQL via LLM will become an increasingly popular alternative. Not sure if it will completely replace the custom query languages though..
6.
by ople 2y ago
Recently there seems to be an bunch of SPL (Splunk) -like query languages popping up: PQL, PRQL, Grafana Explore Logs syntax, Kusto.. Probably others as well. Does yet another similar but slightly different language make sense? Why not leve
7.
by ople 2y ago
Very interesting observations! Merge performance tuning seems often overlooked even though it's a key aspect of sustained ClickHouse performance. I also like that the blog is quite compact and gets the points across without getting too
8.
ClickHouse vs. Elasticsearch: The Billion-Row Matchup(clickhouse.com)
1 points by ople 2y ago|0 comments
9.
Cold brew coffee in 3 minutes using acoustic cavitation(unsw.edu.au)
537 points by ople 2y ago|351 comments
10.
by ople 3y ago
I have had the same experience. I was constantly bumping into unexpected limitations. Moving to CH felt like the opposite, with many more ”Wow, I didn’t expect this to be possible but it is” experiences. There is a place for BQ but it is go
11.
by ople 3y ago
I have pretty much exactly the same experience. However, I do feel that they are trying to really do the right thing with the new 3.0 architecture, addressing the deficiencies (most importantly performance and full-fledged SQL) while keepin
12.
by ople 4y ago
Unsurprisingly IBM pioneered this with "Capacity on Demand" https://www.ibm.com/docs/en/power9?topic=environment-capacit... I'm guessing it has had some popularity among customers as it has been aro
13.
by ople 10y ago
Hehe.. In retrospect the whole team was in fairly good spirit although the situation was stressful. A lot of this was due to the top management giving the time and space for the specialists to do their thing and the very understanding respo
14.
by ople 10y ago
The reading of the metadata required quite a lot of random acces. We were fairly sure that if a high-end array and controller with fast disks is struggling with it, then a traditional clustered solution with slower node local disks would no
15.
by ople 10y ago
We copied the raw image file of the corrupted metadata filesystem (MDT in Lustre lingo) to the ramdisk. Then we mounted it via loopback and copied the files to tarballs. The bit that was really slow on the spinning disk was reading the mill
16.
by ople 10y ago
The metadata target (MDT) in the MDS is actually "ldiskfs" which is an enhanced version of ext4. One possibility may be to use ZFS in the future as the support in Lustre seems to be quite stable now. It seems pretty impossible to
17.
by ople 10y ago
It was filesystem-level corruption in Lustre. The underlying disk arrays and other hardware have comprehensive redundancy.
18.
by ople 10y ago
Author here. We have had an automated deletion policy on our previous filesystems but opted out this time: There are users that have temporary files that they want to persist on the /wrk and we have plenty of capacity. We definitely le
19.
by ople 10y ago
Author here: We considered that but as the access pattern was likely pretty much random, the performance would have been terrible. Due to the break we had nearly a 1000 clustered servers sitting idle so it was reasonably quick to do the ra