super_ar

5 ms·

1.
Why autonomous agents still can't run on their own(glassflow.ai)
1 points by super_ar 13d ago|0 comments
2.
by super_ar 5mo ago
Good question. I wouldn’t say this replaces Flink in general. If you already run Flink and are comfortable with it, it’s a very powerful system. Where we saw friction with Flink was mainly: 1.) Operational overhead (jobs, state backends, ch
3.
Show HN: 500k+ events/sec transformations for ClickHouse ingestion(github.com)
13 points by super_ar 5mo ago|4 comments
4.
by super_ar 6mo ago
I am seeing this pattern a lot lately. Teams start with a simple flow: logs/metrics → Vector → ClickHouse Works well as long as they run simple transformations via Vector. When they start adding things like dedupe, longer time windows,
5.
by super_ar 9mo ago
Really cool!
6.
by super_ar 9mo ago
Looks cool! Do you have any idea who "good" it is at detecting AI-generated text?
7.
by super_ar 9mo ago
This is interesting. Just wondering about your traffic volume and how long you have been running lcoalpdf? For us, it is more like 5% of the traffic from GEO, but we have been running the company for 2 years and have created a lot of handwr
8.
by super_ar 1y ago
Yes, same machine.
9.
by super_ar 1y ago
There is another test that we published on our docs page. You can check it out here: Setup: https://docs.glassflow.dev/load-test/setup Results: https://docs.glassflow.dev/load-test/results
10.
by super_ar 1y ago
Fair point. Thanks for calling it out! To clarify, we’re focused on a specific use case: Kafka to ClickHouse pipelines with exactly-once guarantees. Kafka can’t provide exactly-once out of the box when writing to external systems like Click
11.
by super_ar 1y ago
Totally fair point. For stable, known workloads, you can get really far with something lightweight on a single machine. The challenge comes when you need fault tolerance, scaling, and delivery guarantees without constantly jumping in to fix
12.
Load Test GlassFlow for ClickHouse: Real-Time Dedup at Scale(glassflow.dev)
22 points by super_ar 1y ago|10 comments
13.
by super_ar 1y ago
Hi HN, A few weeks ago, we shared GlassFlow: Open Source streaming ETL to dedup and join streams from Kafka for ClickHouse ( https://news.ycombinator.com/item?id=43953722 ). One of the top questions we received was: “How well
14.
by super_ar 1y ago
Great question! RMT can work well when eventual consistency is acceptable and real-time accuracy isn't critical. But in use cases where results need to be correct immediately (dashboards, alerts, monitoring, etc.), waiting on backgroun
15.
by super_ar 1y ago
Thanks for asking those questions. Duplicates often come from how systems interact with Kafka, not from Kafka itself. For example, if a service retries sending a message after a timeout or if you collect similar data from multiple sources (
16.
by super_ar 1y ago
Great to hear that you are considering it for zenskar. We don't have a publicly available load test, but in internal checks it was able to handle 15k requests per second (locally on a MacBook Pro/M2 Docker). What is the load that
17.
by super_ar 1y ago
Good question! RMT does deduplication, but its dependency on background merges that you can't control can lead to incorrect results in queries until the merge is complete. We wanted something that cleans the duplicates in real time. Gl
18.
by super_ar 1y ago
It is a combination of both. We have a fantastic product designer colleague who takes care of the product, and a few friends who designed the website. I will forward your message to them. I am sure you've made their day. Thank you! :)
19.
by super_ar 1y ago
Thanks, Sai! Great question. The deduplication works based on the user-defined key, not the entire row. You can specify which field (e.g. a primary key like event_id) to use as the deduplication key. Within a defined time window, GlassFlow
20.
Show HN: GlassFlow – OSS streaming dedup and joins from Kafka to ClickHouse(github.com)
78 points by super_ar 1y ago|32 comments
21.
ClickHouse Denormalization is not the answer to slow JOINs(glassflow.dev)
18 points by super_ar 1y ago|5 comments
22.
Limits of ClickHouse ReplacingMergeTree and Materialized Views for Data Streams(glassflow.dev)
3 points by super_ar 1y ago|0 comments
23.
Kafka to ClickHouse Ingestion Methods(glassflow.dev)
2 points by super_ar 1y ago|0 comments
24.
ClickHouse ReplacingMergeTree doesn't work for streaming data(glassflow.dev)
1 points by super_ar 1y ago|0 comments
25.
by super_ar 2y ago
It looks awesome! What are you guys using under the hood? I've lately seen a lot of companies building on top of ClickHouse.
26.
by super_ar 2y ago
Have seen this happening too often when my sister, brother, or wife were using the iPhone time limit for apps.