vadman97

7 ms·

1.
Simple Data Isolation in ClickHouse(highlight.io)
2 points by vadman97 2y ago|0 comments
2.
by vadman97 2y ago
If there was only one alert criteria, that'd be simple. Our alerts can be configured for any data filters (eg. only matching logs with column `level='error'`); we would have to create a unique MV for each alerts' filter
3.
by vadman97 2y ago
We insert from our alerts worker because we want the aggregation to happen per alert (with the aggregated data filtered by the particular alert definition). As each alert is evaluated, we run the following [1] INSERT INTO ... SELECT ... sta
4.
Alert Evaluations: Incremental Merges in ClickHouse(highlight.io)
38 points by vadman97 2y ago|7 comments
5.
by vadman97 2y ago
Forget the exact details, but we ran into issues with tuning the two setting you're describing on the Cloud instance (these settings were user level settings that seemed to get reset by ClickHouse Cloud because they were updated based
6.
by vadman97 2y ago
Once we hit >100k inserts per second, async inserts didn't work well for us because we had limited control over the background async insert batching happening on the cluster. The background inserts would be too small, resulting in m
7.
by vadman97 2y ago
We do via the OpenTelemetry Elixir SDK (and the Phoenix extension) https://opentelemetry.io/docs/languages/erlang/
8.
by vadman97 2y ago
Materialized views can help change the ORDER BY with 0 downtime: * Create a new version of the table with the new ORDER BY. * Create a materialized view that will insert from the old table to the new table. * Update your application code to
9.
by vadman97 2y ago
Exploring bloom filter index merges would be an interesting addition. I do wish it were easier to profile merge performance to break down where most of the CPU time is being spent.
10.
by vadman97 2y ago
3 nodes of 120 GiB and 30 vCPU each.
11.
by vadman97 2y ago
I see what you did there.
12.
Optimizing ClickHouse MergeTree CPU Usage(highlight.io)
1 points by vadman97 2y ago|0 comments
13.
Migrating semi-structured search from OpenSearch to ClickHouse(highlight.io)
3 points by vadman97 3y ago|0 comments
14.
by vadman97 3y ago
Our browser client would help with tracing what network requests are being made by said apps since it would capture all network requests. However, Shopify may restrict what our browser client can do in your frontend store (I'm guessing
15.
Measuring the performance impact of session replay(highlight.io)
27 points by vadman97 3y ago|1 comments
16.
An open-source session replay benchmark(highlight.io)
3 points by vadman97 3y ago|1 comments
17.
by vadman97 3y ago
Hey HN, I'm Vadim, the CTO at highlight.io. We offer session replay to engineering teams and have had a lot of questions around client-side performance. We just published a benchmark along with a short blog post covering the results. I
18.
by vadman97 3y ago
Presumably, they mean millions of logs a day when saying "millions of events a day", of which some subset corresponds to errors.
19.
by vadman97 3y ago
Our errors ingest is quite affordable ( https://www.highlight.io/pricing ) and let's you filter out data you are not interested in ingesting. Our logs ingest is even cheaper because of the underlying ClickHouse storage w
20.
by vadman97 3y ago
Regarding model adaptation, we haven't yet explored a fine-tuned model, but it makes a lot of sense for a given class of errors. For a given code-base that is using highlight, the errors will typically be of a given language / inf
21.
by vadman97 3y ago
> Have you looked into finetuning the embedding model to your use case? Not yet, though this is definitely one of the next steps for us. The `gte-large` model we use is trained on a variety of text, but a hypothesis is one trained or fin
22.
Google launches earthquake alerts on Android in India(techcrunch.com)
2 points by vadman97 3y ago|0 comments
23.
Show HN: Using LLMs and Embeddings to classify application errors(github.com)
65 points by vadman97 3y ago|10 comments
24.
by vadman97 3y ago
Agreed, but the argument for causing damage still requires a human actor to execute GPT-4's advice. Should we be censoring LLMs just because we are offended by the output or find it misleading? There is plenty of misinformation on the
25.
OpenAI GPT-4V technical paper released detailing problems(techcrunch.com)
11 points by vadman97 3y ago|3 comments
26.
OpenTelemetry for Cloudflare Workers(github.com)
3 points by vadman97 3y ago|0 comments
27.
Amazon brings generative AI to Alexa(techcrunch.com)
5 points by vadman97 3y ago|1 comments
28.
Kamal: Yet another way to deploy your web apps (from the maker of Ruby on Rails)(kamal-deploy.org)
5 points by vadman97 3y ago|1 comments
29.
by vadman97 3y ago
ClickHouse Async insert docs [1]. We ran into some challenges with async inserts at highlight.io [2]. Namely, ClickHouse Cloud has an async flush size configured (that can't be changed AFAIK) that isn't large enough for our scale.
30.
by vadman97 3y ago
How do you think about the query syntax? Are you defining your own or are you following an existing specification? I particularly love the trace view you have, connecting a frontend HTTP request to server side function-level tracing.

More ›