jchandra

6 ms·

1.
by jchandra 5mo ago
Yeah, that’s consistent. topK keeps the obvious tokens, but subtle context gets eroded over time rather than dropped all at once.
2.
by jchandra 5mo ago
Fair point, the gap isn’t huge in that plot, and both degrade at low ratios. The difference is more in how they degrade: TopK can have sharper, localized failures, while HAE tends to be a bit more smooth. That doesn’t always show up strong
3.
by jchandra 5mo ago
Thanks, really appreciate the pointer. Will dig into it.
4.
by jchandra 5mo ago
Haha, that’s a very fair reading :) Yeah, the latency hit is definitely real. That said, most of what I’ve run so far is CPU-bound, which likely exaggerates it quite a bit so I didn’t want to draw strong conclusions from that. Would need pr
5.
by jchandra 5mo ago
I completely agree.Right now this is all on a synthetic setup to isolate the behavior and understand the reconstruction vs memory tradeoff. Real models will definitely behave differently. I’ve started trying this out with actual models, but
6.
by jchandra 5mo ago
That’s a great point and yeah, I’d agree SVD itself isn’t new at all. On downsides: definitely a few. The biggest one is latency - SVD is fairly heavy, so even though it’s amortized (runs periodically, not per token), it still adds noticeab
7.
by jchandra 5mo ago
In this prototype, OLS + SVD isn’t per-token, it runs only when the recycle bin fills (amortized over multiple tokens). That said, it’s still heavier than Top-K. I haven’t benchmarked end-to-end latency yet; this is mainly exploring the acc
8.
by jchandra 5mo ago
I’ve been exploring KV cache optimization for LLM inference. Most methods (Top-K, sliding window) prune tokens. This works on average, but fails selectively — a few tokens cause large errors when removed. I tried reframing the problem as ap
9.
High-Fidelity KV Cache Summarization Using Entropy and Low-Rank Reconstruction(jchandra.com)
64 points by jchandra 5mo ago|17 comments
10.
by jchandra 1y ago
Totally fair point — at the end of the day, it's all about getting the best model performance. I was mostly trying to highlight how, under the hood, a lot of modern HPO algos really boil down to smart scheduling decisions.
11.
Hyperparameter Tuning Is a Resource Scheduling Problem(jchandra.com)
2 points by jchandra 1y ago|3 comments
12.
by jchandra 1y ago
Pickle still is good for custom objects (JSON loses methods and also order), Graphs & circular refs (JSON breaks), Functions & lambdas (Essential for ML & distributed systems) and is provided out of box.
13.
AI Supply Chain Attack: How Malicious Pickle Files Backdoor Models(jchandra.com)
4 points by jchandra 1y ago|7 comments
14.
by jchandra 2y ago
pytorch save/load still are pickle based models. Its fine for trusted sources but when you start using from untrusted sources then there is always a risk of ACE. If you want to execute it, would suggest to try it in a sandbox env like
15.
by jchandra 2y ago
joblib is not fully secure because it still relies on Pickle internally. The reason it is slightly better in pickle is due to fact that pickle file gets immediately executed when it gets imported whereas joblib doesn’t execute code just by
16.
How Pickle Files Backdoor AI Models(jchandra.com)
6 points by jchandra 2y ago|6 comments
17.
by jchandra 2y ago
2
18.
by jchandra 2y ago
our approach wasn’t about over-engineering, we were trying to leverage our existing investments (like Confluent BYOC) while optimizing for flexibility, cost, and performance. We wanted to stay loosely coupled to adapt to cloud restrictions
19.
by jchandra 2y ago
We did have a discussion on Self vs Managed and TCOs associated with it. 1> We have multi regional setup so it came up with Data Sovereignty requirements. 2> Vendor Lock ins - Few of the services were not available in that geographic
20.
by jchandra 2y ago
As for BigQuery, while it's a great tool, we faced challenges with high-volume, small queries where costs became unpredictable as it is priced per data volume scanned. Clustered tables, Materialised views helped to some extent, but the
21.
We built a modern data stack from scratch and reduced our bill by 70%(jchandra.com)
83 points by jchandra 2y ago|59 comments
22.
Building a Modern Data Stack: Our 70% Cost Reduction with Open Source(jchandra.com)
3 points by jchandra 2y ago|0 comments
23.
Anti-Vaxxer on Linux mailing list(lkml.iu.edu)
2 points by jchandra 5y ago|0 comments
24.
by jchandra 5y ago
https://www.greenhouse.io/ down as well.
25.
by jchandra 6y ago
IMO, i dont think terraform is the right tool for containerized services. I had experimented with terraform and ansible for deployments earlier but i could see simpler deployments using serverless or apex. Informative article though.
26.
Pentagon Planned for a Bitcoin Rebellion(forbes.com)
87 points by jchandra 6y ago|90 comments
27.
Ask HN: What are the dont's in a tech interview
3 points by jchandra 6y ago|2 comments
28.
Barnum Effect(en.wikipedia.org)
2 points by jchandra 6y ago|0 comments
29.
Show HN: Go Anthem(youtu.be)
2 points by jchandra 6y ago|0 comments