amitport

6 ms·

1.
by amitport 1mo ago
I've been considering something like this for a while. It only makes sense as a free, open-source, decentralized sharing protocol (where anyone can host theorems and no one can limit sharing them). If a company were to manage to comme
2.
by amitport 3mo ago
C# did not ship with async/await, and Java didn't have virtual threads back then. I am specifically referring to the initial choices made in C#'s foundation.
3.
by amitport 3mo ago
The mystery of why .NET got so many things right is simply that C# was built several years later by the exact same Microsoft engineers who had previously worked on extending Java, giving them a perfect blank slate to fix the architectural f
4.
by amitport 3mo ago
Let’s look at the actual historical record, because the pattern is undeniable: the root cause of this conflict is the absolute refusal of the regional and local leadership to ever accept a sovereign Jewish state within any borders. In 1937
5.
by amitport 3mo ago
"parents and grandparents," You don't have to go into historical events. This is still happening now. Jews are still fighting for their survival and the moment Israel stops fighting, millions of Jews will die.
6.
by amitport 3mo ago
To be fair, NPM sucked long before it got acquired by Github/Microsoft. And to be fair 2: The other package repos also suck.
7.
by amitport 4mo ago
Hi, thanks! I appreciate your input and generally agree. The TDS article wasn't really aimed at the HN crowd, but it did help a bit with the more general audience. I do plan to also develop an interactive guide that breaks down post-ro
8.
by amitport 5mo ago
For context: https://towardsdatascience.com/how-a-2021-quantization-algor...
9.
Show HN: A 4-year-old "TurboQuant" implementation(github.com)
3 points by amitport 5mo ago|3 comments
10.
A Note on TurboQuant and the Earlier Eden Work(arxiv.org)
2 points by amitport 5mo ago|0 comments
11.
by amitport 5mo ago
I recently wrote a beginner-friendly explanation of this situation in TDS: https://towardsdatascience.com/how-a-2021-quantization-algor...
12.
by amitport 5mo ago
Hi, I’m the author of the post above. tl;dr TurboQuant is a recent paper from Google and NYU that has gained massive traction in mainstream media and the AI community. As implementations of TurboQuant are integrated into various popular pro
13.
by amitport 5mo ago
In the vLLM documentation quoted above, TurboQuant (which is a restricted version of EDEN) is referred to as a specific case of HIGGS. Note the symmetry: EDEN acts as a special case of HIGGS; hence, HIGGS functions as a generalization of ED
14.
by amitport 5mo ago
Thanks for the pushback, and I appreciate the reference to classical information theory. While I probably overstated things by using the very general phrase "taking advantage," I want to be very precise about the claim, as I belie
15.
by amitport 5mo ago
Thanks for that! It is worth noting that taking advantage of the post-rotation distribution was not actually done until DRIVE (2021), which was made possible via our proper scaling. Furthermore, applying a Lloyd-Max codebook post-rotation
16.
by amitport 5mo ago
Thanks for that! Note that the residual chain is empirically and theoretically inferior to our unbiased scale; furthermore, it requires an additional bit in certain cases. Additionally, TurboQuant was not the first to apply EDEN to KV-cac
17.
by amitport 5mo ago
Those works did cite DRIVE/EDEN :) HIGGS is an extension of EDEN (using the well known method for blockwise Lloyd-Max). The proper framing of this "TurboQuant" layer in vllm (which does not include JQL) is precisely EDEN 22 w
18.
by amitport 5mo ago
I believe our claim at this point is more fundamental than just lack of citation. The quantizer in TurboQuant is EDEN quantization (2021) applied to the KV-cache. It is neither a novel quantizer nor an improvement in quantization techniqu
19.
by amitport 5mo ago
Thanks for the quick response and for being willing to update the explainer. I really appreciate the clarification.
20.
by amitport 5mo ago
When you use TurboQuant, you are essentially using the EDEN quantizer under a different name applied to KV-cache. Both EDEN and its 1-bit variant have been implemented in PyTorch, JAX, and TensorFlow across numerous open-source libraries an
21.
by amitport 5mo ago
The note includes extensive experiments and reproduces many of the figures from the TurboQuant paper in our Section 5. Honestly, I think our case is pretty clear-cut as is. I am not sure what the overhead for those specific benchmarks would
22.
by amitport 5mo ago
TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at
23.
by amitport 5mo ago
They must mean 'The Arab World Has Lost America,' don't they?
24.
by amitport 6mo ago
We do mention and the paper you shared. Please read our paper to see how the rotation-aware bias correction we introduced efficiently fixes the bias and provides a better worst-case error.
25.
by amitport 6mo ago
To be clear, I am not claiming they stole an idea. They have made significant independent research. However, a specific part regarding the treatment of rotation with bias correction relates to prior work, and it would be appropriate to have
26.
by amitport 6mo ago
In this context, the rotation is for spreading energy and ensuring predictable coordinate distributions rather than diagonalization; it makes coordinate-wise quantization much more computationally efficient, though it throws away learnable
27.
by amitport 6mo ago
Yes, the important part is that the normalized error does not increase with the dimension of the vector (which does happen when using biased quantizers) It is expected that bigger vectors have proportionally bigger error, nothing can be d
28.
by amitport 6mo ago
This is a great development for KV cache compression. I did notice a missing citation in the related works regarding the core mathematical mechanism, though. The foundational technique of applying a geometric rotation prior to extreme quant
29.
by amitport 6mo ago
r is a single value per vector. You don't have to quantize it, you can keep it and quantize the billion+ other coordinates of the vector.
30.
by amitport 9mo ago
His point, I believe, was that it is early in the innovation cycle and they very well be replaced quickely with different solutions/paradigms.

More ›