6 ms·
Hi, If you ever get to writing a blog post about kv cache quantisation, i'm interested in quantising K differently than V
by Ey7NFZ3P0nzAe 8d ago
Hi,
If you ever get to writing a blog post about kv cache quantisation, i'm interested in quantising K differently than V
- stared 7d agoIf you like, you can run these tests yourself as well. It is ~$500 per a single combination, and assuming no failed runs.
- Ey7NFZ3P0nzAe 5d agoI don't have anywhere near that kind of money to spend. Sorry.