8 ms·
Yeah q8 made so littler difference back when I was testing such things I'd be surprised if people could quickly notice that as a change. It's got to be either f
by NineStarPoint 9d ago
Yeah q8 made so littler difference back when I was testing such things I'd be surprised if people could quickly notice that as a change. It's got to be either further quantized or some other type of optimization that kicks in when people notice the drop.
- selectodude 9d agoNVFP4 would buy them a huge increase in capacity but I think it would be noticeable.
- Caracas288 9d agoWhy doesn't someone just try to measure this next time!?
- embedding-shape 9d agoCan't really measure without being sure you aren't being messed around with, when it's a remote platform. Stupidly easy to detect when people run such benchmarks/tests against you as well.