5 ms·I run the KV cache at Q8 even on that model. Is it not working well for you?by behohippy 2y agoI run the KV cache at Q8 even on that model. Is it not working well for you?