4 ms·

I'm running the 5 bit quant on a 3060 12G. Get a handful of tokens/s. Prompts take 10-20 minutes to complete but it works.