Go see yourself :) https://github.com/ggerganov/llama.cpp/blob/master/examples/quantize/quantize.cpp https://github.com/ggerganov/llama.cpp/blob/master/example

Go see yourself :) https://github.com/ggerganov/llama.cpp/blob/master/examples/quantize/quantize.cpp https://github.com/ggerganov/llama.cpp/blob/master/examples/... There's too many schemes right now with 4_0 and 5_1 really popular between LLM geeks.