8 ms·
Does this support Qwen 3.6 (e.g. 27B) and the myriad of llama.cpp options (batch sizes, quantization, etc.)? I'd love to see some performance data.
by roger_ 2mo ago
Does this support Qwen 3.6 (e.g. 27B) and the myriad of llama.cpp options (batch sizes, quantization, etc.)?
I'd love to see some performance data.
- sig_kill 2mo agoYes! I’ve worked on the settings interface between our runtime and llamacpp, these are documented and available via our config.toml file