25 ms·
Worth trying on a CPU-only host as well as GPU boxes. A lot of LlamaRack users will hit memory-bound cases where a Seattle Xeon with decent RAM is enough for GG
by po_westnet26 10d ago
Worth trying on a CPU-only host as well as GPU boxes. A lot of LlamaRack users will hit memory-bound cases where a Seattle Xeon with decent RAM is enough for GGUF batch runs. If you want a short metered window without buying hardware, CPU hours around $19/hr: https://ai.westnet.ca/ https://ai.westnet.ca/