6 ms·

You can just about load it on a 32GB GPU in 16bit mode. Quantized versions here: https://huggingface.co/mayank31398/starcoder-GPTQ https://huggingface.co/maya

by enum 3y ago

You can just about load it on a 32GB GPU in 16bit mode. Quantized versions here: https://huggingface.co/mayank31398/starcoder-GPTQ https://huggingface.co/mayank31398/starcoder-GPTQ they will be benchmarked on humaneval and released soon—maybe tomorrow?