7 ms·
To have a gpu inference, you need a gpu. I have a demo that runs 8B llama on any computer with 4 gigs of ram https://galqiwi.github.io/aqlm-rs/about.html https
by om8 1y ago
To have a gpu inference, you need a gpu. I have a demo that runs 8B llama on any computer with 4 gigs of ram
https://galqiwi.github.io/aqlm-rs/about.html https://galqiwi.github.io/aqlm-rs/about.html