15 ms·
I was a fan of coheres general purpose LLM. Command A I think? Before they came out with their reasoning model. More competition is better.
by moojacob 3mo ago
I was a fan of coheres general purpose LLM. Command A I think? Before they came out with their reasoning model.
More competition is better.
- SubiculumCode 3mo agoI always forget the VRAM requirements on these MOE things
- sipjca 3mo agofwiw because of the relatively few activated params offloading to system RAM is quite feasible, you can see the endless amount of people doing this on r/localllama with qwen3.6 35a3b
- bitwize 3mo agoI ran Gemma4 26B A4B on an 8yo PC with a fucking GTX and it did rather well.
- doodlesdev 3mo agoWell, that's pretty impressive. Care to share your setup to do that? How much DDR3/DDR4 do you have, too?
- bitwize 3mo agoI... downloaded a 4-bit quantized GGUF of the model, used llama.cpp to run it, and pointed OpenCode at that. My machine is an 8-core Gen1 Ryzen 7, 32 GiB of DDR4, (I think) 4 GiB of VRAM on the graphics.
- doodlesdev 3mo agoWell... That's much simpler than I imagined LOL, thank you for answering!