7 ms·
This is all lovely and I wish them the best. But please don't use ollama, or their quants. Not only is the app itself slower than pure llamacpp. But their quan
by intothemild 2mo ago
This is all lovely and I wish them the best.
But please don't use ollama, or their quants. Not only is the app itself slower than pure llamacpp. But their quants are often no where near the best.
I really hope people start with something like unsloth, as their software and quants are really much better all around.
- CharlesW 2mo agoFor anyone who may not be aware it exists: https://unsloth.ai/docs/new/studio https://unsloth.ai/docs/new/studio
- unsnap_biceps 2mo agoI really wish they would figure out a better way to ship it for Mac. Last time I tried it a few months ago, it spewed a bunch of stuff into the system's pip archive and broke other software.
- deleted 2mo ago[deleted]
- fwipsy 2mo agoUnsloth has better quants, but when I tried their studio a couple months back it seemed like a buggy mess. Ollama seemed very basic but low-frustration.