8 ms·
I promise this Ollama replacement will give you the fastest t/s you've ever had
- wowitsbase 8d agoA small developer team and I had worked on this for a while for personal reasons, so I have decided to port it to cpp and release it to the general public. The stats are in the github. I promise if you have a gpu this will improve your speeds by at least 50%, even if you believe your configuration is optimized.
- deleted 8d ago[deleted]
- FloatArtifact 8d agoDoes it have Intel Arc support?
- wowitsbase 8d agoSomewhat, but most of the optimizations are cuda based. I would guess it would be a little faster, like 20-40%, but not 200% like on Nvidia.