5 ms·
For running llms _locally_ on Android, there's "pocketpal" (~7tok/s on a pixel 7 pro for some quant of llama 3.2 3B). (Not sure if it uses ollama though)
by seb314 2y ago
For running llms _locally_ on Android, there's "pocketpal" (~7tok/s on a pixel 7 pro for some quant of llama 3.2 3B).
(Not sure if it uses ollama though)