5 ms·
it's not a binary thing. At some point Apple becomes too slow with very large models. If you can just run a model at 1 token per second and it takes 30 mins to
by ekianjo 4d ago
it's not a binary thing. At some point Apple becomes too slow with very large models. If you can just run a model at 1 token per second and it takes 30 mins to process a long context, it's useless