8 ms·
It's possible to run with a cpu and then use system memory.
by AeiumNE 3y ago
It's possible to run with a cpu and then use system memory.
- alfor 3y agoDo you know what kind of speed you can expect with inference on CPU?
- MacsHeadroom 3y agoYes, around 2 tokens per second on a newer CPU with fast RAM.