5 ms·
What's the inference time without gpu?
by petulla 3y ago
What's the inference time without gpu?
- lm2s 3y agoIt might the time mentioned at the bottom of the page since the author isn't sure that the GPU is being used: >How to speed this up—right now my Llama prompts often take 20+ seconds to complete.