7 ms·
For a bit of comparison, if you've tested, how fast are 13B or 7B on the same setup?
by dandongus 3y ago
For a bit of comparison, if you've tested, how fast are 13B or 7B on the same setup?
- lolinder 3y agoReally fast. I didn't bother timing, but they're faster than ChatGPT by a long shot. I didn't spend very long with them because the quality is so much worse than the 65B. I should probably go back and try again to see if it's worth it for the extra speed, now that I've played with 65B for a while.