6 ms·

I got one token every 8 minutes or so.

• 2Gkashmiri 4y ago
  Is that good? Not good?
  • Tepix 4y ago
    Usually you want tokens per second, not seconds per token. So it's a bad sign.
  • gorbypark 4y ago
    A token is approximately 4 characters. So, four characters per 8 minutes is pretty slow. This comment would take 1224 minutes to generate, if I was an AI.
• popol12 4y ago
  Using which model ? On a pretty mid range i5 11th gen I'm getting 0.35 token/s, using the 7B model. Haven't tried the bigger models.