7 ms·

I ran into the same looping issue with that model.

• danielhanchen 1y ago
  Definitely give temperature = 1.0, top_k = 64, top_p = 0.95, min_p = 0.0 a try, and maybe repeat_penalty = 1.1