6 ms·
I've been trying the Q4_K_M version, and sometimes it gets stuck in a loop. Gemma 4 doesn’t have this issue.
by bwv848 5mo ago
I've been trying the Q4_K_M version, and sometimes it gets stuck in a loop. Gemma 4 doesn’t have this issue.
- yencabulator 5mo agoThis has happened before with quantizations and other backends (ones not used by the research lab). Give it a week, download latest versions of everything, and try again.
- Readerium 5mo agoperhaps increasing repitition_penalty might be helpful
- mobiuscog 5mo agoI'm having the same issues, the more I use it. The repetition penalty doesn't seem to help. I get some really amusing 'reflective' responses, but I think it needs a bit more cooking. Maybe I'll try another variant.