6 ms·
This is extremely useful and seems like what I need to fix all of my structured output woes, when my model gets chatty for no reason. The issue poorly glosses
by protosam 2y ago
This is extremely useful and seems like what I need to fix all of my structured output woes, when my model gets chatty for no reason.
The issue poorly glosses over explaining what feature they are talking about, here's a link to the docs about it: https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md https://github.com/ggerganov/llama.cpp/blob/master/grammars/...