8 ms·
I was going to point that out. What I will add is that constrained generation is supported by the major inference engine like llama.cpp, vllm and the likes, so
by lelag 1y ago
I was going to point that out.
What I will add is that constrained generation is supported by the major inference engine like llama.cpp, vllm and the likes, so what you are describing is actually trivial on locally hosted models, you just have to provide a regex that prevent them to use the letter 'e' in the output.
- Der_Einzige 1y agoYou can do this more properly with the antislop sampler and we are working on a follow up paper to our previous work on this exact problem. https://github.com/sam-paech/antislop-sampler https://github.com/sam-paech/antislop-sampler https://arxiv.org/abs/2306.15926 https://arxiv.org/abs/2306.15926