5 ms·
https://arxiv.org/abs/2606.00206 https://arxiv.org/abs/2606.00206 In this paper they nerf an LLMs ability to emit waffling thinking tokens like "wait", "but",
by robertkarl 3mo ago
https://arxiv.org/abs/2606.00206 https://arxiv.org/abs/2606.00206
In this paper they nerf an LLMs ability to emit waffling thinking tokens like "wait", "but", "alternatively", and the models (they're old, small models in the paper) terminate reasoning faster and perform better. I bet Anthropic is tuning this on their backend.
- meatmanek 3mo agoThis is super cool. Do you know if any of the inference backends (llama.cpp, vllm, etc) support this technique?
- iaw 3mo agovLLM supports "banning" certain tokens but I don't know if it can dynamically reduce them. To my knowledge you can also "ban" with llama.cpp but it is passed in the API call rather than to the server at initialization.
- orbital-decay 3mo agoI imagine Anthropic would rather train a small control model instead of resorting to sampling hacks
- addandsubtract 3mo agoDidn't they originally introduce those tokens to make the models smarter by second guessing their "thoughts"?