7 ms·

The part about taking control of a reasoning model's output length using <think></think> tags is interesting. > In s1, when the LLM tries to stop thinking with

The part about taking control of a reasoning model's output length using <think></think> tags is interesting. > In s1, when the LLM tries to stop thinking with "</think>", they force it to keep going by replacing it with "Wait". I had found a few days ago that this let you 'inject' your own CoT and jailbreak it easier. Maybe these are related? https://pastebin.com/G8Zzn0Lw https://pastebin.com/G8Zzn0Lw https://news.ycombinator.com/item?id=42891042#42896498 https://news.ycombinator.com/item?id=42891042#42896498

causal 2y ago
This even points to a reason why OpenAI hides the "thinking" step: it would be too obvious that the context is being manipulated to induce more thinking.
zamalek 2y ago
It's weird that you need to do that at all, couldn't you just reject that token and use the next most probable?