5 ms·
Does anyone here understand "interleaved scratchpads" mentioned at the very bottom of the footnotes: > All evals were run with a 64K thinking budget, interleav
by sync 10mo ago
Does anyone here understand "interleaved scratchpads" mentioned at the very bottom of the footnotes:
> All evals were run with a 64K thinking budget, interleaved scratchpads, 200K context window, default effort (high), and default sampling settings (temperature, top_p).
I understand scratchpads (e.g. [0] Show Your Work: Scratchpads for Intermediate Computation with Language Models) but not sure about the "interleaved" part, a quick Kagi search did not lead to anything relevant other than Claude itself :)
[0] https://arxiv.org/abs/2112.00114 https://arxiv.org/abs/2112.00114
- dheerkt 10mo agobased on their past usage of "interleaved tool calling" it means that the tool can be used while the model is thinking. https://aws.amazon.com/blogs/opensource/using-strands-agents-with-claude-4-interleaved-thinking/ https://aws.amazon.com/blogs/opensource/using-strands-agents...
- davidsainez 10mo agoAFAICT, kimi k2 was the first to apply this technique [1]. I wonder if Anthropic came up with it independently or if they trained a model in 5 months after seeing kimi’s performance. 1: https://www.decodingdiscontinuity.com/p/open-source-inflection-point-kimi2-ai-competitive-dynamics https://www.decodingdiscontinuity.com/p/open-source-inflecti...
- BoorishBears 10mo agoOpenAI has been doing this since at least O3 in January, Anthropic has been doing it since 4 in May. And the July Kimi K2 release wasn't a thinking model, the model in that article was released less than 20 days ago.