7 ms·
Isn't that what "Attention is all you need" was about anyway? Does not sound like news to me.
by guhcampos 2mo ago
Isn't that what "Attention is all you need" was about anyway? Does not sound like news to me.
- FeepingCreature 2mo agoThis is all obvious and not news, but there's a lot of people doing fighting retreats against LLM intelligence where the degree of obviousness matters.
- XenophileJKO 2mo agoAnticipatory token selection was obvious when you used structured output on gpt 3.5. The model would construct/hallucinate pre-conditions to satisfy some final output value that the model was predisposed to.
- bunderbunder 2mo agoIt possibly goes a little further, though, since we’re talking about multi-step agentic loops here. So in addition to next token prediction there are of iterations of collecting input from outside sources. I’m still hesitant to interpret this as “thinking ahead” without at least seeing some more back-and-forth in the literature first, though. This just seems like one of those spots where it makes sense to give other researchers some time to come up with additional hypotheses to explain the observations instead of focusing on the first one anyone proposes in isolation.