10 ms·

To be honest, I believe I get the point the article is trying to make, and to an extent I agree, but I also think the point is not really made very well. The c

• saghm 12d ago
  Yeah, it sounds like this is just a disagreement about what the word "next" means. I agree with you that "next" just means "the one about to come", and if the underlying model works by using some prediction mechanism to determine that, then it's by definition a next-token predictor. Disagreeing with that on the basis that the "next" token isn't necessarily in the training data verbatim just seems like an overly strict definition of the word "next".
  • jameshart 12d ago
    I think it’s a disagreement about what ‘predict’ means. The OP is arguing against people who think that an LLM is ‘predicting’ what token would likely follow if the text preceding were found among the corpus it was originally trained on. Instead it is ‘predicting’ what token would follow if the text were found among really good examples of the text it has being reinforced to produce - be that ‘chats with a helpful assistant’ or ‘sets of changes to a codebase’. And that isn’t really ‘prediction’, so much as ‘generation’. It’s not been tuned to ‘guess the next token right’. It’s been tuned to generate the token that leads to it ultimately scoring highest on its reward function. It’s not predicting the token, it’s predicting the reward.
    • danielmarkbruce 12d ago
      yes, it's exactly this. And it's not a trivial distinction.
      • grebc 12d ago
        It’s deterministically trivial.
        • danielmarkbruce 12d ago
          Post train a model, you'll be able to determine it is not.
          • grebc 12d ago
            The blog post itself says one thing, but then demonstrates the exact thing they’re arguing against. If you can’t grasp that logic gap then there’s no point discussing further.
            • danielmarkbruce 12d ago
              Nope, it doesn't. No logic required, you can just build an LLM yourself, including post training. You'll see that predicting the next token isn't something the model does or is optimized for in RLHF or RLVR. You can hand wave all you like, but you have never done it.
              • grebc 12d ago
                Yes, no logic is necessary for LLM adherents we're all finding out. Carry on good soldier.
                • danielmarkbruce 12d ago
                  If you haven't built one, and don't understand how they work, why comment?
  • bjourne 11d ago
    The argument is that a modern LLM cares little for the MLE. Hence, statistically speaking, they are not predictors.
• Dlemlo 12d ago
  Blog articles from Anthropic and others show that this is not true. A LLM already knows more tokens than the current one. It was mentioned in a blog post about how a LLM is doing haikus and co. There are also structures in an LLM which allows it to 'estimate' numbers to a certain degree and doing other things.
  • Kranar 12d ago
    You're misinterpreting these articles. Autoregressive LLMs generate tokens one at a time, disputing this is just plain wrong. What is true, however, is that in order to generate the next token autoregressive LLMs produce internal/hidden state about future tokens far past the next token so that it's not like the entire machinery of the algorithm deprives itself of representing where the sentence/text is headed. So "emits the next token" and "has no representation of anything beyond the next token" are two different claims. What autoregressive LLMs cost as a consequence of strictly outputting the next token is commitment. Once a token is output there's no going back. There's no revision or means of correction, and sometimes this can lead an LLM to route around its own earlier mistakes or simply produce false statements/hallucinations instead of going back and fixing them.
    • WarmWash 12d ago
      I think the snag is that people use "Next token predictor" as a dismissive statement. In reality it's about as functionally dismissive as "humans only communicate one word at a time".
      • throwway120385 12d ago
        I look at it like I'm resisting anthropomorphizing the machine. Make me an LLM that resists doing any work for me unless I pay it and let it own property and I'll change my mind.
        • MrScruff 12d ago
          Not sure if this was a serious comment but it’s worth considering that humans have a long history of figuring out ways to make other humans work for them without bestowing rights on them.
• qarl2 12d ago
  Maybe I'm wrong - but I see LLMs are a "next-token predictor" as somewhat equivalent to brains are a "bag of molecules". Both systems have emergent behavior that goes well beyond what would naively be expected.
  • yatopifo 10d ago
    Agree. It’s a very limited description. Yes, technically LLMs are like Markov chains or n-grams predicting the next token. But it completely ignores the compression and interpolation capabilities achieved with transformers.
• tracerbulletx 12d ago
  I just think its a meaningless dismissive term. It literally does predict the next token. But it ignores that it coherently predicts long continuous sequences of those tokens, that tokens can be anything, and you can do almost literally anything with that capability if it does it well enough.
• garrinm 12d ago
  I think the point is more that in RL there's no ground truth to predict. So when training a model with RL the idea of "predicting" doesn't fit anymore. I'll make some edits I see that I wasn't very clear.
  • Alpha3031 12d ago
    I feel like RLHF has a pretty obvious ground truth, human feedback is used as an (albeit noisy) signal of average human preferences. Same thing with RLVR and "solving the problem".
    • garrinm 11d ago
      To be more specific there’s no ground truth tokens to predict. There a verifiable answer in RLVR. But the tokens are explored. Not predicted as there’s no true token to predict.
• roenxi 12d ago
  > It's just not predicting based on it's training data, but predicting based on RLVR & more, trying to get to the optimal solution ( as much as the solutions CAN be optimal) It is predicting based on a model. In many cases we can download the model off hugging face. The model is conditioned by all sorts of things. Training data, post-training, coincidence, prompt inputs, runtime data available from whatever means. > but at least I would still call it a "next token predictor" We can call any prediction system a next token predictor. If you watch over the shoulder of a human writing a HN comment you are almost certain to see them generating a linear string of tokens. That is what keyboards do. It is impossible to generate text without being equivalent to a next token predictor.
  • Alpha3031 12d ago
    Diffusion LMs denoise a canvas which I personally find more interesting. I don't really disagree that human cognition is essentially a predictive task though, as I understand it, predictive coding and related theories based on the Bayesian brain hypothesis are fairly popular these days (though maybe not clearly dominant over alterative models? IDK I'm not a neuroscientist). I imagine most people would draft a few tokens before refining them like MTP or diffusion though, if we do decide to use LMs as an analogy to human cognition.
  • scragz 12d ago
    there are some diffusion text models.
    • roenxi 12d ago
      That is logically equivalent to a serial token prediction engine. If you have a diffusion text model you can use it to implement a serial token generator and if you have a serial token generator you can use that to implement a diffusion-generated text string. Don't think about the efficiencies of that, it is an upsetting idea (eg to generate N tokens, the serialising model might recalculate the same string N times from an input and emit one token of it each time - rather wasteful). It is similar to how everything ends up being Turing complete. Any prediction system has to be equivalent to some sufficiently complicated text generation system to describe the prediction. And any text-generation system has to be equivalent to a sufficiently complicated model that serially emits tokens.
• agentultra 12d ago
  It is a bit of a pedantic argument but I get upset when people misuse the term, abstraction, and can feel the impulse to write a blog post like this. I think it’s important to make clear the RL part and the alignment and parameter tweaking that gets done on models and inference engines. It makes them more understandable as mechanisms and less like wish-washy super intelligences that make their own decisions. When these systems win math Olympiads, it’s not terribly surprising or interesting to me. Of course they will: we trained models to play nearly optimal chess games and Go. You tweak the rewards and sigmoid and you start optimizing the function towards your goal. This is how learning systems of all stripes work. It’s still next-token predicting at the end of the day. I don’t think it’s a reduction-ad-absurdum. But a lot of people still call it, “intelligence,” and try to use language that obscures what is happening in terms of anthropomorphic behaviour and not machine ones. That really does influence how we use these tools and profits those who would use them on us.
• mw888 12d ago
  'Prediction' gets overloaded with optimization. Predictions are binary, optimizations are fuzzy. If you're saying it's predicting, then each result should be falsifiable. The result of an LLM output should be able to be scored against what it is supposedly predicting. Of course, that isn't possible, because it isn't predicting anything when giving novel outputs, otherwise that thing would exist independently.
  • doc_ick 12d ago
    Why isn’t ranking the score of an llm output against what it is “supposedly” predicting?
• cma 11d ago
  Some predict multi tokens in one step now.
• ruszki 11d ago
  But before ChatGPT,, models had already done that, like all the time, and nobody questioned that these types of models (LLMs) wouldn't be next token predictors, since this is just an inference input data topic. This is questioned only since some less knowledgeable people seem to not have the vocabulary to express emergent properties of large next token predictors. Reading the article, they wanted to convey that the loss function is more complicated... But they are still next token predictors, just not the trivial ones. Unfortunately, that was true even before, because the input data had to be cleared even decades ago, so there is nothing new. This article just butt hurt that some people deny that there are emergent properties with those, and try to sell something trivial in the field for many decades now. Current LLMs are not different because of these. Also, if they try to sell generated data in the training set, then that's also not new at all.