4 ms·

Emitting and predicting are different things though. Prediction implies there is some "truth" or event or something that you can test against. Prediction implie

Emitting and predicting are different things though. Prediction implies there is some "truth" or event or something that you can test against. Prediction implies the model just learns from existing text, and optimizes to predict the next token in training data. That's just not true.

dijksterhuis 12d ago
prediction is a very specific term of art in the field of machine learning. generally speaking, machine learning models like LLMs are based on probability; performing a statistical prediction of the likely y given some input x Probability(y | x) that's why we refer to outputs as a prediction. it is likelihoods and stuff. the output is never definitely correct as we're not dealing with heuristic processes. > Prediction implies there is some "truth" or event or something that you can test against there absolutely is a ground truth during training. the core predict-the-next-most-likely-token part of an LLM has a ground truth next-token. that's why you don't end up with generated text like: fish spurious send cattle chocolate phone happy meaning ball orange board canada. > optimizes to predict the next token in training data that is the optimization goal in training the next-most-likely-token core of an LLM, it basically translates to maximise the likelihood of predicting the next token x_i given the previous tokens L(θ) = −log Π^n_{i=1} f_θ(x_i | x1, ..., x_{i−1}) https://arxiv.org/pdf/2012.07805 https://arxiv.org/pdf/2012.07805 (GPT2 but the point still stands) (edit: sorry for the ADHD edits)
- danielmarkbruce 12d ago
  Read through the article and comments. You are talking solely about pre-training. I'm talking about post training. Respectfully, you are miles out of your depth. GPT-2 didn't use any reinforcement learning and is often given as a toy example. That release was 2019 and models now go through a various phases of training with different objective functions and optimizers.
  - deleted 12d ago
    [deleted]
  - dijksterhuis 12d ago
    From GP, i.e. the context for this local part of the thread > Autoregressive LLMs generate tokens one at a time, disputing this is just plain wrong. next-token prediction i.e. the bit built during pre-training. at no point in your reply to GP did you specify that you were referring to post-training. respectfully, it seems like this one is on you pal :shrug: > GPT-2 didn't use any reinforcement learning and is often given as a toy example. That release was 2019 and models now go through a various phases of training with different objective functions and optimizers. yeah. so? the toy example works for pre-training. see above.
    - danielmarkbruce 12d ago
      All modern LLMs that actually get used go through post-training. The finished product is something which has been through post training. So they are not next token prediction machines.
      - dijksterhuis 12d ago
        > The finished product is something which has been through post training. again, the finished product wasn't what was discussed by GP, and you didn't clarify that you were switching to discussing RL (which is still probabilistic btw)
        danielmarkbruce 12d ago
        Yes, it was. Nobody says a system or product works a certain way and means the system while it's half built. "Bridges drop cars in the water!". Right. You aren't in this field. You are clearly wrong and just can't handle it.
        dijksterhuis 12d ago

dijksterhuis 12d ago

prediction is a very specific term of art in the field of machine learning. generally speaking, machine learning models like LLMs are based on probability; performing a statistical prediction of the likely y given some input x Probability(y | x) that's why we refer to outputs as a prediction. it is likelihoods and stuff. the output is never definitely correct as we're not dealing with heuristic processes. > Prediction implies there is some "truth" or event or something that you can test against there absolutely is a ground truth during training. the core predict-the-next-most-likely-token part of an LLM has a ground truth next-token. that's why you don't end up with generated text like: fish spurious send cattle chocolate phone happy meaning ball orange board canada. > optimizes to predict the next token in training data that is the optimization goal in training the next-most-likely-token core of an LLM, it basically translates to maximise the likelihood of predicting the next token x_i given the previous tokens L(θ) = −log Π^n_{i=1} f_θ(x_i | x1, ..., x_{i−1}) https://arxiv.org/pdf/2012.07805 https://arxiv.org/pdf/2012.07805 (GPT2 but the point still stands) (edit: sorry for the ADHD edits)

danielmarkbruce 12d ago
Read through the article and comments. You are talking solely about pre-training. I'm talking about post training. Respectfully, you are miles out of your depth. GPT-2 didn't use any reinforcement learning and is often given as a toy example. That release was 2019 and models now go through a various phases of training with different objective functions and optimizers.
- deleted 12d ago
  [deleted]
- dijksterhuis 12d ago
  From GP, i.e. the context for this local part of the thread > Autoregressive LLMs generate tokens one at a time, disputing this is just plain wrong. next-token prediction i.e. the bit built during pre-training. at no point in your reply to GP did you specify that you were referring to post-training. respectfully, it seems like this one is on you pal :shrug: > GPT-2 didn't use any reinforcement learning and is often given as a toy example. That release was 2019 and models now go through a various phases of training with different objective functions and optimizers. yeah. so? the toy example works for pre-training. see above.
  - danielmarkbruce 12d ago
    All modern LLMs that actually get used go through post-training. The finished product is something which has been through post training. So they are not next token prediction machines.
    - dijksterhuis 12d ago
      > The finished product is something which has been through post training. again, the finished product wasn't what was discussed by GP, and you didn't clarify that you were switching to discussing RL (which is still probabilistic btw)
      - danielmarkbruce 12d ago
        Yes, it was. Nobody says a system or product works a certain way and means the system while it's half built. "Bridges drop cars in the water!". Right. You aren't in this field. You are clearly wrong and just can't handle it.
        dijksterhuis 12d ago