5 ms·
This isn’t accurate - most of the style comes from the fine tuning and reinforcement learning, not from the original training data. At some point people got th
by gooodvibes 11mo ago
This isn’t accurate - most of the style comes from the fine tuning and reinforcement learning, not from the original training data.
At some point people got this idea that LLMs just repeat or imitate their training data, and that’s completely false for today’s models.
- bediger4000 11mo agoSo LLMs have gotten creativity recently?
- gooodvibes 11mo agoNo, my point has nothing to do with creativity. It's about the fact that their output is taylored to look and sound in a certain way in the later stages of model training, it's not representative of the original text data the base model was trained on.
- idonotknowwhy 11mo agoAgreed. The pre-2025 base models don't write like this.
- incomingpain 11mo ago>This isn’t accurate - most of the style comes from the fine tuning and reinforcement learning, not from the original training data. Fine tuning, reinforcement, etc are all 'training' in my books. Perhaps this is your confusion over 'people got this idea'
- gooodvibes 11mo ago> Fine tuning, reinforcement, etc are all 'training' in my books. They are but they have nothing to do with how frequent anything is in literature which was your main point.