4 ms·
LLMs learn a distribution during pre-training, not only an average. Then, by giving them context or by post-training, you can make them sample non-average part
by rytill 2mo ago
LLMs learn a distribution during pre-training, not only an average.
Then, by giving them context or by post-training, you can make them sample non-average parts of the distribution they learned.
- OtomotO 2mo ago> Then, by giving them context or by post-training, you can make them sample non-average parts of the distribution they learned. How do you derive that something is "below average" or "average" or "above average"?
- rytill 2mo agoWell, it’s up to the user or post-trainer of the LLM what they believe to be above average. Then they can design around that. In the case of real world LLMs and post-training, what is above average is defined roughly as: labeled good by expert humans, and scoring high on RL environments related to coding like debugging, passing tests, or running efficiently and verifiably correctly.
- nextaccountic 2mo ago> How do you derive that something is "below average" or "average" or "above average"? One technique is RLHF: have an human expert assess it.
- OtomotO 2mo agoMhm, I just wonder how many samples they get and how much time they have to come to the conclusion. Like a short example is easier to grade, but not in the same ballpark as a whole codebase.
- u8080 2mo ago>How do you derive that something is "below average" or "average" or "above average"? How do you? I mean, that was your point basis.