7 ms·
But even so, the human picks the prompts and only publishes the AI outputs they think read nicely. There is information gain even in that.
by jbenjoseph 4y ago
But even so, the human picks the prompts and only publishes the AI outputs they think read nicely. There is information gain even in that.
- throwanem 4y agoAt the moment that's probably true, but is it guaranteed to remain so?
- jbenjoseph 4y agoEven so, I don't think there is any evidence that LLM performance degrades when it is trained on its own output, and there is no intuitive reason it should.
- throwanem 4y agoWhy not? Training a model destroys information.
- jbenjoseph 4y agoI have seen no evidence for that, only the opposite: https://arxiv.org/abs/2210.11610 https://arxiv.org/abs/2210.11610 Intuitively, training a simple enough linear statistical model with its own output should be a NOP. But LLMs are anything but simple models, so I think the non-linearities may be synthesizing new useful information. Similarly to how all of maths can be synthesized from a few basic axioms with enough intelligence or computation.