5 ms·
So does human content. Much of the original data that GPT was trained on is Reddit posts and web pages on the internet - stuff that isn't exactly known for bein
by ComplexSystems 1y ago
So does human content. Much of the original data that GPT was trained on is Reddit posts and web pages on the internet - stuff that isn't exactly known for being a source of high quality facts.
- lelanthran 1y agoThat's still very different to "drift error compounding". If you output a mere 5% drift error and then use that as input you only need a few cycles (single digits) before your output is more erroneous than correct. We are already partly into the second cycle. By the fifth the LLM would be mostly useless.