6 ms·
I agree more with Yann LeCunn's salty reply. Over long run, knowing how autoregressive language models work from scratch will be just one step in having foundat
by LarsDu88 23d ago
I agree more with Yann LeCunn's salty reply. Over long run, knowing how autoregressive language models work from scratch will be just one step in having foundational understanding, and they might become dated... like knowing how a CRT monitor work. Something of historical interest and good for learning, but not crucial to being well-rounded.
There are other types of models like diffusion models right now that are showing more efficiency and have a higher ceiling for improvement. Understanding math and fundamentals are more important.
- lern_too_spel 23d agoYann has been consistently wrong about the limits of LLMs.
- LarsDu88 23d agoHe has, but I think he's right in the sense that there will be better models. It would be premature to think that the current frontier model architectures will last into the future. It would be like making DVD engineering a standard part of high school curricula
- handoflixue 22d ago> It would be like making DVD engineering a standard part of high school curricula I fail to see how PG suggesting that a 17 year old, today, should study something... is in any way a parallel to formalizing it as part of the curriculum. I would, in fact, expect PG to recommend something completely different in 10 years. The whole point of hacking is to understand the world around you, today
- carbondrone 22d agoWell good for them, I think their comment is heavily suggesting they're dissatisfied if said limits, and want to see them broken.
- aaron695 22d ago[dead]