5 ms·
LLMs does have that, or at least it’s very likely that we will eventually be able to manipulate LLMs in a modular way (see https://news.ycombinator.com/item?id
by renonce 2y ago
LLMs does have that, or at least it’s very likely that we will eventually be able to manipulate LLMs in a modular way (see https://news.ycombinator.com/item?id=40429540 https://news.ycombinator.com/item?id=40429540). One point remains: humans learn language with much fewer tokens than LLMs need, which suggests presence of a priori knowledge about the world. The LLM metaphor is finetuning, so babies are born with a base model and then finetuned with environment data, but it’s still within LLM scope.
- tgv 2y ago> presence of a priori knowledge about the world 1. A certain architecture (e.g. a module that enables syntactic processing) is not knowledge about the world. 2. We model the world according to our capabilities. 3. Modular language models have been tried, but did not meet with success. 4. The link you include is about the conceptual space, which is not (directly) related to human syntactic processing. 5. The question is not about metaphors, but about reality. 6. Babies aren't born with a base model and fine-tuned. They learn. This is the metaphor NNs are actually based on.