9 ms·
> Generally, the models are compressing their understanding of all text, and in doing so, they're learning high order concepts Are these higher order concepts
by zodzedzi 3y ago
> Generally, the models are compressing their understanding of all text, and in doing so, they're learning high order concepts
Are these higher order concepts accessible to us? E.g. can we list those learned concepts?
(Re-reading the paper you linked now...)
- tikkun 3y agoMy understanding is that the answer is generally: not yet. (I wish, I suspect we'll be able to learn some interesting things about the universe, about humans, and so on, by seeing what LLMs found to be highly explanatory / high order concepts)