5 ms·

I think this line of reasoning is misguided. What’s striking and more important to focus on are the abstract reasoning abilities of these systems. Language, as

I think this line of reasoning is misguided. What’s striking and more important to focus on are the abstract reasoning abilities of these systems. Language, as you mentioned, abstracts real world objects and phenomena, so it’s a good approximation of the real world. Thus, if an LLM can reason this well using language, it’s safe to say that perhaps they’re doing something akin to what the human mind does. Your critique about lack of grounding in these systems is an easy problem to solve. It’s as easy as teaching an LLM to associate words with real world objects or phenomena. Image-classification models, text-2-image models, audio transcription models, and many other modal specific systems already do this to some extent. And more recently there has been a push towards multi-modal language models(Deepmind’s flamingo), so this line of argument will be debunked very soon. I actually believe GPT-4 will be multi-modal and it’s capabilities will dispel majority of these criticisms

lee101 4y ago
[flagged]
Syntonicles 4y ago
I agree. We've [potentially] given the grounding assimilation step a boost already, since language is already organized. Imagine that a language model is fully integrated with sense-data that exceeds human first-hand experience. Perhaps they are trained on and can generate realistic 3D models of objects, and derive estimates of their internal construction, weight, etc. Perhaps they recall infrared emissions or opacity to EM wavelengths. Would we truly "know" what we're talking about by that standard? I'm not actually sure why we don't consider generative image models to be grounded already. They seem to be able to modify, transform and rotate imagery. That indicates spatial understanding to me, and I'm not sure how much more we must require of them without having to exclude blind or otherwise disabled humans from our definition of comprehension.

lee101 4y ago

[flagged]

Syntonicles 4y ago

I agree. We've [potentially] given the grounding assimilation step a boost already, since language is already organized. Imagine that a language model is fully integrated with sense-data that exceeds human first-hand experience. Perhaps they are trained on and can generate realistic 3D models of objects, and derive estimates of their internal construction, weight, etc. Perhaps they recall infrared emissions or opacity to EM wavelengths. Would we truly "know" what we're talking about by that standard? I'm not actually sure why we don't consider generative image models to be grounded already. They seem to be able to modify, transform and rotate imagery. That indicates spatial understanding to me, and I'm not sure how much more we must require of them without having to exclude blind or otherwise disabled humans from our definition of comprehension.