6 ms·
AGI must be multimodal. If it can't understand images, video, sound, smells, tastes, it doesn't have a full understanding of the world.
by bufferoverflow 1y ago
AGI must be multimodal. If it can't understand images, video, sound, smells, tastes, it doesn't have a full understanding of the world.
- gabipurcaru 1y agomultimodality would be very useful, but on the other hand humans can't see infrared, and can't smell ~most things that other animals can
- deleted 1y ago[deleted]
- altruios 1y agoDoes it need a 'full' understanding? smell and taste is useful for biological life... but we don't need that in our thinking machines, do we? I agree AGI must be multimodal. I don't think that multimodal is 'set in place', nor must it be conveniently, human-centricly mapped from our senses.
- AnimalMuppet 1y agoI think a big part of intelligence is being able to correlate things. Well, the more modes, the more ability to correlate. For example, smell is a component of ER triage. Some different problems smell differently. And if I had a robot chef, but the chef couldn't actually taste... yeah, not sure I trust it very far as a chef.
- m3kw9 1y agoThe way we need AGI is it needs to match us, we are evolved to operate quite optimally with the constraints and given physics in this world
- deleted 1y ago[deleted]
- whatnow37373 1y agoI think AGI implies it: I can learn something through audio and apply it visually and the other wat around. I don’t think that’s some abstract human quirk. Isn’t that what enabled literacy? It seems kind of obvious intelligence is beneath the modality, agnostic about it.
- bastawhiz 1y agoIt absolutely tickles me to think AGI would be like "I think it's going to rain, I can smell the petrichor" or ask for its salsa to be free from cilantro because it tastes like soap. "Hey Siri, what does this taste like to you?" is such an absolutely unhinged interaction
- readthenotes1 1y agoMust it understand emojis? I don't understand most emojis. But I guess I never claimed to possess NGI
- enturbulated 1y agoAmong other things, The Fine Article argues that the current approach of gluing together various models of different modalities is, in the end, going fail to reach AGI. Better track to try to build a single model which processes multiple modalities all at once.
- genewitch 1y agoI wonder if it will end up being a game-like loop where it processes everything that's come in since the last delta. Here's, you know, 4x200samples of audio, 2 frames of video, and here's all the mems sensor data during the delta, etc Then you just work on getting the delta as small as possible, I assumed 5ms for audio, e.g.
- robotresearcher 1y agoBrains have very obvious mode-specialized chunks. That doesn’t mean that’s the only way to do it, but it’s an interesting fact.
- _Algernon_ 1y agoIs a blind person less intelligent than a person capable of seeing? If so, by how much? In my view, intelligence isn't about what senses you have available, but how intelligently you use the information you have available to you.
- andoando 1y agoI think human/animal intelligence at its basis is spatial-temporal. That is we can model and reason about events in space through time. Our senses I believe map to this spatial-temporal model. Blind people can reason about the world the same way as those who can see, because what were really doing is modeling space, and light, audio, touch etc are just ways of gaining information
- Dylan16807 1y agoThat sounds really hard to test. If someone fails entire categories of question but does better at the ones they can do and focused on, is that a good result or a bad result? When it comes to "understanding of the world", I'd say the average blind person has less. But the gaps in their understanding are generally not particularly important parts.
- _Algernon_ 1y ago>When it comes to "understanding of the world", I'd say the average blind person has less. But the gaps in their understanding are generally not particularly important parts. Is understanding of the world equivalent to intelligence though? In my view intelligence is about optimalising the mapping from the percept sequence to action. In other words, given a sequence of percepts, does it determine the utility maximizing action. Imagine two chess bots on uneven footing. One plays regular chess with perfect knowledge of the board. The other plays fog of war chess—it only sees the pieces on squares it attacks. In this case, the former could play suboptimally and still win against the latter. The latter can have perfect information about probabilities of pieces on tiles and act perfectly utility maximising in response and still lose. My argument is that the latter is still more intelligent despite losing. There is a difference between action and intelligence. Similarly a human doesn't become smarter or dumber by adding or removing senses. They may make smarter or dumber decisions, but that is purely attributable to the extra information available to them.
- dlivingston 1y agoWhy limit it to just human senses? Imagine an AI with all of the above... plus sensors for electromagnetic fields, non-visible light, non-audible sounds, hyper-sensitivity to air flow / pressure, to humidity... no idea how you would even being to train these things into multimodality, but so many "senses" would be emergent.
- bufferoverflow 1y agoI didn't limit anything. Read what I wrote. I agree, AIs should have all the possible sensors. We don't have much data for the non-human sensors in human environments though.
- Glyptodon 1y agoI'm out sure it needs to be connected to senses to be multimodal - it's not like blind people lack GI because of not seeing.
- bufferoverflow 1y agoAsk a blind person if some color scheme looks good. Ask them to color-grade a film. Ask them if the laser level shows level. Blind people, while definitely not lacking intelligence completely, have major gaps in their understanding of the world.
- solomonb 1y ago[flagged]
- jillesvangurp 1y agoAre blind people not intelligent? Is it actually important to have a full understanding of the world? What about people that grew up in isolation? I think there are still some tribes in the Amazon that have had little or no contact with modern civilization. Are these people not intelligent? There are some deep philosophical topics lurking here. But the bottom line is that you can obviously have intelligent conversations with blind people. Doing that with a person that is both deaf and blind is a bit challenging, for obvious reasons. But if they otherwise have a normal brain you might be able to learn to communicate with them in some way and they might be able to be observed doing things that are smart/intelligent. And some people that are deaf and blind actually manage to learn to write and speak. And there have been a few cases of people like that getting academic degrees. Clearly sight and hearing are not that essential to intelligence. Having some way to communicate via touch or something else is probably helpful for communicating and sharing information. But just a simple chat might be all that's needed for an AGI.
- bufferoverflow 1y agoBlind people have a lot less understanding of the visual part of the world. For example, if you have a chemical reaction, the result of which relies on color, they can't tell you the result (without using some seeing tool like a camera).