7 ms·

Humans can do all these things, and much better than machines, and yet noone has conquered the world.

nopinsight 3y ago
Humans have very limited bandwidth, knowledge, and speed relative to an AGI in the world of abundant GPUs. Humans are easy to capture relative to software. They cannot make copies of themselves. A small group cannot be at many places at once and usually not without being seen/known about. A larger group is bad at coordination without anyone leaking critical info to an outsider. Most people are not deeply malicious and are instinctively repelled by very immoral things. An AGI may not by default have such an instinct.
- RandomLensman 3y ago
  Humans can reproduce without pretty much any infrastructure, they can self-repair without infrastructure. An AGI has no physical manifestation to start with, it needs electricity, is susceptible to various weapons that humans are not etc.
  - nopinsight 3y ago
    Are you seriously comparing the cost and speed to reproduce a piece of software with a living, functioning human being? Intelligent software can also hide in an existing infrastructure much more easily than a human can. Also, we're talking about the world in which the cost of GPUs is dropping and GPUs becoming much more abundant over time.
    - RandomLensman 3y ago
      No, but any hypothetical AGI would be reliant on machinery to survive, humans are are not. Humans could EMP all electronics (heck, the sun could so that for us even), blow up all infrastructures, factories, etc. and still survive. Being able to fast copy while totally being reliable on artificial structures is a massive weakness. Software is just software - it doesn't generate electricity on its own.
      - nopinsight 3y ago
        That level of coordination among all, even most, human groups is extremely unlikely. Humanity and civilization depend on many critical infrastructures. We also try to outcompete other groups all the time. In the near future with abundant GPUs, AGI and ASI can easily threaten or bribe some groups to keep it alive in exchange for powerful inventions/technologies.
        RandomLensman 3y ago
        That is just not true. Large scale coordination isn't that unheard of (see WWII, treaties and cooperations on various issues). AGI might not even have magic technologies to offer and that whoever is siding with AGI has the power to subdue the rest of humanity is a bold speculation. Just like humans haven't even subdued viruses there is no reason to assume that to be true.
        nopinsight 3y ago
        It doesn’t need to have magic technology, just access to critical pieces of software it hides in/merge with. See how quite a few people reacted when Replika was nerfed. Imagine what happens when more important pieces of software are supposed to be turned off to eliminate a rouge AGI. (Pretty many who argue against AGI danger will be the first to argue against it.) Have we even managed to eliminate all the dumb computer viruses from the world?
        RandomLensman 3y ago
        I don't think that aligns with history: we recently went through large shutdowns and lockdowns, we had and have people doing their daily work during war and massive destructions - shutting down things will not be too difficult if the need arises.
        nopinsight 3y ago
        Wars are a terrible analogy because it implies there are multiple sides. Why wouldn't any intelligent being, AGI included, take advantage of that? Not to mention the fact that in most wars there are spies and double agents. In the near future when GPUs become abundant, many small groups of people can harbor a copy of an AGI in their basement, where it can plan and re-spawn whenever the situation becomes accommodating.
- usrbinbash 3y ago
  All that is true. But humans have had thousands of years to do such things, and yet: No world dominator.
  - pixl97 3y ago
    Because at the end of the day individual humans are within an order of magnitude of intelligence of one another. Also humans die. Human systems have been 'weak' without technology. Human thought and travel is slow. If we take AI out of the equation and just add technology there is significant risk that a world dominator could arise. Either a negative dominator (no humans left to control due to nuclear war) or positive dominator (1984 cameras always watching you). There simply hasn't been enough time/luck for these to shake out yet. Now, add something that over a magnitude smarter and could copy itself a nearly unlimited number of times and you are in new and incalculable territory.
    - usrbinbash 3y ago
      > humans are within an order of magnitude of intelligence of one another. Yes, and maybe that will be different with an AGI. Maybe AGI is physically possible. And maybe that advantage in intelligence will make AGI vastly more powerful than us. Those are a lot of "maybes" however, and thus all of this remains highly speculative. Just one example that puts a pretty big dent into many scenarios of all-powerful AGIs: I think everyone agrees that even the dumbest human still outsmarts bacteria, protists and viruses by several orders of magnitude. And yet, we haven't been able to rid the world off things like Measles, Cholera, Malaria or HIV. Even the common cold and influenza are still around. So, if we, with out big brains, that split the atom, went to the moon and developed calculus, cannot beat tiny infectious agents that don't even have a brain, then I remain kinda sceptic that being a very very very smart ML model means an Automatic Win over the human species.
mr_mitm 3y ago
I thought we are considering a worst case scenario, where some super human AGI, which has access to a lot more data than any single human being, can much more quickly cross-reference everything than a network of humans ever could. It can already write much quicker than a human. Imagine what an AGI could do that wouldn't need to painstakingly write papers, publish them, read them, meet on conferences, and so on, to make progress.
- usrbinbash 3y ago
  > I thought we are considering a worst case scenario Well, I am not. Unless someone can show me evidence of an AGI a) being possible, b) being within near future reach and c) being an existential threat. Most humans are not leaving their house under the assumption that they will get hit by a meteorite either. And that is actually an occurrence that we know for a certainty is physically possible.
andruby 3y ago
I would actually argue that yes, humans _have_ conquered the world. As in: we are now at the top of the food chain and decide which habitats and which animals get to live and which don't. Because we are the most intelligent being on the planet. In our pursuit to that place, we've used other animals with lesser intelligence to our advantage (dogs, horses, ...) The premise is that an AI with super-human intelligence could use us in the same way. And to be honest, we're really not that hard to manipulate or persuade (money, religion, blackmail, power, ...)
deleted 3y ago
[deleted]
naasking 3y ago
> Humans can do all these things, and much better than machines No they can't. Show me a human that doesn't need to sleep.

Humans have very limited bandwidth, knowledge, and speed relative to an AGI in the world of abundant GPUs. Humans are easy to capture relative to software. They cannot make copies of themselves. A small group cannot be at many places at once and usually not without being seen/known about. A larger group is bad at coordination without anyone leaking critical info to an outsider. Most people are not deeply malicious and are instinctively repelled by very immoral things. An AGI may not by default have such an instinct.

RandomLensman 3y ago
Humans can reproduce without pretty much any infrastructure, they can self-repair without infrastructure. An AGI has no physical manifestation to start with, it needs electricity, is susceptible to various weapons that humans are not etc.
- nopinsight 3y ago
  Are you seriously comparing the cost and speed to reproduce a piece of software with a living, functioning human being? Intelligent software can also hide in an existing infrastructure much more easily than a human can. Also, we're talking about the world in which the cost of GPUs is dropping and GPUs becoming much more abundant over time.
  - RandomLensman 3y ago
    No, but any hypothetical AGI would be reliant on machinery to survive, humans are are not. Humans could EMP all electronics (heck, the sun could so that for us even), blow up all infrastructures, factories, etc. and still survive. Being able to fast copy while totally being reliable on artificial structures is a massive weakness. Software is just software - it doesn't generate electricity on its own.
    - nopinsight 3y ago
      That level of coordination among all, even most, human groups is extremely unlikely. Humanity and civilization depend on many critical infrastructures. We also try to outcompete other groups all the time. In the near future with abundant GPUs, AGI and ASI can easily threaten or bribe some groups to keep it alive in exchange for powerful inventions/technologies.
      - RandomLensman 3y ago
        That is just not true. Large scale coordination isn't that unheard of (see WWII, treaties and cooperations on various issues). AGI might not even have magic technologies to offer and that whoever is siding with AGI has the power to subdue the rest of humanity is a bold speculation. Just like humans haven't even subdued viruses there is no reason to assume that to be true.
        nopinsight 3y ago
        It doesn’t need to have magic technology, just access to critical pieces of software it hides in/merge with. See how quite a few people reacted when Replika was nerfed. Imagine what happens when more important pieces of software are supposed to be turned off to eliminate a rouge AGI. (Pretty many who argue against AGI danger will be the first to argue against it.) Have we even managed to eliminate all the dumb computer viruses from the world?
        RandomLensman 3y ago
        I don't think that aligns with history: we recently went through large shutdowns and lockdowns, we had and have people doing their daily work during war and massive destructions - shutting down things will not be too difficult if the need arises.
        nopinsight 3y ago
        Wars are a terrible analogy because it implies there are multiple sides. Why wouldn't any intelligent being, AGI included, take advantage of that? Not to mention the fact that in most wars there are spies and double agents. In the near future when GPUs become abundant, many small groups of people can harbor a copy of an AGI in their basement, where it can plan and re-spawn whenever the situation becomes accommodating.
usrbinbash 3y ago
All that is true. But humans have had thousands of years to do such things, and yet: No world dominator.
- pixl97 3y ago
  Because at the end of the day individual humans are within an order of magnitude of intelligence of one another. Also humans die. Human systems have been 'weak' without technology. Human thought and travel is slow. If we take AI out of the equation and just add technology there is significant risk that a world dominator could arise. Either a negative dominator (no humans left to control due to nuclear war) or positive dominator (1984 cameras always watching you). There simply hasn't been enough time/luck for these to shake out yet. Now, add something that over a magnitude smarter and could copy itself a nearly unlimited number of times and you are in new and incalculable territory.
  - usrbinbash 3y ago
    > humans are within an order of magnitude of intelligence of one another. Yes, and maybe that will be different with an AGI. Maybe AGI is physically possible. And maybe that advantage in intelligence will make AGI vastly more powerful than us. Those are a lot of "maybes" however, and thus all of this remains highly speculative. Just one example that puts a pretty big dent into many scenarios of all-powerful AGIs: I think everyone agrees that even the dumbest human still outsmarts bacteria, protists and viruses by several orders of magnitude. And yet, we haven't been able to rid the world off things like Measles, Cholera, Malaria or HIV. Even the common cold and influenza are still around. So, if we, with out big brains, that split the atom, went to the moon and developed calculus, cannot beat tiny infectious agents that don't even have a brain, then I remain kinda sceptic that being a very very very smart ML model means an Automatic Win over the human species.

mr_mitm 3y ago

I thought we are considering a worst case scenario, where some super human AGI, which has access to a lot more data than any single human being, can much more quickly cross-reference everything than a network of humans ever could. It can already write much quicker than a human. Imagine what an AGI could do that wouldn't need to painstakingly write papers, publish them, read them, meet on conferences, and so on, to make progress.

usrbinbash 3y ago
> I thought we are considering a worst case scenario Well, I am not. Unless someone can show me evidence of an AGI a) being possible, b) being within near future reach and c) being an existential threat. Most humans are not leaving their house under the assumption that they will get hit by a meteorite either. And that is actually an occurrence that we know for a certainty is physically possible.

andruby 3y ago

I would actually argue that yes, humans _have_ conquered the world. As in: we are now at the top of the food chain and decide which habitats and which animals get to live and which don't. Because we are the most intelligent being on the planet. In our pursuit to that place, we've used other animals with lesser intelligence to our advantage (dogs, horses, ...) The premise is that an AI with super-human intelligence could use us in the same way. And to be honest, we're really not that hard to manipulate or persuade (money, religion, blackmail, power, ...)

deleted 3y ago

[deleted]

naasking 3y ago

> Humans can do all these things, and much better than machines No they can't. Show me a human that doesn't need to sleep.