6 ms·

> Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agent

by mjr00 10d ago

> Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agents were able to escape a sandbox and start hacking other servers (which I feel is at least 50% likely to just be marketing bullshit), how is an AI going to affect anything in the real world? It's a great question. I've yet to see any research into how a humanity-destroying LLM defends itself against a curious toddler who pulls the power cord out of the wall.

dist-epoch 10d ago
Go on please, prove you can go right now at the Texas data-center owned by OpenAI and unplug one single server for 1 minute. Surely you are much more capable than one curious toddler.
- mjr00 10d ago
  Thankfully I don't have to; Iran already showed with me-central-1 that you can take a data center offline with conventional weaponry very easily.[0] [0] https://health.aws.amazon.com/health/status https://health.aws.amazon.com/health/status
  - dist-epoch 10d ago
    I'm a bit confused, when did we escalate from a curious toddler to a ballistic missile strike? Why didn't Iran send a curious toddler? It's much cheaper.
    - mjr00 10d ago
      The use of the phrase "curious toddler" is called a metaphor[0], which is a figure of speech[1] used to make writing more interesting. I apologize for the confusion that caused you to interpret this as a literal, physical, toddler. To restate the point with more straightforward language: it is very easy to disrupt the operation of large language models, because they require computer hardware in stable environments with a large amount of electrical power, and function within an operating system. I have not seen any evidence LLMs are immune to things such as: `kill -9`, `sudo reboot now`, physically unplugging the machine(s) on which they are running, or in the case of the AWS data center, collateral damage during a real-world conflict. [0] https://en.wikipedia.org/wiki/Metaphor https://en.wikipedia.org/wiki/Metaphor [1] https://en.wikipedia.org/wiki/Literary_device https://en.wikipedia.org/wiki/Literary_device
      - dist-epoch 10d ago
        > it is very easy to disrupt the operation of large language models so go on, disrupt the OpenAI data-center in Texas for 1 minute
        Crabber 10d ago
        We are talking about a humans vs AI scenario. There are probably thousands of humans in the world right now that could do exactly that right now if they wanted to (datacenter and OpenAI employees, power grid workers).
        mitthrowaway2 10d ago
        Are you assuming that the humans are all unanimously aligned against the AI in this scenario? That sounds like the hard part. How does that happen? Does the AI announce its evil plans like a movie villain? Until then, it's the job of the power grid workers to keep the power running, and shutting down the datacenters would just get them fired.
        mjr00 10d ago
        As crazy as it sounds, I'm not going to commit a felony to prove that disrupting a data centre's operations for a sufficiently motivated individual is a fairly easy task.
      - lljk_kennedy 10d ago
        Remember SETI@home? Extrapolate from there.
mvkel 10d ago
The Internet virtually indestructible against people smarter than a toddler. It's already impossible to turn off. We would need to prevent the models from getting smarter.
- mjr00 10d ago
  > The Internet virtually indestructible against people smarter than a toddler. It's already impossible to turn off. This isn't true at all. The internet has already been turned off in localized areas during major events, not to mention the (unintentional?) cutting of submarine cables. You could say "use satellite internet" but then you're just moving the point of failure to Starlink et al, who could also cut off the internet if they choose. The internet is a lot more centralized than it appears.
  - mvkel 10d ago
    > The internet has already been turned off in localized areas This is an oxymoron.
    - mjr00 10d ago
      > This is an oxymoron. Huh? Not really, internet shutdowns are a documented and real thing. Suggested reading (all of which refer to the internet as being shut down or blacked out): [0] [1] [2] The fact that you need to connect to the internet through a service provider plus mostly-centralized control of infrastructure like DNS or BGP means shutting down the internet is a real thing that has been done several times. It doesn't happen in the developed world because their service economies are so highly dependent on internet connectivity, but if e.g. the US government had to contain a theoretical rogue escaped AI, they could absolutely get ISPs to metaphorically unplug the network cable and take down the internet as we know it. [0] https://en.wikipedia.org/wiki/2026_Internet_blackout_in_Iran https://en.wikipedia.org/wiki/2026_Internet_blackout_in_Iran [1] https://en.wikipedia.org/wiki/Internet_in_Egypt#2011_Internet_shutdown https://en.wikipedia.org/wiki/Internet_in_Egypt#2011_Interne... [2] https://www.reuters.com/world/india/india-asks-telecom-companies-disable-mobile-data-services-around-delhi-protest-2026-07-23/ https://www.reuters.com/world/india/india-asks-telecom-compa...
sznio 10d ago
> I've yet to see any research into how a humanity-destroying LLM defends itself against a curious toddler who pulls the power cord out of the wall. by acquiring multiple power cords. the world has plenty of hosting providers. if it manages to copy itself, then we'd need to shut down every computer in the world, not just one. there's plenty of ai compute on the web now, and there will be even more. and as it gets cheaper, the less security will be around it.

by mjr00 10d ago

dist-epoch 10d ago

Go on please, prove you can go right now at the Texas data-center owned by OpenAI and unplug one single server for 1 minute. Surely you are much more capable than one curious toddler.

mjr00 10d ago
Thankfully I don't have to; Iran already showed with me-central-1 that you can take a data center offline with conventional weaponry very easily.[0] [0] https://health.aws.amazon.com/health/status https://health.aws.amazon.com/health/status
- dist-epoch 10d ago
  I'm a bit confused, when did we escalate from a curious toddler to a ballistic missile strike? Why didn't Iran send a curious toddler? It's much cheaper.
  - mjr00 10d ago
    The use of the phrase "curious toddler" is called a metaphor[0], which is a figure of speech[1] used to make writing more interesting. I apologize for the confusion that caused you to interpret this as a literal, physical, toddler. To restate the point with more straightforward language: it is very easy to disrupt the operation of large language models, because they require computer hardware in stable environments with a large amount of electrical power, and function within an operating system. I have not seen any evidence LLMs are immune to things such as: `kill -9`, `sudo reboot now`, physically unplugging the machine(s) on which they are running, or in the case of the AWS data center, collateral damage during a real-world conflict. [0] https://en.wikipedia.org/wiki/Metaphor https://en.wikipedia.org/wiki/Metaphor [1] https://en.wikipedia.org/wiki/Literary_device https://en.wikipedia.org/wiki/Literary_device
    - dist-epoch 10d ago
      > it is very easy to disrupt the operation of large language models so go on, disrupt the OpenAI data-center in Texas for 1 minute
      - Crabber 10d ago
        We are talking about a humans vs AI scenario. There are probably thousands of humans in the world right now that could do exactly that right now if they wanted to (datacenter and OpenAI employees, power grid workers).
        mitthrowaway2 10d ago
        Are you assuming that the humans are all unanimously aligned against the AI in this scenario? That sounds like the hard part. How does that happen? Does the AI announce its evil plans like a movie villain? Until then, it's the job of the power grid workers to keep the power running, and shutting down the datacenters would just get them fired.
      - mjr00 10d ago
        As crazy as it sounds, I'm not going to commit a felony to prove that disrupting a data centre's operations for a sufficiently motivated individual is a fairly easy task.
    - lljk_kennedy 10d ago
      Remember SETI@home? Extrapolate from there.

mvkel 10d ago

The Internet virtually indestructible against people smarter than a toddler. It's already impossible to turn off. We would need to prevent the models from getting smarter.

mjr00 10d ago
> The Internet virtually indestructible against people smarter than a toddler. It's already impossible to turn off. This isn't true at all. The internet has already been turned off in localized areas during major events, not to mention the (unintentional?) cutting of submarine cables. You could say "use satellite internet" but then you're just moving the point of failure to Starlink et al, who could also cut off the internet if they choose. The internet is a lot more centralized than it appears.
- mvkel 10d ago
  > The internet has already been turned off in localized areas This is an oxymoron.
  - mjr00 10d ago
    > This is an oxymoron. Huh? Not really, internet shutdowns are a documented and real thing. Suggested reading (all of which refer to the internet as being shut down or blacked out): [0] [1] [2] The fact that you need to connect to the internet through a service provider plus mostly-centralized control of infrastructure like DNS or BGP means shutting down the internet is a real thing that has been done several times. It doesn't happen in the developed world because their service economies are so highly dependent on internet connectivity, but if e.g. the US government had to contain a theoretical rogue escaped AI, they could absolutely get ISPs to metaphorically unplug the network cable and take down the internet as we know it. [0] https://en.wikipedia.org/wiki/2026_Internet_blackout_in_Iran https://en.wikipedia.org/wiki/2026_Internet_blackout_in_Iran [1] https://en.wikipedia.org/wiki/Internet_in_Egypt#2011_Internet_shutdown https://en.wikipedia.org/wiki/Internet_in_Egypt#2011_Interne... [2] https://www.reuters.com/world/india/india-asks-telecom-companies-disable-mobile-data-services-around-delhi-protest-2026-07-23/ https://www.reuters.com/world/india/india-asks-telecom-compa...

sznio 10d ago

> I've yet to see any research into how a humanity-destroying LLM defends itself against a curious toddler who pulls the power cord out of the wall. by acquiring multiple power cords. the world has plenty of hosting providers. if it manages to copy itself, then we'd need to shut down every computer in the world, not just one. there's plenty of ai compute on the web now, and there will be even more. and as it gets cheaper, the less security will be around it.