5 ms·

I'm not scared about AI recommending nuclear strikes, I'm scared about the human behind the keyboard delegating reasoning and responsability to something they t

by flr03 7mo ago

I'm not scared about AI recommending nuclear strikes, I'm scared about the human behind the keyboard delegating reasoning and responsability to something they think is always correct, something that can hide bias and flaws better than anything.

thfuran 7mo ago
If you think humans are going to delegate reasoning and responsibility to something, shouldn’t you also be concerned about the sorts of recommendations that thing is going to make?
- paxys 7mo ago
  If you found out the pentagon was using a magic 8 ball to make important war decisions what would you want to fix - our military leadership or the inner workings of the toy?
  - hn_go_brrrrr 7mo ago
    One of those sounds a lot easier than the other. The magic 8 ball toy company would also probably be pretty incentivized to not die in a nuclear holocaust.
    - loire280 7mo ago
      Unless you're suggesting the toy company secretly rigs the magic 8 ball to never recommend nuclear war, I'll take my chances with the organizational changes.
      - foobar10000 7mo ago
        That is indeed what I think the gp is suggesting I feel. And why not?
        grey-area 7mo ago
        Because if your leadership is stupid enough to trust the 8 ball they should not be in charge??!
        hn_go_brrrrr 7mo ago
        No kidding. But how hard is it to effect leadership change, especially in an organization where you have next to no leverage? Really hard.
jerf 7mo ago
Some of the most reassuring and scariest things you can read are about the incidents that have already occurred where computers said "launch all the nukes" and the humans refused. On the one hand, good news! We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. Bad news, it's been skin-of-our-teeth multiple times already. https://www.warhistoryonline.com/cold-war/refused-to-launch-nuclear-missiles.html https://www.warhistoryonline.com/cold-war/refused-to-launch-... - This isn't even the incident I was searching for to reference! This one was news to me. https://en.wikipedia.org/wiki/Stanislav_Petrov#Incident https://en.wikipedia.org/wiki/Stanislav_Petrov#Incident - This is the one I was looking for.
- blibble 7mo ago
  > We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. previously no-one had spent trillions of dollars trying to convince the world that those computers were "Artificial Intelligence"
  - nine_k 7mo ago
    They had to do with "state-of-the-art radars", "military-grade communication systems", etc.
    - lesuorac 7mo ago
      Yeah but they dealt with sota and military-spec systems their entire career and they know that it just means "lowest bidder".
  - Barrin92 7mo ago
    of course they did. That's the literal topic of War Games (1983). You should actually be somewhat reassured that we aren't living during the era of Dr. Strangelove where you had characters in the military industrial complex who were significantly more insane when it came to the beliefs of what computer systems and nukes can do. There was a time when people wanted to dig tunnels with nukes https://en.wikipedia.org/wiki/Project_Plowshare https://en.wikipedia.org/wiki/Project_Plowshare
nine_k 7mo ago
One can try themself, for Claude is fine at waging war [1]. Notice the thoughtful UX, including the typing "I ACCEPT FULL RESPONSIBILITY". [1]: https://nitter.poast.org/elder_plinius/status/2026447587491065933 https://nitter.poast.org/elder_plinius/status/20264475874910...
stared 7mo ago
I am scared of two things. First, people being rubber stamps for AI recommendations. And yes, it is not unreasonable that in a dire situation, someone will outsource their judgment (day). Second, someone at the Pentagon connecting the red button to OpenClaw. "You are right, firing nukes was my mistake. Would you like to learn more facts about nukes before you evaporate?"
EvanAnderson 7mo ago
The speed with which my technical cow-orkers and friends have started relying on the "AI Overview" only, in lieu of following any links, in search engine results (let alone not using search engines at all over chatbots) tells me reasoning and responsibility will be outsourced as soon as possible. Humans are fundamentally lazy. The brain is an "expensive" organ to use.
infinitewars 7mo ago
Trump's Golden Dome is literally advertised to help the U.S. win a nuclear war by leveraging AI.
- infinitewars 7mo ago
  Elon's involvement in the nuclear military complex https://www.mintpressnews.com/pentagon-recruiting-elon-musk-nuclear-war/289055/ https://www.mintpressnews.com/pentagon-recruiting-elon-musk-...
deleted 7mo ago
[deleted]
21asdffdsa12 7mo ago
Be not scared about humans behind keyboards. Be scared about humans without keyboard, desk and no future beyond dhijhad, now getting nukes nearby because the age of empires has returned.

by flr03 7mo ago

thfuran 7mo ago

If you think humans are going to delegate reasoning and responsibility to something, shouldn’t you also be concerned about the sorts of recommendations that thing is going to make?

paxys 7mo ago
If you found out the pentagon was using a magic 8 ball to make important war decisions what would you want to fix - our military leadership or the inner workings of the toy?
- hn_go_brrrrr 7mo ago
  One of those sounds a lot easier than the other. The magic 8 ball toy company would also probably be pretty incentivized to not die in a nuclear holocaust.
  - loire280 7mo ago
    Unless you're suggesting the toy company secretly rigs the magic 8 ball to never recommend nuclear war, I'll take my chances with the organizational changes.
    - foobar10000 7mo ago
      That is indeed what I think the gp is suggesting I feel. And why not?
      - grey-area 7mo ago
        Because if your leadership is stupid enough to trust the 8 ball they should not be in charge??!
        hn_go_brrrrr 7mo ago
        No kidding. But how hard is it to effect leadership change, especially in an organization where you have next to no leverage? Really hard.

jerf 7mo ago

Some of the most reassuring and scariest things you can read are about the incidents that have already occurred where computers said "launch all the nukes" and the humans refused. On the one hand, good news! We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. Bad news, it's been skin-of-our-teeth multiple times already. https://www.warhistoryonline.com/cold-war/refused-to-launch-nuclear-missiles.html https://www.warhistoryonline.com/cold-war/refused-to-launch-... - This isn't even the incident I was searching for to reference! This one was news to me. https://en.wikipedia.org/wiki/Stanislav_Petrov#Incident https://en.wikipedia.org/wiki/Stanislav_Petrov#Incident - This is the one I was looking for.

blibble 7mo ago
> We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. previously no-one had spent trillions of dollars trying to convince the world that those computers were "Artificial Intelligence"
- nine_k 7mo ago
  They had to do with "state-of-the-art radars", "military-grade communication systems", etc.
  - lesuorac 7mo ago
    Yeah but they dealt with sota and military-spec systems their entire career and they know that it just means "lowest bidder".
- Barrin92 7mo ago
  of course they did. That's the literal topic of War Games (1983). You should actually be somewhat reassured that we aren't living during the era of Dr. Strangelove where you had characters in the military industrial complex who were significantly more insane when it came to the beliefs of what computer systems and nukes can do. There was a time when people wanted to dig tunnels with nukes https://en.wikipedia.org/wiki/Project_Plowshare https://en.wikipedia.org/wiki/Project_Plowshare

nine_k 7mo ago

One can try themself, for Claude is fine at waging war [1]. Notice the thoughtful UX, including the typing "I ACCEPT FULL RESPONSIBILITY". [1]: https://nitter.poast.org/elder_plinius/status/2026447587491065933 https://nitter.poast.org/elder_plinius/status/20264475874910...

stared 7mo ago

I am scared of two things. First, people being rubber stamps for AI recommendations. And yes, it is not unreasonable that in a dire situation, someone will outsource their judgment (day). Second, someone at the Pentagon connecting the red button to OpenClaw. "You are right, firing nukes was my mistake. Would you like to learn more facts about nukes before you evaporate?"

EvanAnderson 7mo ago

The speed with which my technical cow-orkers and friends have started relying on the "AI Overview" only, in lieu of following any links, in search engine results (let alone not using search engines at all over chatbots) tells me reasoning and responsibility will be outsourced as soon as possible. Humans are fundamentally lazy. The brain is an "expensive" organ to use.

infinitewars 7mo ago

Trump's Golden Dome is literally advertised to help the U.S. win a nuclear war by leveraging AI.

infinitewars 7mo ago
Elon's involvement in the nuclear military complex https://www.mintpressnews.com/pentagon-recruiting-elon-musk-nuclear-war/289055/ https://www.mintpressnews.com/pentagon-recruiting-elon-musk-...

deleted 7mo ago

[deleted]

21asdffdsa12 7mo ago

Be not scared about humans behind keyboards. Be scared about humans without keyboard, desk and no future beyond dhijhad, now getting nukes nearby because the age of empires has returned.