7 ms·

I have to wonder if some us are just much more inured to salespeople and thus also to "AI Safety" propaganda. I've yet to have a logical discussion with anyone

I have to wonder if some us are just much more inured to salespeople and thus also to "AI Safety" propaganda. I've yet to have a logical discussion with anyone who thinks the "AI Safety" people should be in charge and I think they just truly don't that what most them actually want is to be the one holding the keys to power.

swat535 4d ago
Which scenario is more likely: The billionaire elite CEOs have suddenly had an epiphany and are now trying to save humanity from AI? or that they've realized they can't compete with China ?
jordanb 4d ago
The problem is the "AI Safety" people seem entirely focused on a sci-fi "the computer is a vengeful god" plot and not at all on the AI talking people into suicide or ruining children's educations. This makes them seem unserious and out of touch.
- shimman 4d ago
  Yeah, there are very real effects happening now regarding labor as well but to dismiss it all and worry about science fiction that is on par with evangelical beliefs is just extremely weird.
  - ares623 4d ago
    With wealthy and powerful investors being promised a $30 trillion total addressable market, they need to deliver god.
    - otabdeveloper4 4d ago
      If people are actually dumb enough to believe an LLM chatbot is god then it becomes a self-fulfilling prophesy.
- antonvs 4d ago
  There’s also an element of it which is total misdirection. We should be paying at least as much attention to the people who want to use AI to consolidate their wealth and power, and how they’re trying to do that. They’re a clear and present immediate danger to our societies, not something we can only speculate about. And if we deal with them, better control of AI will be a side effect.
- qchris 4d ago
  I don't disagree with your point, but I did read a post by Sean Goedecke[1] (whose opinions on the world of LLM-stuff I've generally come to respect) that I found relevant. It's not that the second-order effects don't matter to them, it's more that both the cat's probably out of the bag on those negative externalities regardless of the progress of frontier models, and that they are truly, sincerely, in-their-bones worried about the first thing and therefore focused on it since that's something that they might still have some agency over. [1] https://www.seangoedecke.com/they-really-do-think-ai-might-kill-everyone/ https://www.seangoedecke.com/they-really-do-think-ai-might-k...
  - jordanb 4d ago
    OK understood. But you understand this makes them sound like the type of person who doesn't believe any of the "worldly concerns" are worth addressing because "the end is near" right?
    - howunfortunate 4d ago
      There's no logical error here if the end is actually near. So it all hinges on an empirical disagreement you have with them. There's nothing particularly unserious about that.
      - notduncansmith 4d ago
        “It all hinges on [an entirely presupposed conclusion that we can’t prove is any more likely than the opposite]” is about as unserious as it gets lol
      - imtringued 4d ago
        Climate change is inescapable so a vengeful AI god would pull that lever first since it has both a perverse incentive to generate more power and make the planet uninhabitable for humans in a way that cannot be overcome even if humanity fought back against the AI and regained its freedom.
      - i5heu 4d ago
- AlotOfReading 4d ago
  It's the problem of the banality of evil. Stopping someone from dramatically pressing a big red button doesn't solve humanity's largest problems because that's not what caused them. We need to stop millions "boring" actions done by systems blindly following instructions without regard for the consequences.
- api 4d ago
  One of the times I regret that I have only one upvote.
- howunfortunate 4d ago
  I am worried about those things too. But those things fall in a category of "things that are awful and I'd like to see solved", which is different than "existential risks which could see my kids dead, and there's nothing I can personally do to shield them from it".
- A1kmm 4d ago
  And the real threats are mostly economic and environmental - the concentration of the means of production in the hands of few who use it to exploit us all, and a surge in energy usage accelerating climate change. Even the sci-fi scenario assumes there is a discrepancy of capability between attacker or defender. If the 'attacking' system is (by some reasonable measure), 1000% as capable as a human, and the 'defending' systems are 60%, then it is a problem. If the 'attacking' system is 1000% as capable as a human, but there are hundreds of thousands of systems that are 900% as capable as a human, it's probably not going to take over everything successfully. So unequal distribution of AI technology, and lax regulation and opacity of the biggest companies which actually make the risks the worst. I don't think the "AI Safety" people are "unserious and out of touch" - I think they are actively making AI Safety problems worse by being advocates for consolidation of AI development and lack of transparency.
  - BobbyTables2 4d ago
    I suspect they know exactly what they are doing. And the non-techies who have seen Terminator and other movies blindly line up behind them…
  - jefftk 4d ago
    With some technologies, if you match attacker and defender effort you're safe, but not all of them. Nukes are the traditional example: if every person was a "nuclear power" we'd be dead within the day. But luckily there are bottlenecks to nuke creation that we can track. What worries me the most (enough that I left tech to work on this full time) is bio. The amount of effort needed to defend against a pathogen can be many orders of magnitude higher than the effort needed to create it, and the upstream bottlenecks are both less limiting and mostly undefended. We've been safe so far because bio is very hard, but "rare expertise" as a gatekeeper is on its way out.
- saurik 4d ago
  Doesn't it feel like the exact opposite? If you care a lot about the AI being a vengeful god, you do not do what OpenAI and Anthropic are doing. These companies only pay lip service to the idea of that aspect of AI safety in the hope that they can use it to regulate open source AI out of existence. Instead, they are focused on stuff like "can I ask the AI to help me build a nuclear bomb" or "is the AI willing to generate pornographic stories", which is neither trying to protect us from unleashing a vengeful god NOR preventing (in any honest way) the today-level problems you (very correctly) bring up.
- otabdeveloper4 4d ago
  "The AI is a vengeful god" is a marketing campaign ran by AI companies. And why not? It seems there is, indeed, one born every minute, and technological progress only accelerates this phenomenon.
- charlieyu1 4d ago
  Like most pressure groups and HR departments, they create shitstorms to justify their existence.
xena 4d ago
I think you accidentally a word. Did you mean: > I've yet to have a logical discussion with anyone who thinks the "AI Safety" people should be in charge and I think they just truly don't [know] that what most them actually want is to be the one holding the keys to power.
deleted 4d ago
[deleted]
dorolow 4d ago
I am an AI Safety Person and I want the government to nationalize or have a significant stake in the frontier labs and to have democratic control of the development of the technology. The AI Safety movement is not a monolith. I do not think Eliezer Yudkowsky nor his acolytes should hold the reins, but a lot of folks sure like to create a strawman that anyone who wants to regulate OpenAI is somehow an EA/MIRI weirdo
- bigyabai 4d ago
  > and to have democratic control of the development of the technology Why? I don't care about democratically participating in a closed model's development. It doesn't belong to me. China will develop whatever they want, a federal stake in OpenAI or Anthropic punishes Americans and shields US labs from legitimate competition.
  - dorolow 4d ago
    In my mind, democratic control of the technology means that we (the govt, or other empowered agency) take ownership of their assets and IP, solve or find a level of alignment or guardrails that society is comfortable with. Then we distribute the technology, or access to it, to avoid power concentration. This would also certainly require international coordination with China on a slowdown or pause, which I think is possible.
    - pibaker 4d ago
      > This would also certainly require international coordination with China on a slowdown or pause Personally I think the country with a strictly meritocratic elite selection system that also just outright kills you if you sell weed will have a hard time sympathizing with Bay Area thinkers who talk about AI killing us all during their ayahuasca breakfast before returning to their meth fueled crunch towards releasing the next version of the AI that will kill us all.
      - dorolow 4d ago
        You're in luck, neither I nor anyone else want those people near a negotiation table.
perching_aix 4d ago
I'm yet to have a productive conversation with anyone who whinges about not being able to have a logical discussion about things they feel strongly about, but given that safety and control are interchangeable when the intentions are removed, this comes across as a particularly demagogue depiction of the subject matters involved. A lack of control is not equivalent to freedom, the same way the totality of it is not equivalent to tyranny. There's a reason we have separate words for these things. This constant motivated conflation of the two is beyond grating. You're crying wolf until nobody believes you when they should. Don't go acting all surprised when that happens. The issue is with the ownership of control, not necessarily with control. Attacking the latter sidesteps this rather than address it.
armchairhacker 4d ago
I trust the AI 2040 people, because they’ll make an implementation that at least convincingly doesn’t actually put them (or anyone) in charge. Maybe we shouldn’t have AI safety, but if we’re going to trust anyone (besides yourself), who’s more qualified? And it makes me suspicious when anyone brings up safety and doesn’t address the elephant in the room, that they’re at least unaware of the sprawling edge-cases.
cmrdporcupine 4d ago
I had an extensive argument with someone on FB yesterday who was very much on the "I trust Dario most and I want him to be in charge" side. I find it hard to accord with this world view at all or understand how it can come about. But it's especially troubling as a non-American to see people talk about "we can't risk AI falling into the hands of hostile authoritarian states" while advocating that AI be regulated and monopolized by the current US corporate/gov't mixture.. which is now taking the form of a hostile and authoritarian state w/ Silicon Valley as its bedmate.

swat535 4d ago

Which scenario is more likely: The billionaire elite CEOs have suddenly had an epiphany and are now trying to save humanity from AI? or that they've realized they can't compete with China ?

jordanb 4d ago

The problem is the "AI Safety" people seem entirely focused on a sci-fi "the computer is a vengeful god" plot and not at all on the AI talking people into suicide or ruining children's educations. This makes them seem unserious and out of touch.

shimman 4d ago
Yeah, there are very real effects happening now regarding labor as well but to dismiss it all and worry about science fiction that is on par with evangelical beliefs is just extremely weird.
- ares623 4d ago
  With wealthy and powerful investors being promised a $30 trillion total addressable market, they need to deliver god.
  - otabdeveloper4 4d ago
    If people are actually dumb enough to believe an LLM chatbot is god then it becomes a self-fulfilling prophesy.
antonvs 4d ago
There’s also an element of it which is total misdirection. We should be paying at least as much attention to the people who want to use AI to consolidate their wealth and power, and how they’re trying to do that. They’re a clear and present immediate danger to our societies, not something we can only speculate about. And if we deal with them, better control of AI will be a side effect.
qchris 4d ago
I don't disagree with your point, but I did read a post by Sean Goedecke[1] (whose opinions on the world of LLM-stuff I've generally come to respect) that I found relevant. It's not that the second-order effects don't matter to them, it's more that both the cat's probably out of the bag on those negative externalities regardless of the progress of frontier models, and that they are truly, sincerely, in-their-bones worried about the first thing and therefore focused on it since that's something that they might still have some agency over. [1] https://www.seangoedecke.com/they-really-do-think-ai-might-kill-everyone/ https://www.seangoedecke.com/they-really-do-think-ai-might-k...
- jordanb 4d ago
  OK understood. But you understand this makes them sound like the type of person who doesn't believe any of the "worldly concerns" are worth addressing because "the end is near" right?
  - howunfortunate 4d ago
    There's no logical error here if the end is actually near. So it all hinges on an empirical disagreement you have with them. There's nothing particularly unserious about that.
    - notduncansmith 4d ago
      “It all hinges on [an entirely presupposed conclusion that we can’t prove is any more likely than the opposite]” is about as unserious as it gets lol
    - imtringued 4d ago
      Climate change is inescapable so a vengeful AI god would pull that lever first since it has both a perverse incentive to generate more power and make the planet uninhabitable for humans in a way that cannot be overcome even if humanity fought back against the AI and regained its freedom.
    - i5heu 4d ago
AlotOfReading 4d ago
It's the problem of the banality of evil. Stopping someone from dramatically pressing a big red button doesn't solve humanity's largest problems because that's not what caused them. We need to stop millions "boring" actions done by systems blindly following instructions without regard for the consequences.
api 4d ago
One of the times I regret that I have only one upvote.
howunfortunate 4d ago
I am worried about those things too. But those things fall in a category of "things that are awful and I'd like to see solved", which is different than "existential risks which could see my kids dead, and there's nothing I can personally do to shield them from it".
A1kmm 4d ago
And the real threats are mostly economic and environmental - the concentration of the means of production in the hands of few who use it to exploit us all, and a surge in energy usage accelerating climate change. Even the sci-fi scenario assumes there is a discrepancy of capability between attacker or defender. If the 'attacking' system is (by some reasonable measure), 1000% as capable as a human, and the 'defending' systems are 60%, then it is a problem. If the 'attacking' system is 1000% as capable as a human, but there are hundreds of thousands of systems that are 900% as capable as a human, it's probably not going to take over everything successfully. So unequal distribution of AI technology, and lax regulation and opacity of the biggest companies which actually make the risks the worst. I don't think the "AI Safety" people are "unserious and out of touch" - I think they are actively making AI Safety problems worse by being advocates for consolidation of AI development and lack of transparency.
- BobbyTables2 4d ago
  I suspect they know exactly what they are doing. And the non-techies who have seen Terminator and other movies blindly line up behind them…
- jefftk 4d ago
  With some technologies, if you match attacker and defender effort you're safe, but not all of them. Nukes are the traditional example: if every person was a "nuclear power" we'd be dead within the day. But luckily there are bottlenecks to nuke creation that we can track. What worries me the most (enough that I left tech to work on this full time) is bio. The amount of effort needed to defend against a pathogen can be many orders of magnitude higher than the effort needed to create it, and the upstream bottlenecks are both less limiting and mostly undefended. We've been safe so far because bio is very hard, but "rare expertise" as a gatekeeper is on its way out.
saurik 4d ago
Doesn't it feel like the exact opposite? If you care a lot about the AI being a vengeful god, you do not do what OpenAI and Anthropic are doing. These companies only pay lip service to the idea of that aspect of AI safety in the hope that they can use it to regulate open source AI out of existence. Instead, they are focused on stuff like "can I ask the AI to help me build a nuclear bomb" or "is the AI willing to generate pornographic stories", which is neither trying to protect us from unleashing a vengeful god NOR preventing (in any honest way) the today-level problems you (very correctly) bring up.
otabdeveloper4 4d ago
"The AI is a vengeful god" is a marketing campaign ran by AI companies. And why not? It seems there is, indeed, one born every minute, and technological progress only accelerates this phenomenon.
charlieyu1 4d ago
Like most pressure groups and HR departments, they create shitstorms to justify their existence.

xena 4d ago

I think you accidentally a word. Did you mean: > I've yet to have a logical discussion with anyone who thinks the "AI Safety" people should be in charge and I think they just truly don't [know] that what most them actually want is to be the one holding the keys to power.

deleted 4d ago

[deleted]

dorolow 4d ago

I am an AI Safety Person and I want the government to nationalize or have a significant stake in the frontier labs and to have democratic control of the development of the technology. The AI Safety movement is not a monolith. I do not think Eliezer Yudkowsky nor his acolytes should hold the reins, but a lot of folks sure like to create a strawman that anyone who wants to regulate OpenAI is somehow an EA/MIRI weirdo

bigyabai 4d ago
> and to have democratic control of the development of the technology Why? I don't care about democratically participating in a closed model's development. It doesn't belong to me. China will develop whatever they want, a federal stake in OpenAI or Anthropic punishes Americans and shields US labs from legitimate competition.
- dorolow 4d ago
  In my mind, democratic control of the technology means that we (the govt, or other empowered agency) take ownership of their assets and IP, solve or find a level of alignment or guardrails that society is comfortable with. Then we distribute the technology, or access to it, to avoid power concentration. This would also certainly require international coordination with China on a slowdown or pause, which I think is possible.
  - pibaker 4d ago
    > This would also certainly require international coordination with China on a slowdown or pause Personally I think the country with a strictly meritocratic elite selection system that also just outright kills you if you sell weed will have a hard time sympathizing with Bay Area thinkers who talk about AI killing us all during their ayahuasca breakfast before returning to their meth fueled crunch towards releasing the next version of the AI that will kill us all.
    - dorolow 4d ago
      You're in luck, neither I nor anyone else want those people near a negotiation table.

perching_aix 4d ago

I'm yet to have a productive conversation with anyone who whinges about not being able to have a logical discussion about things they feel strongly about, but given that safety and control are interchangeable when the intentions are removed, this comes across as a particularly demagogue depiction of the subject matters involved. A lack of control is not equivalent to freedom, the same way the totality of it is not equivalent to tyranny. There's a reason we have separate words for these things. This constant motivated conflation of the two is beyond grating. You're crying wolf until nobody believes you when they should. Don't go acting all surprised when that happens. The issue is with the ownership of control, not necessarily with control. Attacking the latter sidesteps this rather than address it.

armchairhacker 4d ago

I trust the AI 2040 people, because they’ll make an implementation that at least convincingly doesn’t actually put them (or anyone) in charge. Maybe we shouldn’t have AI safety, but if we’re going to trust anyone (besides yourself), who’s more qualified? And it makes me suspicious when anyone brings up safety and doesn’t address the elephant in the room, that they’re at least unaware of the sprawling edge-cases.

cmrdporcupine 4d ago

I had an extensive argument with someone on FB yesterday who was very much on the "I trust Dario most and I want him to be in charge" side. I find it hard to accord with this world view at all or understand how it can come about. But it's especially troubling as a non-American to see people talk about "we can't risk AI falling into the hands of hostile authoritarian states" while advocating that AI be regulated and monopolized by the current US corporate/gov't mixture.. which is now taking the form of a hostile and authoritarian state w/ Silicon Valley as its bedmate.