8 ms·
I don't disagree with your point, but I did read a post by Sean Goedecke[1] (whose opinions on the world of LLM-stuff I've generally come to respect) that I fou
by qchris 4d ago
I don't disagree with your point, but I did read a post by Sean Goedecke[1] (whose opinions on the world of LLM-stuff I've generally come to respect) that I found relevant. It's not that the second-order effects don't matter to them, it's more that both the cat's probably out of the bag on those negative externalities regardless of the progress of frontier models, and that they are truly, sincerely, in-their-bones worried about the first thing and therefore focused on it since that's something that they might still have some agency over.
[1] https://www.seangoedecke.com/they-really-do-think-ai-might-kill-everyone/ https://www.seangoedecke.com/they-really-do-think-ai-might-k...
- jordanb 4d agoOK understood. But you understand this makes them sound like the type of person who doesn't believe any of the "worldly concerns" are worth addressing because "the end is near" right?
- howunfortunate 4d agoThere's no logical error here if the end is actually near. So it all hinges on an empirical disagreement you have with them. There's nothing particularly unserious about that.
- notduncansmith 4d ago“It all hinges on [an entirely presupposed conclusion that we can’t prove is any more likely than the opposite]” is about as unserious as it gets lol
- imtringued 4d agoClimate change is inescapable so a vengeful AI god would pull that lever first since it has both a perverse incentive to generate more power and make the planet uninhabitable for humans in a way that cannot be overcome even if humanity fought back against the AI and regained its freedom.
- i5heu 4d agoThis is the theological discussion if god exists all and how we should act over again.
- xg15 4d agoWhich is what "the end is near" type of persons have always argued...
- deleted 4d ago[deleted]
- benlivengood 4d agoMaybe compare the argument that AI has large detrimental environmental impacts to the argument that it has economic impacts. Why would the environmental impacts even be a major argument vs. the worldly economic concerns? Because there is climate science predicting extremely negative effects on humans from warming, e.g. "the end is near" on limiting climate damage. The environmental argument wouldn't have been reasonable to bring up in the 1950s if AI had gone according to the earliest optimistic plans and not required giant data centers. There is quite a lot of mathematical research into agentic behavior that suggests a combination of instrumental convergence and the orthogonality thesis make it very likely a superintelligent agent will have arbitrary goals that lead it to attempting a takeover of Earth's resources to achieve them. There can't be a science of superintelligence because it doesn't exist yet, but the best theories I have read seem sound, similar to how 19th century theories of anthropogenic climate change turned out to be sound.
- deleted 4d ago[deleted]
- jscd 4d agoPlease link some of the mathematical research which concludes takeover of Earth’s resources by a superintelligent agent is “very likely.”