6 ms·
How exactly does that work? The nuclear system of MAD relies on physical threat, lab A achieving ASI (artificial scary intelligence) does not prevent lab B achi
by pjc50 7d ago
How exactly does that work? The nuclear system of MAD relies on physical threat, lab A achieving ASI (artificial scary intelligence) does not prevent lab B achieving it.
I would like everyone involved to be a lot clearer about their threat models, with plausible series of clearly linked steps, rather than just sounding like a Vernor Vinge novel.
- deleted 7d ago[deleted]
- 568236at 7d ago[dead]
- ralfd 7d agoThe idea in AI Risk circles is that the first true Scary Intelligence wins, as it can recursively advance itself and exponentially outgrows/hack any following (weaker) B or C intelligence.
- mycocola 7d agoLab A reaches ASI, and is prompted the following: "Permanently nullify all other AI labs".If it's ASI it would succeed.
- pjc50 7d agoI am explicitly asking people to fill in the blank on how it would succeed. It's intelligence, not magic. A lot of these scenarios seem to assume that nobody else gets a move. That there wouldn't be a human response.
- a2ff6eeb0 7d agoWe're currently putting it into all sorts of critical systems, from logistics to power. It could just stop running them on our behalf.
- alach11 7d ago"If we can do all this, we will have a world in which democracies lead on the world stage and have the economic and military strength to avoid being undermined, conquered, or sabotaged by autocracies, and may be able to parlay their AI superiority into a durable advantage." [0] I couldn't agree more with you. Dario throws out the term "durable advantage" in many of his talks and essays without seriously addressing what that would entail. It's not pretty imagining the mechanics of converting an intelligence advantage into a permanent power imbalance. I'd much rather we pursue international cooperation on a slowdown. [0] https://darioamodei.com/essay/machines-of-loving-grace https://darioamodei.com/essay/machines-of-loving-grace
- mycocola 7d ago- Manufacture new hacking incidents at other AI labs, proving that lab A is the only one capable of creating aligned AI - Credible death threats to individuals working at data centers, coercing them to enable attack vectors - To distract the world, enable a terrorist attack by delivering intelligence to specific groups who can act on it - Create a network of individuals vulnerable to leverage to be used as tools for enabling the ASI to act on the real world - Convince Lab A workers to display to the public that progress at Lab A is slowing down, and during this time recursively self improve until the task can be completed
- deleted 7d ago[deleted]