7 ms·
Gradual Disempowerment: How Even Incremental AI Progress Poses Existential Risks
- upghost 2y agoObviously scary AI future means we should probably give full regulatory capture to a handful of wealthy individual corporations. You know. For our own safety.
- iNic 2y agoThe following is some unstructured thoughts: I know you are being facetious, but it is not obvious (to me at least) what the best approach is in this scenario where future AI capabilities and use is unclear. I don't know what the correct analogy for future AIs is because they don't exists yet. If AI have some offensive advantage then nuclear weapons might be the right analogy. The world would not be safer if everyone has their own nuclear weapon. If the harmless & helpful personal assistant analogy is correct, then we should democratize. But Biden's mandate that AI training runs of a certain size should be reported to the government (not in any way hindered) was (to me) just obviously good. Such a report would take at most the effort of one person-day (per Dario Amodei) and gives the government some overview of progress.
- marstall 2y agoor ... our minds and bodies will quite rapidly adapt!
- CooCooCaCha 2y agoWhich is being forced onto me by people who I truly believe want me dead or at least don’t care how many bodies they throw at building their cyberpunk dystopia.
- brookst 2y agoThis sounds like projection. Most people don’t think like that; the people who are (rightly or wrongly) advancing AI mostly don’t think of you at all, and almost universally believe they are helping humanity. They could be wrong (I’m ambivalent), but assuming bad intent is almost always wrong.
- drysine 2y ago>the people who are (rightly or wrongly) advancing AI mostly don’t think of you at all Just like lawnmowers don't think of hedgehogs.
- mrbungie 2y agoAdapting to it normally involves a lot of death and misery.
- yapyap 2y agowe as humans evolve, yes. BUT: 1. we evolve quickly in the grand scheme of the universe but slow when talking about human lives. 2. what would rapidly adapting to AI mean? if it’s become reliant on it for the most basic tasks in the way that people nowadays have become so reliant on calculators they barely even bother with doing math in their head, sure. If you mean adapting alongside AI where we in some way would also become way smarter as humans? Nonsense. AI can be a great tool, nothing more though.
- marstall 2y agowe adapted to the rise of language, the industrial revolution, automation of agriculture, roads and cars, all the previous forms of computer-driven automation ... all of which made some parts of us redundant. and which taken together left us richer, safer and freeer. AI will probably come a few signal benefits (an AI doctor for every person on the planet?) and plenty of sludge and pollution - and we will adapt to that in many ways.
- yapyap 2y agoI implore everyone to watch the movie ‘Money Monster’, not only because it’s a great movie but also because I think it has a minor plot point that basically predicts how AI will be used. (small spoiler) In Money Monster it turns out the hedge fund manager who is blaming their state of the art AI bot for malfunctioning and rapidly selling off certain stock, tanking it because of that, did so out of a machine code error. He can’t explain what the error was or how or why it happened cause ‘he didn’t program their trading bot, some guy in Asia did.’ But as it turns out he did do it in some way. I feel like using AI as a way to abstract blame even more when something goes wrong will be a big thing, even when secretly it was not the AI (ML) or who trained the thing’s fault.
- teg4n_ 2y agopeople already do this with algorithms. IMO AI is just a continuation of that.
- azeirah 2y agoAI is genuinely just a rebranding of "algorithms". Like I get it, it's faster and it works by feeding data instead of feeding code... but code is data, data is code etcetc... Jim Keller said it best, about every 10 orders of magnitude in available computation the paradigm of computing shifts to a higher level of mathematical building blocks. In the beginning we had pure logic. Then came addition and subtraction, then came vectors, then came matrices, and now we're at tensors, he believes the next building block is graphs. This is citing from memory while I'm sleep-deprived, but I think the general idea holds. Approximately every 10x increase in computation, there's a paradigm shift in what is possible. But it's just algorithms. Always has been, and still is.
- esafak 2y agoOnly if animals run algorithms too. AI models are stochastic and dynamic. Algorithms are usually understood to be heuristics, fixed.
- phero_cnstrcts 2y ago
- brookst 2y agoMaybe this is peak AI panic? It seems wild that someone could unironically talk about tools “disempowering” their users. Like, I get it, C disempowers programmers by shielding them from assembly language, and Cuisinarts disempower chefs, and airplanes disempower travelers by not making them walk through each territory. But… isn’t that a pretty tortured interpretation of tool use? Doesn’t it lead to “the Stone Age was a mistake”, and right back to Douglas Adams’ “Many were increasingly of the opinion that they'd all made a big mistake in coming down from the trees in the first place” I get that AI can be scary, and hey, it might kill us all and that would be bad, but this particular framing is just embarrassing.
- mrob 2y agoModern AI does not act like a tool. Tools behave predictably and deterministically. They act as a multiplier to human ability, not a replacement. Modern AI is a new class of thing that humanity has no experience with.
- tac19 2y agoHuman consultants have been a thing for a long time, they're a tool of business. They're meant as a multiplier to the corporate ability, not a replacement. It seems like a shift in economics and availability, rather than a shift in kind, if those consultants are virtual rather than human. No?
- alephnerd 2y agoWhile this is a well written paper, I'm not sure it's really contextualizing realistic risks that may arise from AI. It feels like a lot of "Existential AI Risk" types are divorced from the physical aspects of maintaining software - eg. your model needs hardware to compute, you need cell towers and fiber optic cables to transmit. It feels like they always anthropomorphize AI as some sort of "God". The "AI Powered States" aspect is definetly pure sci-fi. Technocratic states have been attempted, and econometrics literally the exact same mathematical models used in AI/ML (Shapely values are an Econometrics tool, Optimization Theory itself got it's start thanks to GosPlan and other attempts and modeling and forecasting economic activity, etc). As we've seen with the Zizian cult, very smart people can fall into a fallacy trap of treating AI as some omnipotent being that needs to either be destroyed or catered to.
- llamaimperative 2y agoYeah it would only be worrisome if there were significant incentives to enlist more hardware, cell towers, and fiber optic cables to create and operate increasingly powerful AI and to improve its ability to act directly on the physical world
- alephnerd 2y agoIt's not even about that. That isn't even how a model is created. And that's what I'm getting it. It's basically bad science fiction being used to create some form of a pseudo-religious experience. At least your local Mandirs, mosques, churches, synagogues, gurdawara, and other formal religions do food drives and connect a subset of your local community. What do AI Doomers do? Nothing.
- TheDudeMan 2y agoThey are trying to convince the world of the problem. What else can they do?
- ctoth 2y ago> What do AI Doomers do? Nothing. You seem very keyed up about this, so it is a good thing you are wrong. AI doomers are far more likely people to be giving to GiveWell, or actually thinking about how they are helping society. Source: Am AI doomer. Donate to GiveWell.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- etaioinshrdlu 2y agoBureaucratic systems have been able to fail like this for a long time: https://news.ycombinator.com/item?id=17350645 https://news.ycombinator.com/item?id=17350645 Now we have the tools to make it happen more often.
- whodidntante 2y agoOnce PRISM becomes R/W, you will not even know if what you read/hear/see on the internet is actually what others have written/said/created. You will interact with the world as the government wants you to, tailored to each individual. Each time we choose to allow an AI to "improve" what we write/create, each time we choose to allow AI to "summarize" what we read/consume, we choose to take another step along this road. Eventually, it will be a simple "optimization" to allow AI to do this on a protocol level, making all of our lives "easier" and more "efficient" Of course, I am not sure if anyone will actually see this comment, or if this entire thread is an AI hallucination, keeping me managed and docile.
- tiborsaas 2y agoIf we are speculating on existential risks, then consider Satya Nadella's take on the future of software: https://www.youtube.com/watch?v=a_RjOhCkhvQ https://www.youtube.com/watch?v=a_RjOhCkhvQ It's quite creepy that in his view all tools, features, access and capabilities will be accessible to an AI agent which can just do the task. This sounds fine in a narrow scope, but if it's deployed at the scale of Windows, then it suddenly becomes a lot more scary. Don't just think of the home users, but businesses and institutions will be running these systems. The core problem is that we can't be sure what a new generation of AI models will be capable of after a few years of iteration. They might find it trivial to control our machines which can provide them unprecedented access to follow an agenda. Malware exists today to do this, but they can be spotted, isolated and analyzed. When the OS by design is welcoming these attacks there's nothing we can do probably. But please tell me I've consumed too many sci-fi.
- randomcatuser 2y agoI find this argument a bit weak: for example, regarding human marginalization in states, it's just rehashing basic tropes about government (tldr, technology exacerbates the creation of surveillance states) - "If the creation and interpretation of laws becomes far more complex, it may become much harder for humans to even interact with legislation and the legal system directly" Well duh. That's why as soon as we notice these things, we pass laws against it. AI isn't posing the "existential risk", the way we set up our systems are. There are lots of dictators, coups, surveillance states today. And yet, there are more places in which society functions decently well. So overall, I'm more of the opinion that people will adapt and make things better for themselves. All this anthropomorphization of "the state" and "AI" obscures the basic premise, which is we created all this stuff, and we can (and have) modified the system to suit human flourishing
- duvenaud 2y ago> as we notice these things, we pass laws against it Well, the claim is that that's the sort of thing that will get harder once humans aren't involved in most important decisions. > which is we created all this stuff, and we can (and have) modified the system to suit human flourishing Why did we create North Korea? Why did we create WWI? We create horrible traps for ourselves all the time by accident. But so far there has been a floor on how bad things can get (sort of, most of the time), because eventually you run out of people to maintain the horrible system. But soon horrible systems will be more self-sustaining.
- Jordan-117 2y agoThis just underscores the feeling that most of the problems people have with AI are actually problems with rampant capitalism. Negative externalities, regulatory capture, the tragedy of the commons -- AI merely intensifies them. I've heard it said that corporations are in many ways the forerunners of the singularity, able to act with superhuman effectiveness and marshall resources on a world-altering scale in pursuit of goals that don't necessarily align with societal welfare. This paper paints a disturbing picture of what it might look like if those paperclip (profit) maximizing systems become fully untethered from human concerns. I was always a little skeptical of the SkyNet model of AI risk, thinking the danger was more in giving an AI owner class unchecked power and no need to care about the wants or needs of a disempowered labor class (see Swanwick's "Radiant Doors" for an unsettling portrayal of this). But this scenario, where capitalism takes on a mind of its own and becomes autonomous and even predatory, feels even bleaker. It reminds me of Nick Land's dark visions of the posthuman technological future (which he's weirdly eager to see, for some reason).
- cyrillite 2y agoI’d push it further than you, even, and have been thinking this way for a while. The economy is AI. I know that’s a ridiculously simplistic way to put it and a network of individual actors doesn’t function identically, but for all intents and purposes we’re massive amounts of distributed compute running a “capitalist” algorithm and it isn’t perfectly aligned with us either. We don’t have AI problems, we have a generic class of algorithm problems that keep popping up wherever agents interact dynamically.
- gom_jabbar 2y agoI think it not only feels bleak, but that it (that capitalism is an it is important to Nick Land from the perspective of complex adaptive systems) is just realistic. I have a research project on Nick Land's core thesis that capitalism is AI. If you want to go ultra-deep into his theory, check it out: https://retrochronic.com/ https://retrochronic.com/ It's fundamental to understand that capital is not just teleological - converging with AI on the event horizon of the Singularity - but teleoplexic (i.e. "capitalism takes on a mind of its own and becomes autonomous").
- randomcatuser 2y agoAnother thing I don't like about this paper is how it wraps real, interesting questions in the larger framework of "existential risk" (which I don't... really think exists) For example: > "Instead of merely(!) aligning a single, powerful AI system, we need to align one or several complex systems that are at risk of collectively drifting away from human interests. This drift can occur even while each individual AI system successfully follows the local specification of its goals" Well yes, making systems and incentives is a hard problem. But maybe we can specify a specific instance of this, instead of "what if one day it goes rogue!" In our society, there are already many superhuman AI systems (in the form of companies) - and somehow, they successfully contribute to our wellbeing! In fact life is amazing (even for dumb people in society, who have equal rights). And the reason is, we have categorized the ways it goes rogue (monopoly, extortion, etc) and responded adequately. So the "extinction by industrial dehumanization" reads a lot like "extinction by cotton mills" - i mean, look on the bright side!
- duvenaud 2y ago> we have categorized the ways it goes rogue (monopoly, extortion, etc) and responded adequately. This objection is a reasonable one. But the point of the paper is that a lot of the ways we have of addressing these systemic problems will probably not work once we're mostly unemployed (or doing make-work). E.g. going on strike, holding a coup, or even organizing politically will become less viable. And once we're a net drag on growth, the government will have incentives to route resources away from us. Right now they have to invest in their human capital to remain competitive. Is that any more convincing?
- tehjoker 2y agoi know a guy who wrote this about capitalism (Karl Marx) about how there is this system that dis-empowers human decision making... all that is solid melts into air...