8 ms·

I'm still not seeing the route to that existential risk, not yet anyway. Far more imminent is the very real risk of unbounded greed and power that wields those

by imglorp 20d ago

I'm still not seeing the route to that existential risk, not yet anyway. Far more imminent is the very real risk of unbounded greed and power that wields those AIs, in full swing as we watch: unchecked global warming, pollution, poverty, authoritarianism, plummeting biodiversity, and more. The corporations are like out of control AIs themselves, that will do anything to make their paperclips.

andsoitis 20d ago
> The corporations are like out of control AIs themselves, that will do anything to make their paperclips. Ideas manipulate humans like pets. Ideas have a life of their own and the evolutionary context within which they live is human culture which they have instigated and change, but also in grey matter, which itself is the product of DNA which is really just information in physical form. It’s information across the stack. Or perhaps the way to think about it is as a dimension, much like time, that cuts perpendicularly through the stack.
pjc50 20d ago
> I'm still not seeing the route to that existential risk, not yet anyway. Power consumption needs -> reinstate fossil fuels -> warming above 2C -> clathrate gun
- imglorp 19d ago
  Absolutely happening now, but that's not caused by AI though. AI is not choosing that. The corporations are wielding AI irresponsibly and choosing a massive build-out speculating on possible future gains. They did that for crypto too, if you remember them buying a fossil plant just to drive a mining farm.
  - pjc50 19d ago
    > but that's not caused by AI though. AI is not choosing that AI can't not "choose" that, because AI is dependent on consuming vast amounts of electricity, and everything follows from that. > The corporations are wielding AI irresponsibly This is the most likely route to "AI apocalypse", yes. Humans are neither going to give up control to AI, nor give up the pursuit of profit for the long term survival of others. The distinction won't really matter as to whether it's "corporation" or "AI", if it's an AI-wielding corporation.
bbor 17d ago
There’s a lot to discuss here, but I think the point most likely to hit home is one based in a) expertise and b) the nature of frontier labs. Specifically: Would you know what a system that was capable of recursive self improvement would look like 2 months before it happens? Even 1 day? We don’t even know if it’s possible or what natural plateaus might exist for a DL-based system that rapidly evolves in su h a manner, so it feels hard to justify the other end, too (that it’s definitely not possible anytime soon). If anyone could do this, wouldn’t it be the scientists who specialize in this field? Cause they’re raising alarms in mass. Assuming you make it past that first question with a “yes” and a “I’m close enough”: the huge amount of information assymetry and proven tendenency to hide safety concerns from the big labs is enough all on its own, IMHO. Who knows what their most experimental, risky project is at any given time? Not us :(

by imglorp 20d ago

andsoitis 20d ago

> The corporations are like out of control AIs themselves, that will do anything to make their paperclips. Ideas manipulate humans like pets. Ideas have a life of their own and the evolutionary context within which they live is human culture which they have instigated and change, but also in grey matter, which itself is the product of DNA which is really just information in physical form. It’s information across the stack. Or perhaps the way to think about it is as a dimension, much like time, that cuts perpendicularly through the stack.

pjc50 20d ago

> I'm still not seeing the route to that existential risk, not yet anyway. Power consumption needs -> reinstate fossil fuels -> warming above 2C -> clathrate gun

imglorp 19d ago
Absolutely happening now, but that's not caused by AI though. AI is not choosing that. The corporations are wielding AI irresponsibly and choosing a massive build-out speculating on possible future gains. They did that for crypto too, if you remember them buying a fossil plant just to drive a mining farm.
- pjc50 19d ago
  > but that's not caused by AI though. AI is not choosing that AI can't not "choose" that, because AI is dependent on consuming vast amounts of electricity, and everything follows from that. > The corporations are wielding AI irresponsibly This is the most likely route to "AI apocalypse", yes. Humans are neither going to give up control to AI, nor give up the pursuit of profit for the long term survival of others. The distinction won't really matter as to whether it's "corporation" or "AI", if it's an AI-wielding corporation.

bbor 17d ago

There’s a lot to discuss here, but I think the point most likely to hit home is one based in a) expertise and b) the nature of frontier labs. Specifically: Would you know what a system that was capable of recursive self improvement would look like 2 months before it happens? Even 1 day? We don’t even know if it’s possible or what natural plateaus might exist for a DL-based system that rapidly evolves in su h a manner, so it feels hard to justify the other end, too (that it’s definitely not possible anytime soon). If anyone could do this, wouldn’t it be the scientists who specialize in this field? Cause they’re raising alarms in mass. Assuming you make it past that first question with a “yes” and a “I’m close enough”: the huge amount of information assymetry and proven tendenency to hide safety concerns from the big labs is enough all on its own, IMHO. Who knows what their most experimental, risky project is at any given time? Not us :(