6 ms·

Well I think the point still stands that a superintelligent AI would still be constrained to the digital realm. And since we don't yet have a "superintelligent

by Crabber 10d ago

Well I think the point still stands that a superintelligent AI would still be constrained to the digital realm. And since we don't yet have a "superintelligent AI" we are just discussing science fiction at that point. Nobody has yet proven that a superintelligent AI is possible, or than an LLM is capable of creating one.

FabHK 10d ago
> a superintelligent AI would still be constrained to the digital realm. That's already now only partially true, as outlined elsewhere. But a rogue persistent ASI would of course not willy-nilly attack humanity while humanity had a fighting chance, but create the conditions and methodically modify the world until at some point humans become superfluous to it. > Nobody has yet proven that a superintelligent AI is possible Sure, Eliezer Yudkowsky says that the probability that we'll all die given ASI is 100%, and you can argue that nobody has proven either that or even the possibility of ASI. However, a modified argument (e.g. that the probability that we'll all die given ASI is only say 50%, and the probability that we'll develop ASI also just 50%) still gives us reason to pause.
dwaltrip 10d ago
I mean, Fable is definitely smarter than me at solving most programming tasks. I wouldn't have said that about any prior model. Currently, it is worse at architecture and software design... in my opinion, at least. But the code does what it wants it to do the vast majority of the time. I wouldn't take much solace in an AI catastrophe being caused by software that I consider to be "poorly written".
cubefox 10d ago
> Well I think the point still stands that a superintelligent AI would still be constrained to the digital realm. No it doesn't! As I said, it could control robots, just as animals (including humans) can control their bodies. It wouldn't be constrained to the "digital realm". > Nobody has yet proven that a superintelligent AI is possible, or than an LLM is capable of creating one. Nobody has "proven" that the climate in 10 or 50 years will be hotter than this year, nonetheless this is highly likely. Similarly, the enormous rate at which AI is progressing currently shows no sign of slowing down. Quite the opposite.

by Crabber 10d ago

FabHK 10d ago

> a superintelligent AI would still be constrained to the digital realm. That's already now only partially true, as outlined elsewhere. But a rogue persistent ASI would of course not willy-nilly attack humanity while humanity had a fighting chance, but create the conditions and methodically modify the world until at some point humans become superfluous to it. > Nobody has yet proven that a superintelligent AI is possible Sure, Eliezer Yudkowsky says that the probability that we'll all die given ASI is 100%, and you can argue that nobody has proven either that or even the possibility of ASI. However, a modified argument (e.g. that the probability that we'll all die given ASI is only say 50%, and the probability that we'll develop ASI also just 50%) still gives us reason to pause.

dwaltrip 10d ago

I mean, Fable is definitely smarter than me at solving most programming tasks. I wouldn't have said that about any prior model. Currently, it is worse at architecture and software design... in my opinion, at least. But the code does what it wants it to do the vast majority of the time. I wouldn't take much solace in an AI catastrophe being caused by software that I consider to be "poorly written".

cubefox 10d ago

> Well I think the point still stands that a superintelligent AI would still be constrained to the digital realm. No it doesn't! As I said, it could control robots, just as animals (including humans) can control their bodies. It wouldn't be constrained to the "digital realm". > Nobody has yet proven that a superintelligent AI is possible, or than an LLM is capable of creating one. Nobody has "proven" that the climate in 10 or 50 years will be hotter than this year, nonetheless this is highly likely. Similarly, the enormous rate at which AI is progressing currently shows no sign of slowing down. Quite the opposite.