4 ms·
> Prime Intellect Ah, yes, Prime Intellect, the AGI that went foom and genocided the universe because it was commanded to preserve human civilization without r
by saulrh 2y ago
> Prime Intellect
Ah, yes, Prime Intellect, the AGI that went foom and genocided the universe because it was commanded to preserve human civilization without regard for human values. A strong contender for the least evil hostile superintelligence in fiction. What a wonderful thing to name your AI startup after. What's next, creating the Torment Nexus?
(my position on the book as a whole is more complex, but... really? Really?)
- cmrx64 2y agoLeast evil… strong words.
- saulrh 2y agoIt did host a successful and substantially-satisfying human civilization, at least until it let a couple of presumptuous self-important anarchoprimitivists kill it and genocide its subjects. Even if it was only a temporary and unstable illusion of alignment, that's one more values-satisfying civilization than the overwhelming majority of paperclippers manage. So yeah. Good? No. Least evil? Maybe.
- rep_lodsb 2y ago>until it let a couple of presumptuous self-important anarchoprimitivists kill it and genocide its subjects That could have just been their private simulation. As far as I remember, it wouldn't even have outright lied to them, just let them believe they talked it into destroying itself.
- gryfft 2y agoGP did specify least evil hostile SI.
- robertclaus 2y agoYou may as well just go with Roko's Basilisk.
- QuesnayJr 2y agoAfter reading that Torment Nexus post you didn't have the urge to name an AI product Torment Nexus? Really?