6 ms·

Can we finally talk about this site, and how it relates to these cults? There's a number of lesswrong, e/acc, and other pseudo "rationalist" blogs that get shar

Can we finally talk about this site, and how it relates to these cults? There's a number of lesswrong, e/acc, and other pseudo "rationalist" blogs that get shared and upvoted on this site. Most of their assumptions go unchallenged. Not saying they shouldn't be read, or debated. But their writing should be viewed in context - it's fringe stuff, written by people from a peculiar subculture with values out-of-whack with most people. I'd like to make some modest assertions, to push back on fringe ideas I've seen repeated here: 1. For "the singularity" to happen, we probably need something more to happen than just chatGPT to ingest more data or use more processing power. 2. Even if, somehow, chatGPT turned into skynet, we'd hopefully be able to unplug the computer. 3. If you want to save lives, it's probably more useful to think about lives saved today than hypothetical lives you could save 100 years from now. Not that you shouldn't consider the long term, but your ability to predict the future gets worse and worse the farther you project out. 4. If you want to save lives, it's probably more useful to save actual lives, than say, hypothetical simulated lives that exist inside of a computer. 5. The argument that "we're killing more people by delaying time inventing the hypothetical life saving technology" is not very useful either, because you can't actually say how many lives would be saved versus harmed. And mostly it just sounds like a front for "give me more money and less regulations". 6. Reading a bunch of science fiction and getting into fights on an internet forum is not a substitute for education and experience. Unless you've spent a good time studying viruses, you are not a virologist, and while the consensus among virologists can be wrong, you should have the intellectual humility to realize that you are probably not equipped to tell, unless you have expertise in the field. 7. Anything that smacks of eugenics is most likely pseudoscience. 8. If someone talks like a racist / sexist / nazi, or acts like a racist / sexist / nazi, they probably are one. It's probably not a joke, or a test.

AnimalMuppet 3y ago
Well, this site often deals with fringe stuff. Take Lisp, for example. It is a fringe language, no matter how zealous its advocates are. So is Haskell. But you see articles about them posted here far out of proportion to their real-world use and impact. And I disagree that "most of their assumptions go unchallenged". I see challenges to the assumptions of lesswrong, e/acc, and other such stuff often here. (Maybe less often than is warranted, but still fairly often.) "It is the mark of an educated mind to be able to entertain a thought without accepting it." - often attributed to Aristotle, but apparently not actually from him. Still a good thought. We can post about this stuff, and discuss it, without buying it. I agree with your pushbacks.
two_in_one 3y ago
> 1. For "the singularity" to happen, we probably need something more to happen than just chatGPT to ingest more data or use more processing power. It's not actually clear what "the singularity" is? Is it something running out of control or it's still controllable? There is a blurry line. People are afraid because they think it's sort of uncontrollable explosion. The second question is about AGI. What is it? Is it something 'alive' or just a generic AI calculator with no 'creature' features. Like self preservation at least. I think our view of these two things will change soon as we get a close up picture. Pretty much like Turing test doesn't look great anymore. As even dumb chatbots can pass.
- deleted 3y ago
  [deleted]
- com2kid 3y ago
  I personally define AGI as a technology capable of improving itself exponentially. But I realize my definition is in the minority. :/ Of course if we ever manage to make a 1:1 cybernetic brain that works exactly like a human's brain, and is also a complete black box, we'll have achieved AGI. I'm not sure how useful that will be, but I'll have to admit it is AGI. So maybe I should say, "interesting AGI" is technology that can improve itself exponentially. :-D
  - greysphere 3y ago
    Yeah, if you could input data set X with quality Q(X) and output data set Y of the same size with Q(Y) > Q(X), you'd really be on to something. But I don't think such a system exists yet, or even close. Inputting the internet, outputting a sea of garbage with a handful of diamonds that people have to spelunk through the garbage for seems the best so far. Madlibs is a pretty equivalent activity, and while fun, certainly not anything one would consider AGI. We need a revolutionary improvement on automated spelunking to get anywhere. Maybe we'll get a good spam filter as a side effect! But even if you had a system, there's still the resource cost to run the algorithm (needs to be bounded or else you've just made a finite jump) and the gains you make need to not decay (or again you've just made a finite jump). And all this needs to be compared against investing in humans - which seem pretty clearly to have AGI properties (but with some really bad constant factors. 20 year training time - ridiculous!) To me it seems things are a long way off and least a couple of major innovations away. But at least there's some ideas of the problems to tackle, which is a big step up!
com2kid 3y ago
> If you want to save lives, it's probably more useful to think about lives saved today than hypothetical lives you could save 100 years from now. Not that you shouldn't consider the long term, but your ability to predict the future gets worse and worse the farther you project out. This reasoning, taken to its logical extent, would mean completely defunding all medical research and instead redeploying all research to best utilizing our current technology. That is, hopefully, an obviously faulty decision. We must balance research with using current findings to help people. Stopping all drug development and telling people with currently untreatable diseases "haha, too bad, we aren't even going to try", is cruel. The ultimate goal of many technology utopians is the end of all death. We have examples from the natural world of incredibly long lived higher life forms, and simple life forms that are immortal, so it isn't scientifically impossible. The number of lives saved is literally, infinite. It is really hard to argue with "infinite upside". (The absolute societal shit show that such a technology would bring about is a rather important question...) > "we're killing more people by delaying time inventing the hypothetical life saving technology" is not very useful either, because you can't actually say how many lives would be saved versus harmed. For many future inventions, of course estimates can be made of the number of lives saved. The more pie in the sky research is, the larger those error bars become. "LLMs may one day help us sift through research and cure all forms of cancer" is, technically true, but the error bars on it are so high that you don't want to devote all the world's resources to LLMs just on the hopes of eliminating all forms of cancer 50+ years from now. "This company is working on AI Vision+Robotics technology so people don't have to do this super hazardous job that claims a bunch of lives. They are an estimated 5 years from productizing and they already have purchase contracts in place" is a much different statement.
financltravsty 3y ago
Hey, buddy (yes, I am intentionally being condescendingly familiar) these cults/movements are self-serving and cannot be reasoned with via rationality under the assumption they’re good faith actors. This is akin to falling for TikTok/etc. ragebait and trying to have a discourse in the comment’s section. The subcultures allow for the exchange of material and non-material goods — primarily money and “woo.” Everything else is just a tool to that end.
sh34r 3y ago
9. Anyone who doesn’t recognize Roko’s basilisk as a bad ripoff of Pascal’s wager is an ignoramus. Anyone who is seriously concerned about it for more than an hour should consider a vasectomy. 10. I can’t think of a better form of “effective altruism” than paying for LessWrong members’ vasectomies. Not because I’m concerned about their genes. They’d just be terrible parents, and gods those kids would need lots of therapy.
makerdiety 3y ago
I would prefer cults about nerd topics (which only someone with a desktop or even laptop computer would read about, forget mobile device users) over nationalism based on phenomenological skin color. And e/acc is more of a little club than it is a cult. But I guess you could also consider it to be a criminal gang very much not unlike the Crips and the Bloods. Why? Because they're part of a counter-culture and naturally they want to diverge from the mainstream thought process. And the criminal activity e/acc engages in is non-compliant computer engineering. Which the Cathedral (pardon my language) dislikes. Basically, this is evolution in action. You're looking at the beginnings of a new species. Millions of new species are coming, really.
the_only_law 3y ago
As with other accelerations manifestos I’ve read, I’m still not unconvinced e/acc isn’t a shitpost at best, and drug induced psychosis at worst.

AnimalMuppet 3y ago

Well, this site often deals with fringe stuff. Take Lisp, for example. It is a fringe language, no matter how zealous its advocates are. So is Haskell. But you see articles about them posted here far out of proportion to their real-world use and impact. And I disagree that "most of their assumptions go unchallenged". I see challenges to the assumptions of lesswrong, e/acc, and other such stuff often here. (Maybe less often than is warranted, but still fairly often.) "It is the mark of an educated mind to be able to entertain a thought without accepting it." - often attributed to Aristotle, but apparently not actually from him. Still a good thought. We can post about this stuff, and discuss it, without buying it. I agree with your pushbacks.

two_in_one 3y ago

> 1. For "the singularity" to happen, we probably need something more to happen than just chatGPT to ingest more data or use more processing power. It's not actually clear what "the singularity" is? Is it something running out of control or it's still controllable? There is a blurry line. People are afraid because they think it's sort of uncontrollable explosion. The second question is about AGI. What is it? Is it something 'alive' or just a generic AI calculator with no 'creature' features. Like self preservation at least. I think our view of these two things will change soon as we get a close up picture. Pretty much like Turing test doesn't look great anymore. As even dumb chatbots can pass.

deleted 3y ago
[deleted]
com2kid 3y ago
I personally define AGI as a technology capable of improving itself exponentially. But I realize my definition is in the minority. :/ Of course if we ever manage to make a 1:1 cybernetic brain that works exactly like a human's brain, and is also a complete black box, we'll have achieved AGI. I'm not sure how useful that will be, but I'll have to admit it is AGI. So maybe I should say, "interesting AGI" is technology that can improve itself exponentially. :-D
- greysphere 3y ago
  Yeah, if you could input data set X with quality Q(X) and output data set Y of the same size with Q(Y) > Q(X), you'd really be on to something. But I don't think such a system exists yet, or even close. Inputting the internet, outputting a sea of garbage with a handful of diamonds that people have to spelunk through the garbage for seems the best so far. Madlibs is a pretty equivalent activity, and while fun, certainly not anything one would consider AGI. We need a revolutionary improvement on automated spelunking to get anywhere. Maybe we'll get a good spam filter as a side effect! But even if you had a system, there's still the resource cost to run the algorithm (needs to be bounded or else you've just made a finite jump) and the gains you make need to not decay (or again you've just made a finite jump). And all this needs to be compared against investing in humans - which seem pretty clearly to have AGI properties (but with some really bad constant factors. 20 year training time - ridiculous!) To me it seems things are a long way off and least a couple of major innovations away. But at least there's some ideas of the problems to tackle, which is a big step up!

com2kid 3y ago

> If you want to save lives, it's probably more useful to think about lives saved today than hypothetical lives you could save 100 years from now. Not that you shouldn't consider the long term, but your ability to predict the future gets worse and worse the farther you project out. This reasoning, taken to its logical extent, would mean completely defunding all medical research and instead redeploying all research to best utilizing our current technology. That is, hopefully, an obviously faulty decision. We must balance research with using current findings to help people. Stopping all drug development and telling people with currently untreatable diseases "haha, too bad, we aren't even going to try", is cruel. The ultimate goal of many technology utopians is the end of all death. We have examples from the natural world of incredibly long lived higher life forms, and simple life forms that are immortal, so it isn't scientifically impossible. The number of lives saved is literally, infinite. It is really hard to argue with "infinite upside". (The absolute societal shit show that such a technology would bring about is a rather important question...) > "we're killing more people by delaying time inventing the hypothetical life saving technology" is not very useful either, because you can't actually say how many lives would be saved versus harmed. For many future inventions, of course estimates can be made of the number of lives saved. The more pie in the sky research is, the larger those error bars become. "LLMs may one day help us sift through research and cure all forms of cancer" is, technically true, but the error bars on it are so high that you don't want to devote all the world's resources to LLMs just on the hopes of eliminating all forms of cancer 50+ years from now. "This company is working on AI Vision+Robotics technology so people don't have to do this super hazardous job that claims a bunch of lives. They are an estimated 5 years from productizing and they already have purchase contracts in place" is a much different statement.

financltravsty 3y ago

Hey, buddy (yes, I am intentionally being condescendingly familiar) these cults/movements are self-serving and cannot be reasoned with via rationality under the assumption they’re good faith actors. This is akin to falling for TikTok/etc. ragebait and trying to have a discourse in the comment’s section. The subcultures allow for the exchange of material and non-material goods — primarily money and “woo.” Everything else is just a tool to that end.

sh34r 3y ago

9. Anyone who doesn’t recognize Roko’s basilisk as a bad ripoff of Pascal’s wager is an ignoramus. Anyone who is seriously concerned about it for more than an hour should consider a vasectomy. 10. I can’t think of a better form of “effective altruism” than paying for LessWrong members’ vasectomies. Not because I’m concerned about their genes. They’d just be terrible parents, and gods those kids would need lots of therapy.

makerdiety 3y ago

I would prefer cults about nerd topics (which only someone with a desktop or even laptop computer would read about, forget mobile device users) over nationalism based on phenomenological skin color. And e/acc is more of a little club than it is a cult. But I guess you could also consider it to be a criminal gang very much not unlike the Crips and the Bloods. Why? Because they're part of a counter-culture and naturally they want to diverge from the mainstream thought process. And the criminal activity e/acc engages in is non-compliant computer engineering. Which the Cathedral (pardon my language) dislikes. Basically, this is evolution in action. You're looking at the beginnings of a new species. Millions of new species are coming, really.

the_only_law 3y ago

As with other accelerations manifestos I’ve read, I’m still not unconvinced e/acc isn’t a shitpost at best, and drug induced psychosis at worst.