6 ms·

I'll give it a shot, but first, a list of disclaimers: Yudkowsky gets a ton of unjustified internet hate and scorn, and I disagree with a lot of it. I read a nu

by pyrois 12y ago

I'll give it a shot, but first, a list of disclaimers: Yudkowsky gets a ton of unjustified internet hate and scorn, and I disagree with a lot of it. I read a number of the sequences, and quite enjoyed them. I also think his reaction to Roko's Basilisk was pretty reasonable: someone on your form comes up with a way to basically guarantee eternal torture for anyone who reads it, and then posts it, thus guaranteeing eternal torture for your forum readers? Who cares that the idea won't actually guarantee any such thing, Roko _thought_ that it might; I would be pissed as hell. Anyway, my point is, I'm only offering a gentle and hopefully reasoned disagreement with someone I regard highly. I'm not jumping on the "fuck that guy" train. Moving on. There was a post that boiled down to the question "Would you rather 3 ||| 3 people (where | is an ascii stand-in for Knuth's up-arrow notation) get a mote of dust in their eye, or one person be horrifically tortured for 50 years?" and his conclusion was basically "you can use math to assign some incredibly small epsilon of suffering to getting a mote of dust in your eye, but eventually, if you sum enough people, it's more suffering overall than one poor person getting horrifically tortured". My problem with his post was that I think that any morality scheme that results in the person getting tortured is fundamentally flawed, and I don't care how much math you throw at me to try and "prove" that it's better. I don't think I've ever heard of an attempt to rigorously derive morality that I agree with. Morality is too contextual, too messy, for us to perfectly capture in those sorts of models. It's especially bad when we try and model morality mathematically, and then take as gospel the result of that model, rather than say "oh, um, that's not a great result, the model must be wrong".

Vraxx 12y ago
That last point about the models actually captures a rather interesting point. Generally I agree that if the model isn't giving results that match observation than it should be concluded that the model is incorrect rather than the observation was flawed. This has worked very well as the scientific method, but I think that it is hinged on the fact that these observations are objective and verifiable. Now I happen to agree with you that I feel this dust result is incorrect, but morality is so undefined and subjective that I can't help but disagree with your conclusion that the model must be wrong. The entire moral question of the dust vs torture doesn't have one defined answer so we can't really conclude anything more than opinions about the model
- pyrois 12y ago
  Fair enough. I should have said "the model fails to accurately reflect my sense of morality".
bryondowd 12y ago
There aren't 3^^^3 atoms in the universe, let alone people. I think it's absurd to complain about a system not holding up when stretched far beyond the boundaries of the universe. If you decrease the number to even the number of humans that ever existed/exist/will exist, then it returns to being negligible in relation to torture. The problem with an if -> then, is when the 'if' is flawed, it doesn't really matter what the 'then' is. The human mind can't grasp 3^^^3 dust motes because it doesn't exist in any meaningful way.
ikeboy 12y ago
What do you say to the argument in http://lesswrong.com/lw/n3/circular_altruism/ http://lesswrong.com/lw/n3/circular_altruism/: >But let me ask you this. Suppose you had to choose between one person being tortured for 50 years, and a googol people being tortured for 49 years, 364 days, 23 hours, 59 minutes and 59 seconds. You would choose one person being tortured for 50 years, I do presume; otherwise I give up on you. >And similarly, if you had to choose between a googol people tortured for 49.9999999 years, and a googol-squared people being tortured for 49.9999998 years, you would pick the former. >A googolplex is ten to the googolth power. That's a googol/100 factors of a googol. So we can keep doing this, gradually - very gradually - diminishing the degree of discomfort, and multiplying by a factor of a googol each time, until we choose between a googolplex people getting a dust speck in their eye, and a googolplex/googol people getting two dust specks in their eye. >If you find your preferences are circular here, that makes rather a mockery of moral grandstanding. If you drive from San Jose to San Francisco to Oakland to San Jose, over and over again, you may have fun driving, but you aren't going anywhere. Maybe you think it a great display of virtue to choose for a googolplex people to get dust specks rather than one person being tortured. But if you would also trade a googolplex people getting one dust speck for a googolplex/googol people getting two dust specks et cetera, you sure aren't helping anyone. Circular preferences may work for feeling noble, but not for feeding the hungry or healing the sick.
- pyrois 12y ago
  He's assuming linearity, which at the very least needs justification. He's assuming that the function that maps from the pair (number of people, type of torture) to suffering is linear in the number of people, and also linear in the type of torture. I don't believe either of those things are true. To put it more clearly, he says: > So we can keep doing this, gradually - very gradually - diminishing the degree of discomfort And I don't agree that you can. The difference between dust-mote and torture is not one of degree, but one of _kind_. It's a discontinuous function (in my opinion). I don't know where the discontinuity is, but it's there.
  - ikeboy 12y ago
    >It's a discontinuous function (in my opinion). There are a finite number of possible brain states, far less than the 3||3 number. All possible "feelings" is likewise less than that. You need to be dividing all possible feelings of pain into two groups, and asserting that the absolute worst of the first group (containing dust specks) is incommensurate with the absolute best of the second group (containing torture). At some point you need to say that you'd pick specks even over 1 minute of torture, even over 1 second, on which I think most people's intuitions would stop saying that. Or you need to find some point between 50 years and 1 second where it become commensurate.
Houshalter 12y ago
I think the dust speck argument would be better if first framed as a preference for yourself: If you were going to live for 3^^^3 lifetimes, would you like to be tortured for 50 years in exchange for one fewer dust specks in your eye during each of those lifetimes. I think most people would answer no. So the debate has nothing to do with morality in general. Instead the debate is about Scope Insensitivity (http://lesswrong.com/lw/hw/scope_insensitivity/ http://lesswrong.com/lw/hw/scope_insensitivity/), that our emotional reaction to things isn't diminished by even infinitesimally small quantities (see also Useless Medical Disclaimers http://lesswrong.com/lw/h4/useless_medical_disclaimers/ http://lesswrong.com/lw/h4/useless_medical_disclaimers/). I think I can make that position seem more reasonable if framed another way: Would you risk a 1/(3^^^3) chance at 50 years of torture, in exchange for getting rid of one dust speck out of your eye? I think most people would say yes. You couldn't even get out of bed in the morning if you weren't willing to take even incredibly small risks. There is a much higher probability than 1/(3^^^3) that you could get in a car accident with injuries that cause 50 years of incredible pain, yet you will still probably risk driving for even trivial things. And if you lived for 3^^^3 lifetimes and took this risk each time, you would likely suffer 50 years of torture during at least one lifetime.
- pyrois 12y ago
  Yes, I agree, I would take the risk, but those are two entirely different things. It's not the same argument framed in a different way at all. In Yudkowsky's argument, it's the option between one person _absolutely guaranteed_ to have 50 years of torture, vs 3^^^3 people _absolutely guaranteed_ to be dust-moted. I think the guarantee changes things significantly.
  - Houshalter 12y ago
    If you consistently take low risks that might result in getting tortured, then you are pretty much guaranteed to eventually lose the bet and suffer. The probability of eventually losing even a small bet approaches 1 if you take it enough times. Just think of it as if you were going to other people and deciding to take this bet for them. 1/3^^^3 chance of torture in exchange for removing a speck of dust from their eye. I think you would be ok with that because it's an obvious choice for ourselves. And if you continue to do this for enough people, eventually one of them will get tortured. After removing dust specks from approximately 3^^^3 people's eye. But you are right, there is a chance no one will get tortured. So do it for 3^^^^3 people then and the probability is 0.99999...

by pyrois 12y ago

Vraxx 12y ago

That last point about the models actually captures a rather interesting point. Generally I agree that if the model isn't giving results that match observation than it should be concluded that the model is incorrect rather than the observation was flawed. This has worked very well as the scientific method, but I think that it is hinged on the fact that these observations are objective and verifiable. Now I happen to agree with you that I feel this dust result is incorrect, but morality is so undefined and subjective that I can't help but disagree with your conclusion that the model must be wrong. The entire moral question of the dust vs torture doesn't have one defined answer so we can't really conclude anything more than opinions about the model

pyrois 12y ago
Fair enough. I should have said "the model fails to accurately reflect my sense of morality".

bryondowd 12y ago

There aren't 3^^^3 atoms in the universe, let alone people. I think it's absurd to complain about a system not holding up when stretched far beyond the boundaries of the universe. If you decrease the number to even the number of humans that ever existed/exist/will exist, then it returns to being negligible in relation to torture. The problem with an if -> then, is when the 'if' is flawed, it doesn't really matter what the 'then' is. The human mind can't grasp 3^^^3 dust motes because it doesn't exist in any meaningful way.

ikeboy 12y ago

What do you say to the argument in http://lesswrong.com/lw/n3/circular_altruism/ http://lesswrong.com/lw/n3/circular_altruism/: >But let me ask you this. Suppose you had to choose between one person being tortured for 50 years, and a googol people being tortured for 49 years, 364 days, 23 hours, 59 minutes and 59 seconds. You would choose one person being tortured for 50 years, I do presume; otherwise I give up on you. >And similarly, if you had to choose between a googol people tortured for 49.9999999 years, and a googol-squared people being tortured for 49.9999998 years, you would pick the former. >A googolplex is ten to the googolth power. That's a googol/100 factors of a googol. So we can keep doing this, gradually - very gradually - diminishing the degree of discomfort, and multiplying by a factor of a googol each time, until we choose between a googolplex people getting a dust speck in their eye, and a googolplex/googol people getting two dust specks in their eye. >If you find your preferences are circular here, that makes rather a mockery of moral grandstanding. If you drive from San Jose to San Francisco to Oakland to San Jose, over and over again, you may have fun driving, but you aren't going anywhere. Maybe you think it a great display of virtue to choose for a googolplex people to get dust specks rather than one person being tortured. But if you would also trade a googolplex people getting one dust speck for a googolplex/googol people getting two dust specks et cetera, you sure aren't helping anyone. Circular preferences may work for feeling noble, but not for feeding the hungry or healing the sick.

pyrois 12y ago
He's assuming linearity, which at the very least needs justification. He's assuming that the function that maps from the pair (number of people, type of torture) to suffering is linear in the number of people, and also linear in the type of torture. I don't believe either of those things are true. To put it more clearly, he says: > So we can keep doing this, gradually - very gradually - diminishing the degree of discomfort And I don't agree that you can. The difference between dust-mote and torture is not one of degree, but one of _kind_. It's a discontinuous function (in my opinion). I don't know where the discontinuity is, but it's there.
- ikeboy 12y ago
  >It's a discontinuous function (in my opinion). There are a finite number of possible brain states, far less than the 3||3 number. All possible "feelings" is likewise less than that. You need to be dividing all possible feelings of pain into two groups, and asserting that the absolute worst of the first group (containing dust specks) is incommensurate with the absolute best of the second group (containing torture). At some point you need to say that you'd pick specks even over 1 minute of torture, even over 1 second, on which I think most people's intuitions would stop saying that. Or you need to find some point between 50 years and 1 second where it become commensurate.

Houshalter 12y ago

I think the dust speck argument would be better if first framed as a preference for yourself: If you were going to live for 3^^^3 lifetimes, would you like to be tortured for 50 years in exchange for one fewer dust specks in your eye during each of those lifetimes. I think most people would answer no. So the debate has nothing to do with morality in general. Instead the debate is about Scope Insensitivity (http://lesswrong.com/lw/hw/scope_insensitivity/ http://lesswrong.com/lw/hw/scope_insensitivity/), that our emotional reaction to things isn't diminished by even infinitesimally small quantities (see also Useless Medical Disclaimers http://lesswrong.com/lw/h4/useless_medical_disclaimers/ http://lesswrong.com/lw/h4/useless_medical_disclaimers/). I think I can make that position seem more reasonable if framed another way: Would you risk a 1/(3^^^3) chance at 50 years of torture, in exchange for getting rid of one dust speck out of your eye? I think most people would say yes. You couldn't even get out of bed in the morning if you weren't willing to take even incredibly small risks. There is a much higher probability than 1/(3^^^3) that you could get in a car accident with injuries that cause 50 years of incredible pain, yet you will still probably risk driving for even trivial things. And if you lived for 3^^^3 lifetimes and took this risk each time, you would likely suffer 50 years of torture during at least one lifetime.

pyrois 12y ago
Yes, I agree, I would take the risk, but those are two entirely different things. It's not the same argument framed in a different way at all. In Yudkowsky's argument, it's the option between one person _absolutely guaranteed_ to have 50 years of torture, vs 3^^^3 people _absolutely guaranteed_ to be dust-moted. I think the guarantee changes things significantly.
- Houshalter 12y ago
  If you consistently take low risks that might result in getting tortured, then you are pretty much guaranteed to eventually lose the bet and suffer. The probability of eventually losing even a small bet approaches 1 if you take it enough times. Just think of it as if you were going to other people and deciding to take this bet for them. 1/3^^^3 chance of torture in exchange for removing a speck of dust from their eye. I think you would be ok with that because it's an obvious choice for ourselves. And if you continue to do this for enough people, eventually one of them will get tortured. After removing dust specks from approximately 3^^^3 people's eye. But you are right, there is a chance no one will get tortured. So do it for 3^^^^3 people then and the probability is 0.99999...