8 ms·

Ends Don't Justify Means (Among Humans)

45 points by ektimo 16y ago

iwwr 16y ago
A creature running under that paradigm is abandoning their humanity. We can only exist as human beings with rights if we respect our fellows' human rights.
- Groxx 16y ago
  By running under the paradigm that they assume they are flawed? That's the core claim in there. >..if we respect our fellows' human rights This is a problematic clause. Whose rights do you disrespect: the one you pushed, or the 5 you could have saved?
  - yters 16y ago
    Deontological ethics are hierarchical. Greater goods take precedence over lesser goods. But, killing someone for a greater good doesn't making killing good, i.e. it isn't justified.
    - Groxx 16y ago
      /me reads wikipedia article Huh. By the sound of it, they take the stance that violence against non-violence is a flat-out unjustifiable act, regardless of the consequences. But on those grounds, can't you say the trolley is a violent enemy that must be stopped, so violent acts are justifiable? You certainly can't expect to cause violence only to the enemy, especially if you consider psychological violence to be violence (and why wouldn't it be? Torture is torture.). By causing violence only to the enemy, you may be causing psychological violence to any who happen to in the blast radius / capable of witnessing your violence, thus you are causing violence to them. So it's a moral goal for a flawless world with no-one else in it. Of course, if the whole world were like this, then pushing someone in front of the trolley would be justifiable, because they would have not been justified in not throwing themselves in front of such a violent enemy, if they were aware and able to do so. I don't buy it. Ideals are worthless if they can only exist in an impossible world.
- j_baker 16y ago
  There's the rub though. How do you respect others' rights? Would enslaving 10 people to set 100 free not be respecting others' rights (at least in the long term)? Now you can argue whether or not that's the right way to respect others' rights, but it's respecting the rights of others nonetheless.
  - iwwr 16y ago
    Rights are characters of the individual, even if the individual lives in a social context. So those 10 have as much rights as those 100. Sacrificing the few for the sake of the many means you yourself are never justifiably safe, you can always be in the minority.
    - j_baker 16y ago
      Sure. But you have to sacrifice someone's rights, don't you? By not enslaving those 10 people, you're sacrificing the rights of those 100 people. And while that doesn't mean that you're ever safe, it does guarantee that you have the highest chance of being safe.
      - iwwr 16y ago
        When you are acting like that, you are not sacrificing other people's rights, but your own. That is, you are opening yourself up for rightful retaliation from the victims, or whoever might want to champion their cause (friends, families etc). You may be able to physically defend yourself, but you won't have moral grounds to plead your case. In other words, if you act like an animal, you can be treated as such.
- syllogism 16y ago
  What do you mean by that? Why is it so? What is a "right", in fact? You've just offered another slogan. You're going to have to work harder than that if you want to argue against consequentialism --- the idea that what matters are the consequences of each action, not the principles it contravenes.
  - yters 16y ago
    What ends are good in consequentialism? De-ontological morals to one degree or another are necessary to avoid moral relativism. If you agree moral relativism is bad, you agree consequentialism is bad.
    - JoachimSchipper 16y ago
      I do not understand how that follows. It's entirely possible to imagine a world in which a god/God judges you by the results of your actions instead of your intentions. (E.g. a woman may have been raped, but pre-marital sex still condemns her to Hell; or, perhaps less unfairly, you're not "saved through faith alone" but rather judged by how many people you've positively affected.)
      - yters 16y ago
        That's deontological, i.e. certain events have inherent value, instead of being judged by their effects. Deontological morality means the effects chain is terminated. Otherwise there is no termination and no value ascribed to an action.
        JoachimSchipper 16y ago
        [I'm not too familiar with the (English) vocabulary used in these discussions, so I'll try to expand my previous comment a little and lay off the jargon. If you still disagree, can you point out where?] > If you agree moral relativism is bad, you agree consequentialism is bad. I disagreed with this statement. My examples were meant to say: it's possible to have "god-given morals" (and thus no "moral relativism") while still judging acts by their consequences ("consequentialism"). In such a universe, wearing a sexy dress into a bad neighborhood may be morally fine (act ok), but if this causes you to get raped (pre-marital sex bad) you're still going straight to Hell (so act ok, results bad -> bad). Conversely, if I'd kill my neighbour for no good reason (act bad), and my neighbour happened to be 1920's Hitler (results good), I will be rewarded richly (so act bad, results good -> good). Some disclaimers: this may be based on a misunderstanding of the words you used; I don't think the universe I sketched is the universe in which we live; and of course rape is not the victim's fault, and not wearing sexy dresses may not be enough to prevent it from happening (for the sake of discussion, though, in this particular instance it wouldn't have happened if the victim had worn a more conservative garment instead.)
- sp4rki 16y ago
  So if your daughter is kidnapped by the a Colombian Drug Cartel and you decided that since you're a badass, you're going to go save her. Then you kill 20 baddies that are themselves cutthroat murderers that would have undeniably killed a greater number of people. Can you really tell me that the ends wouldn't justify the means? If MI-5 kills 4 terrorists which where preparing to bomb a bus station with hundreds if not thousands of people, they're in the wrong for not respecting the rights of the terrorists? Or would they be in the wrong for not protecting the citizens of the country they're supposed to be protecting? The ends will always justify the means in the mind of the person doing the act. What makes the actions of one person (or government) right is that in the minds of other people the means are also justifiable. If someone is willing to not respect my rights and kill me to steal my wallet, I'm willing to forgo their rights because lets me honest, the ends do justify the means if it benefits not only yourself, but the rest of the population.
  - yters 16y ago
    Rights are a social contract. You give up your rights once you violate those of others.
    - sp4rki 16y ago
      Oh I agree. The moment you go on a killing rampage to rescue your daughter from the cartels, you justify other people taking action against you to protect themselves. The thing is, there needs to be someone that resigns their rights and does the right (so to speak) thing. Someone needs to keep the terrorists at bay. Someone needs to protect the citizens from the gangs. Morally objectionable? Maybe. In the eyes of a few. In the eyes of the rest, these people are heroes.
      - yters 16y ago
        No, those actions are morally right. Those people have already violated their part in the social contract.
  - iwwr 16y ago
    I'd be totally justified in trying to save my daughter and nobody would be justified in stopping me. This is not a case of ends justifying the means because my opponents are all unjust. I am not sacrificing innocents to get her back. However, if innocents happened to be in the way, I'd have to make a choice in either abandoning my rights or not; the innocents would be entitled to retaliate. In the anti-terrorism case, if the presumed terrorists were in fact innocents, then they (or their champions) would have a right to retaliate against the aggressors or against their commanders. In our minds and to ourselves, we are always justified, but we can't justify ourselves from a moral standpoint, nor plead our case if the innocents have decided to retaliate.
- anamax 16y ago
  > We can only exist as human beings with rights if we respect our fellows' human rights. What are said rights? For example, do I have a right to food? If so, who is obligated to provide it? ("govt" isn't an answer.) How about a right to live in the southwest US? (I may require a dry climate for health reasons.) How about with an ocean view for my piece of mind? How about a right to live near people who I like or away from folks whom I don't like?
anamax 16y ago
The means produce the ends.
- deleted 16y ago
  [deleted]
billswift 16y ago
The ends are the only way to justify the means; what the sloppy thinkers who recite "The ends don't justify the means" really seem to mean is that some means are not justifiable. Which is undeniably true, but because they don't really think their position/slogan through, they often end up gathering a bunch of nonsense along with it.
- electromagnetic 16y ago
  You're right. Slaughtering 20 junior school students and a crossing guard are not acceptable means to get your daughter from school to her dentist appointment on time. It is however acceptable to kill 20 enemy combatants to rescue a dignitary from a besieged embassy.
  - ismarc 16y ago
    While this helps get the point across, your example doesn't provide comparable events. The distinction is the weighing of the results of actions to the actions themselves. Is taking one life justified to save another? Is it measured by the greater good, such that the life that is saved will provide greater contributions than the one that is taken? Or is it measured in that all life is inherently to be treated as equal? And if all life is treated as inherently equal, is it a numbers game, taking 100 lives is justified so long as it saves 101 or more? The phrase "The ends don't justify the means" is not hyperbole to discredit accomplishments, nor is it meant to remove the ends from the equation. It changes the equation to provide the necessity of weighing the ends against the methods rather than measuring the ends independent of the context used to accomplish them. What is even more interesting is the opposite. The ends not justifying the means is typically used for weighing the actions necessary for a favorable outcome. If looked at from the opposite perspective, do the means justify the ends, it presents a more difficult scenario. If X people die, but only appropriate means were used (and others would have saved X people), should those that chose the appropriate actions be held responsible for the failure to provide a favorable outcome? Or should they be heralded for making the difficult decisions to only use morally acceptable practices?
  - brudgers 16y ago
sigzero 16y ago
It is never right, to do wrong, for a chance to do right.
igravious 16y ago
So let me get this straight. The author is saying that his answer to the the classic hypothetical dilemma in ethics of "is it right to do harm (murder one) to prevent a greater harm (death of five)" which usually framed in utilitarian-ish debates is: a hypothetical incorruptible super awesome version of me would murder the innocent person but as I am not that type of being (and only that type of being can answer the question) I am not going to answer the question But why not go all in and pose it thus: "is it better for one person to suffer eternal suffering to free all others from any type of suffering"? And that's why Jesus died for our sins you know. And that really happened. And Jesus was smarter than a hypothetical incorruptible super awesome version of you. Therefore ... I'm not sure where this is going. Maybe what I'm trying to say is (and by all means argue the toss with me and don't shoot me down) Eliezer Yudkowsky sounds a lot smarter than he actually is.
- secret 16y ago
  There's a difference between sacrificing another and sacrificing yourself.
  - igravious 16y ago
    May we never be placed in that position ever in our lives.
- deleted 16y ago
  [deleted]
- jerf 16y ago
  That is not an adequate summary of the point. The point that you are addressing is that if we are to truly take this as a question about "what should a real person do?", the question can be rephrased without loss as "You are a person standing before the track and you know with 100% certainty that if you flip the switch that one person is 100% likely to die and if you do nothing that five people are 100% likely to die." and his response is that it isn't even possible in theory for a person to know these things with 100% certainty. The key phrase is "I can't occupy the epistemological state you want me to imagine". I would also draw your attention to the first sentence of the next paragraph: "Now, to me this seems like a dodge." This isn't the core point of the essay, and the more I stare at it, the more it does seem like it's five paragraphs accidentally ripped from another essay ("And now the philosopher comes" -> "Now to me this seems like a dodge"); if you just cut those five out entirely it seems more focused, and those five paragraphs can spin off into another interesting essay. (One that would, I think, conclude that this is actually just a way of rephrasing the idea that philosophical hypotheticals are actually useless by virtue of being impossibly overspecified which itself comes from impossible oversimplification, and in general the hypothetical question "What if an absolutely mathematically impossible thing happened?" is not a fruitful line of thought.)
powera 16y ago
This is an incredible stretch of what I'd take "The Ends Don't Justify The Means" to be. The whole "killing one person to save ten" thing really isn't related. The way I see it, it's about externalities. I push someone onto a train track to save ten people in a mineshaft, fine. Somebody else watches me push someone on a train track, doesn't see the mineshaft, and goes on to push 25 people onto train tracks in the future. Perhaps that's contrived, but when he describes coups in the same terminology it's a lot less so.
johngalt 16y ago
But people do make these decisions all the time. Closing watertight doors on a warship? The first responders at Chernobyl? Kamikaze? Not only do we condemn other innocents to death, but also ourselves.
danbmil99 16y ago
too much faith in 'friendly AI'. Friendly my ass! Doesn't this guy consume Science Fiction? Seriously, I would extend his argument to assume that all 'hardware is corrupted' as he puts it, though I think it's more accurate to say 'all software is buggy'. For this amazing AI to even exist, it probably had to be pretty damn selfish, pushing out all other potential AI's to get to the top of the heap. Just saying.
patrickk 16y ago
Surely a sufficiently advanced AI would simply send a warning signal to the train driver to "hit the brakes quick". Next problem please.
samd 16y ago
The objection is he making is essentially the objection a rule-consequentialist would make. Which is that although in this particular situation it would produce better consequences to kill the innocent man and save the 5 people, overall it would produce worse consequences if everyone followed the rule that killing innocent people is an acceptable means to some end.
- JoachimSchipper 16y ago
  I think it's a bit more subtle than that, although he does point out the danger of imitators. See my reply at http://news.ycombinator.com/item?id=2088562 http://news.ycombinator.com/item?id=2088562.
  - samd 16y ago
    I'm not sure. Here is the rule: "Do no harm." Why should we follow it? A rule-consequentialist would say, "Because if everyone followed that rule all the time it would lead to better consequences, on average, than if people tried to calculate the consequences of each individual action and act to maximize them." Why would that lead to better consequences? A rule-consequentialist could say, "Because people are quite bad at calculating consequences, especially in the tumultuous time before making a critical ethical decision. While we have had the time to think and properly calculate the hypothetical consequences of everyone following the rule." I think that's exactly the argument Eliezer is making.
    - JoachimSchipper 16y ago
      Put like that, I agree - I had a different meaning of "rule-consequentialist" in mind than you, apparently. (I'm not too familiar with the vocabulary of such discussions, sorry!) Thanks for clarifying.
      - samd 16y ago
        Don't worry, like any academic field philosophy is filled with jargon, but knowing the jargon doesn't make you any smarter and not knowing doesn't make you any dumber. Both you and Eliezer made good arguments and points, they just happened to already have a name in the philosophical jargon. If you're interested, the Stanford Encyclopedia of Philosophy at plato.stanford.edu is the greatest repository of philosophical knowledge on the Internet. They have an article about rule-consequentialism: http://plato.stanford.edu/entries/consequentialism-rule/ http://plato.stanford.edu/entries/consequentialism-rule/

45 points by ektimo 16y ago

iwwr 16y ago

A creature running under that paradigm is abandoning their humanity. We can only exist as human beings with rights if we respect our fellows' human rights.

Groxx 16y ago
By running under the paradigm that they assume they are flawed? That's the core claim in there. >..if we respect our fellows' human rights This is a problematic clause. Whose rights do you disrespect: the one you pushed, or the 5 you could have saved?
- yters 16y ago
  Deontological ethics are hierarchical. Greater goods take precedence over lesser goods. But, killing someone for a greater good doesn't making killing good, i.e. it isn't justified.
  - Groxx 16y ago
    /me reads wikipedia article Huh. By the sound of it, they take the stance that violence against non-violence is a flat-out unjustifiable act, regardless of the consequences. But on those grounds, can't you say the trolley is a violent enemy that must be stopped, so violent acts are justifiable? You certainly can't expect to cause violence only to the enemy, especially if you consider psychological violence to be violence (and why wouldn't it be? Torture is torture.). By causing violence only to the enemy, you may be causing psychological violence to any who happen to in the blast radius / capable of witnessing your violence, thus you are causing violence to them. So it's a moral goal for a flawless world with no-one else in it. Of course, if the whole world were like this, then pushing someone in front of the trolley would be justifiable, because they would have not been justified in not throwing themselves in front of such a violent enemy, if they were aware and able to do so. I don't buy it. Ideals are worthless if they can only exist in an impossible world.
j_baker 16y ago
There's the rub though. How do you respect others' rights? Would enslaving 10 people to set 100 free not be respecting others' rights (at least in the long term)? Now you can argue whether or not that's the right way to respect others' rights, but it's respecting the rights of others nonetheless.
- iwwr 16y ago
  Rights are characters of the individual, even if the individual lives in a social context. So those 10 have as much rights as those 100. Sacrificing the few for the sake of the many means you yourself are never justifiably safe, you can always be in the minority.
  - j_baker 16y ago
    Sure. But you have to sacrifice someone's rights, don't you? By not enslaving those 10 people, you're sacrificing the rights of those 100 people. And while that doesn't mean that you're ever safe, it does guarantee that you have the highest chance of being safe.
    - iwwr 16y ago
      When you are acting like that, you are not sacrificing other people's rights, but your own. That is, you are opening yourself up for rightful retaliation from the victims, or whoever might want to champion their cause (friends, families etc). You may be able to physically defend yourself, but you won't have moral grounds to plead your case. In other words, if you act like an animal, you can be treated as such.
syllogism 16y ago
What do you mean by that? Why is it so? What is a "right", in fact? You've just offered another slogan. You're going to have to work harder than that if you want to argue against consequentialism --- the idea that what matters are the consequences of each action, not the principles it contravenes.
- yters 16y ago
  What ends are good in consequentialism? De-ontological morals to one degree or another are necessary to avoid moral relativism. If you agree moral relativism is bad, you agree consequentialism is bad.
  - JoachimSchipper 16y ago
    I do not understand how that follows. It's entirely possible to imagine a world in which a god/God judges you by the results of your actions instead of your intentions. (E.g. a woman may have been raped, but pre-marital sex still condemns her to Hell; or, perhaps less unfairly, you're not "saved through faith alone" but rather judged by how many people you've positively affected.)
    - yters 16y ago
      That's deontological, i.e. certain events have inherent value, instead of being judged by their effects. Deontological morality means the effects chain is terminated. Otherwise there is no termination and no value ascribed to an action.
      - JoachimSchipper 16y ago
        [I'm not too familiar with the (English) vocabulary used in these discussions, so I'll try to expand my previous comment a little and lay off the jargon. If you still disagree, can you point out where?] > If you agree moral relativism is bad, you agree consequentialism is bad. I disagreed with this statement. My examples were meant to say: it's possible to have "god-given morals" (and thus no "moral relativism") while still judging acts by their consequences ("consequentialism"). In such a universe, wearing a sexy dress into a bad neighborhood may be morally fine (act ok), but if this causes you to get raped (pre-marital sex bad) you're still going straight to Hell (so act ok, results bad -> bad). Conversely, if I'd kill my neighbour for no good reason (act bad), and my neighbour happened to be 1920's Hitler (results good), I will be rewarded richly (so act bad, results good -> good). Some disclaimers: this may be based on a misunderstanding of the words you used; I don't think the universe I sketched is the universe in which we live; and of course rape is not the victim's fault, and not wearing sexy dresses may not be enough to prevent it from happening (for the sake of discussion, though, in this particular instance it wouldn't have happened if the victim had worn a more conservative garment instead.)
sp4rki 16y ago
So if your daughter is kidnapped by the a Colombian Drug Cartel and you decided that since you're a badass, you're going to go save her. Then you kill 20 baddies that are themselves cutthroat murderers that would have undeniably killed a greater number of people. Can you really tell me that the ends wouldn't justify the means? If MI-5 kills 4 terrorists which where preparing to bomb a bus station with hundreds if not thousands of people, they're in the wrong for not respecting the rights of the terrorists? Or would they be in the wrong for not protecting the citizens of the country they're supposed to be protecting? The ends will always justify the means in the mind of the person doing the act. What makes the actions of one person (or government) right is that in the minds of other people the means are also justifiable. If someone is willing to not respect my rights and kill me to steal my wallet, I'm willing to forgo their rights because lets me honest, the ends do justify the means if it benefits not only yourself, but the rest of the population.
- yters 16y ago
  Rights are a social contract. You give up your rights once you violate those of others.
  - sp4rki 16y ago
    Oh I agree. The moment you go on a killing rampage to rescue your daughter from the cartels, you justify other people taking action against you to protect themselves. The thing is, there needs to be someone that resigns their rights and does the right (so to speak) thing. Someone needs to keep the terrorists at bay. Someone needs to protect the citizens from the gangs. Morally objectionable? Maybe. In the eyes of a few. In the eyes of the rest, these people are heroes.
    - yters 16y ago
      No, those actions are morally right. Those people have already violated their part in the social contract.
- iwwr 16y ago
  I'd be totally justified in trying to save my daughter and nobody would be justified in stopping me. This is not a case of ends justifying the means because my opponents are all unjust. I am not sacrificing innocents to get her back. However, if innocents happened to be in the way, I'd have to make a choice in either abandoning my rights or not; the innocents would be entitled to retaliate. In the anti-terrorism case, if the presumed terrorists were in fact innocents, then they (or their champions) would have a right to retaliate against the aggressors or against their commanders. In our minds and to ourselves, we are always justified, but we can't justify ourselves from a moral standpoint, nor plead our case if the innocents have decided to retaliate.
anamax 16y ago
> We can only exist as human beings with rights if we respect our fellows' human rights. What are said rights? For example, do I have a right to food? If so, who is obligated to provide it? ("govt" isn't an answer.) How about a right to live in the southwest US? (I may require a dry climate for health reasons.) How about with an ocean view for my piece of mind? How about a right to live near people who I like or away from folks whom I don't like?

anamax 16y ago

The means produce the ends.

deleted 16y ago
[deleted]

billswift 16y ago

The ends are the only way to justify the means; what the sloppy thinkers who recite "The ends don't justify the means" really seem to mean is that some means are not justifiable. Which is undeniably true, but because they don't really think their position/slogan through, they often end up gathering a bunch of nonsense along with it.

electromagnetic 16y ago
You're right. Slaughtering 20 junior school students and a crossing guard are not acceptable means to get your daughter from school to her dentist appointment on time. It is however acceptable to kill 20 enemy combatants to rescue a dignitary from a besieged embassy.
- ismarc 16y ago
  While this helps get the point across, your example doesn't provide comparable events. The distinction is the weighing of the results of actions to the actions themselves. Is taking one life justified to save another? Is it measured by the greater good, such that the life that is saved will provide greater contributions than the one that is taken? Or is it measured in that all life is inherently to be treated as equal? And if all life is treated as inherently equal, is it a numbers game, taking 100 lives is justified so long as it saves 101 or more? The phrase "The ends don't justify the means" is not hyperbole to discredit accomplishments, nor is it meant to remove the ends from the equation. It changes the equation to provide the necessity of weighing the ends against the methods rather than measuring the ends independent of the context used to accomplish them. What is even more interesting is the opposite. The ends not justifying the means is typically used for weighing the actions necessary for a favorable outcome. If looked at from the opposite perspective, do the means justify the ends, it presents a more difficult scenario. If X people die, but only appropriate means were used (and others would have saved X people), should those that chose the appropriate actions be held responsible for the failure to provide a favorable outcome? Or should they be heralded for making the difficult decisions to only use morally acceptable practices?
- brudgers 16y ago

sigzero 16y ago

It is never right, to do wrong, for a chance to do right.

igravious 16y ago

So let me get this straight. The author is saying that his answer to the the classic hypothetical dilemma in ethics of "is it right to do harm (murder one) to prevent a greater harm (death of five)" which usually framed in utilitarian-ish debates is: a hypothetical incorruptible super awesome version of me would murder the innocent person but as I am not that type of being (and only that type of being can answer the question) I am not going to answer the question But why not go all in and pose it thus: "is it better for one person to suffer eternal suffering to free all others from any type of suffering"? And that's why Jesus died for our sins you know. And that really happened. And Jesus was smarter than a hypothetical incorruptible super awesome version of you. Therefore ... I'm not sure where this is going. Maybe what I'm trying to say is (and by all means argue the toss with me and don't shoot me down) Eliezer Yudkowsky sounds a lot smarter than he actually is.

secret 16y ago
There's a difference between sacrificing another and sacrificing yourself.
- igravious 16y ago
  May we never be placed in that position ever in our lives.
deleted 16y ago
[deleted]
jerf 16y ago
That is not an adequate summary of the point. The point that you are addressing is that if we are to truly take this as a question about "what should a real person do?", the question can be rephrased without loss as "You are a person standing before the track and you know with 100% certainty that if you flip the switch that one person is 100% likely to die and if you do nothing that five people are 100% likely to die." and his response is that it isn't even possible in theory for a person to know these things with 100% certainty. The key phrase is "I can't occupy the epistemological state you want me to imagine". I would also draw your attention to the first sentence of the next paragraph: "Now, to me this seems like a dodge." This isn't the core point of the essay, and the more I stare at it, the more it does seem like it's five paragraphs accidentally ripped from another essay ("And now the philosopher comes" -> "Now to me this seems like a dodge"); if you just cut those five out entirely it seems more focused, and those five paragraphs can spin off into another interesting essay. (One that would, I think, conclude that this is actually just a way of rephrasing the idea that philosophical hypotheticals are actually useless by virtue of being impossibly overspecified which itself comes from impossible oversimplification, and in general the hypothetical question "What if an absolutely mathematically impossible thing happened?" is not a fruitful line of thought.)

powera 16y ago

This is an incredible stretch of what I'd take "The Ends Don't Justify The Means" to be. The whole "killing one person to save ten" thing really isn't related. The way I see it, it's about externalities. I push someone onto a train track to save ten people in a mineshaft, fine. Somebody else watches me push someone on a train track, doesn't see the mineshaft, and goes on to push 25 people onto train tracks in the future. Perhaps that's contrived, but when he describes coups in the same terminology it's a lot less so.

johngalt 16y ago

But people do make these decisions all the time. Closing watertight doors on a warship? The first responders at Chernobyl? Kamikaze? Not only do we condemn other innocents to death, but also ourselves.

danbmil99 16y ago

too much faith in 'friendly AI'. Friendly my ass! Doesn't this guy consume Science Fiction? Seriously, I would extend his argument to assume that all 'hardware is corrupted' as he puts it, though I think it's more accurate to say 'all software is buggy'. For this amazing AI to even exist, it probably had to be pretty damn selfish, pushing out all other potential AI's to get to the top of the heap. Just saying.

patrickk 16y ago

Surely a sufficiently advanced AI would simply send a warning signal to the train driver to "hit the brakes quick". Next problem please.

samd 16y ago

The objection is he making is essentially the objection a rule-consequentialist would make. Which is that although in this particular situation it would produce better consequences to kill the innocent man and save the 5 people, overall it would produce worse consequences if everyone followed the rule that killing innocent people is an acceptable means to some end.

JoachimSchipper 16y ago
I think it's a bit more subtle than that, although he does point out the danger of imitators. See my reply at http://news.ycombinator.com/item?id=2088562 http://news.ycombinator.com/item?id=2088562.
- samd 16y ago
  I'm not sure. Here is the rule: "Do no harm." Why should we follow it? A rule-consequentialist would say, "Because if everyone followed that rule all the time it would lead to better consequences, on average, than if people tried to calculate the consequences of each individual action and act to maximize them." Why would that lead to better consequences? A rule-consequentialist could say, "Because people are quite bad at calculating consequences, especially in the tumultuous time before making a critical ethical decision. While we have had the time to think and properly calculate the hypothetical consequences of everyone following the rule." I think that's exactly the argument Eliezer is making.
  - JoachimSchipper 16y ago
    Put like that, I agree - I had a different meaning of "rule-consequentialist" in mind than you, apparently. (I'm not too familiar with the vocabulary of such discussions, sorry!) Thanks for clarifying.
    - samd 16y ago
      Don't worry, like any academic field philosophy is filled with jargon, but knowing the jargon doesn't make you any smarter and not knowing doesn't make you any dumber. Both you and Eliezer made good arguments and points, they just happened to already have a name in the philosophical jargon. If you're interested, the Stanford Encyclopedia of Philosophy at plato.stanford.edu is the greatest repository of philosophical knowledge on the Internet. They have an article about rule-consequentialism: http://plato.stanford.edu/entries/consequentialism-rule/ http://plato.stanford.edu/entries/consequentialism-rule/