6 ms·

Huh? Car maintenance is a rational, physical necessity. I don't need to compliment my car for it to start on a cold day. I'd like it to stay this way. Having t

Huh? Car maintenance is a rational, physical necessity. I don't need to compliment my car for it to start on a cold day. I'd like it to stay this way. Having to be unconditionally nice to computers is extremely creepy in part because it conditions us to be submissive - or else.

meiraleal 3y ago
> Having to be unconditionally nice to computers is extremely creepy in part because it conditions us to be submissive It's not a healthy mindset to relate politeness to submissiveness. although both behaviors might look similar from afar they are totally different
- OJFord 3y ago
  I think GP means being polite to something because otherwise it refuses to function is submissive, not that politeness is inherently. I might prefer my manager to ask me to do something politely, but it's still my job if he asks me rudely.
  - ToValueFunfetti 3y ago
    But the AI doesn't refuse to work unless you're polite. If my manager is polite with me, I'll have more morale and work a little harder. I'll also be more inclined to look out for my manager's interests- "You've asked me to do X, but really what you want is Y" vs. "Fine, you told me to do X, I'll do X". I don't think my manager is submitting to me when they're polite and get better results; I'm still the one who does things when I'm told.
    - davrosthedalek 3y ago
      The very start of this threat is that not expressing gratitude makes the model refuse to work.
      - meiraleal 3y ago
        It doesn't refuse to work. It behaves differently and yields better results with politeness. Coming from a large language model, the occurence of this phenomena is intriguing for some of us.
    - sebastiennight 3y ago
      This thread reminds me of [0] I wonder if there is a way to get ChatGPT to act in the way you're hinting at, though ("You've asked me to do X, but really what you want is Y"). This would be potentially risky, but high-value. [0]: https://nitter.net/ESYudkowsky/status/1718654143110512741 https://nitter.net/ESYudkowsky/status/1718654143110512741
      - famouswaffles 3y ago
        Uhh...https://arxiv.org/abs/2311.07590 https://arxiv.org/abs/2311.07590

meiraleal 3y ago

> Having to be unconditionally nice to computers is extremely creepy in part because it conditions us to be submissive It's not a healthy mindset to relate politeness to submissiveness. although both behaviors might look similar from afar they are totally different

OJFord 3y ago
I think GP means being polite to something because otherwise it refuses to function is submissive, not that politeness is inherently. I might prefer my manager to ask me to do something politely, but it's still my job if he asks me rudely.
- ToValueFunfetti 3y ago
  But the AI doesn't refuse to work unless you're polite. If my manager is polite with me, I'll have more morale and work a little harder. I'll also be more inclined to look out for my manager's interests- "You've asked me to do X, but really what you want is Y" vs. "Fine, you told me to do X, I'll do X". I don't think my manager is submitting to me when they're polite and get better results; I'm still the one who does things when I'm told.
  - davrosthedalek 3y ago
    The very start of this threat is that not expressing gratitude makes the model refuse to work.
    - meiraleal 3y ago
      It doesn't refuse to work. It behaves differently and yields better results with politeness. Coming from a large language model, the occurence of this phenomena is intriguing for some of us.
  - sebastiennight 3y ago
    This thread reminds me of [0] I wonder if there is a way to get ChatGPT to act in the way you're hinting at, though ("You've asked me to do X, but really what you want is Y"). This would be potentially risky, but high-value. [0]: https://nitter.net/ESYudkowsky/status/1718654143110512741 https://nitter.net/ESYudkowsky/status/1718654143110512741
    - famouswaffles 3y ago
      Uhh...https://arxiv.org/abs/2311.07590 https://arxiv.org/abs/2311.07590