5 ms·

I have noticed, and I am concerned that they were the leaders of the Superalignment team.

by pfist 2y ago

transcriptase 2y ago
On the other hand, they clearly they weren’t concerned enough about the issue to continue working on it.
- dontupvoteme 2y ago
  One could argue that at this point openai is being Extended and Embraced by Microsoft and is unlikely to have much autonomy or groundbreaking impact one way or another.
- llamaimperative 2y ago
  Ah yes, a scientist refusing to work on the hydrogen bomb couldn't have been all that concerned about it.
- scarmig 2y ago
  If your ostensible purpose is being sidelined by decision makers, trying to fight back is often a good option, but sometimes you fail. Admitting failure and focusing on other approaches is the right choice at that point.
- HarHarVeryFunny 2y ago
  The Anthropic folk were concerned enough that they left, and are indeed continuing to work on it [AI safety]. Now, we've have the co-leads of the super-alignment/safety team leaving too. Certainly not a good look for OpenAI. There really doesn't seem to be much of a mission left at OpenAI - they have a CEO giving off used car salesman vibes who recently mentioned considering allowing their AI to generate porn, and now releasing a flirty AI-girlfriend as his gift to humanity.
  - koe123 2y ago
    On the other hand, Anthropic founders reason for leaving also yielded them angle to start a new successful company, now worth 9+ figures. Given that I’m not sure I’ll take their concerns about the state of OpenAI at face value.
    - HarHarVeryFunny 2y ago
      I've watched all the Dario/Daniela interviews I can find, and I guess that's a fair way of putting it. It seems they genuinely felt (& were) constrained at OpenAI in being able to follow a safety-first agenda, and have articulated it as being "the SF way" to start a new company when you have a new idea (so maybe more cultural than looking for an angle), as well as being able to follow a dream of working together. From what we've seen of OpenAI, it does seem that any dissenting opinions will be bulldozered by Altman's product and money making focus. The cult-like "me too" tweets that huge swathes of the staff periodically send seems highly indicative of the culture. Anthropic does seem genuine about their safety-first agenda, and strategy to lead by example, and have been successful in getting others to follow their safe scaling principles, AI constitution (cf OpenAI's new "Model Spec") and open research into safety/interpretability. This emphasis on safe and steerable models seems to have given them an advantage in corporate adoption.
zer00eyz 2y ago
Sam Altman superalinged them right out the door...
treme 2y ago
Reads like beginnings of a good dystopian movie script
dontupvoteme 2y ago
Turns out we already have alignment, it's called capitalism.
- mbgerring 2y ago
  This is true and we do not talk about it enough. Moreover, Capitalism is itself an unaligned AI, and understanding it through that lens clarifies a great deal.
  - lowmagnet 2y ago
    oh no, it's just a real world reinforcement model
  - panarky 2y ago
    People experience existential terror from AI because it feels like massive, pervasive, implacable forces that we can't understand or control, with the potential to do great harm to our personal lives and to larger social and political systems, where we have zero power to stop it or avoid it or redirect it. Forces that benefit a few at the expense of the many. What many of us are actually experiencing is existential terror about capitalism itself, but we don't have the conceptual framework or vocabulary to describe it that way. It's a cognitive shortcut to look for a definable villain to blame for our fear, and historically that's taken the form of antisemitism, anti-migrant, anti-homeless, even ironically anti-communist, and we see similar corrupted forms of blame in antivax and anti-globalist conspiracy thinking, from both the left and the right. While there are genuine x-risk hazards from AI, it seems like a lot of the current fear is really a corrupted and misplaced fear of having zero control over the foreboding and implacable forces of capitalism itself. AI is hypercapitalism and that is terrifying.
    - kridsdale1 2y ago
      Ted Chiang on the Ezra Klein podcast said basically the same thing: AI Doomerism is actually capitalist anxiety.

I have noticed, and I am concerned that they were the leaders of the Superalignment team.

by pfist 2y ago

I have noticed, and I am concerned that they were the leaders of the Superalignment team.

transcriptase 2y ago

On the other hand, they clearly they weren’t concerned enough about the issue to continue working on it.

dontupvoteme 2y ago
One could argue that at this point openai is being Extended and Embraced by Microsoft and is unlikely to have much autonomy or groundbreaking impact one way or another.
llamaimperative 2y ago
Ah yes, a scientist refusing to work on the hydrogen bomb couldn't have been all that concerned about it.
scarmig 2y ago
If your ostensible purpose is being sidelined by decision makers, trying to fight back is often a good option, but sometimes you fail. Admitting failure and focusing on other approaches is the right choice at that point.
HarHarVeryFunny 2y ago
The Anthropic folk were concerned enough that they left, and are indeed continuing to work on it [AI safety]. Now, we've have the co-leads of the super-alignment/safety team leaving too. Certainly not a good look for OpenAI. There really doesn't seem to be much of a mission left at OpenAI - they have a CEO giving off used car salesman vibes who recently mentioned considering allowing their AI to generate porn, and now releasing a flirty AI-girlfriend as his gift to humanity.
- koe123 2y ago
  On the other hand, Anthropic founders reason for leaving also yielded them angle to start a new successful company, now worth 9+ figures. Given that I’m not sure I’ll take their concerns about the state of OpenAI at face value.
  - HarHarVeryFunny 2y ago
    I've watched all the Dario/Daniela interviews I can find, and I guess that's a fair way of putting it. It seems they genuinely felt (& were) constrained at OpenAI in being able to follow a safety-first agenda, and have articulated it as being "the SF way" to start a new company when you have a new idea (so maybe more cultural than looking for an angle), as well as being able to follow a dream of working together. From what we've seen of OpenAI, it does seem that any dissenting opinions will be bulldozered by Altman's product and money making focus. The cult-like "me too" tweets that huge swathes of the staff periodically send seems highly indicative of the culture. Anthropic does seem genuine about their safety-first agenda, and strategy to lead by example, and have been successful in getting others to follow their safe scaling principles, AI constitution (cf OpenAI's new "Model Spec") and open research into safety/interpretability. This emphasis on safe and steerable models seems to have given them an advantage in corporate adoption.

zer00eyz 2y ago

Sam Altman superalinged them right out the door...

treme 2y ago

Reads like beginnings of a good dystopian movie script

dontupvoteme 2y ago

Turns out we already have alignment, it's called capitalism.

mbgerring 2y ago
This is true and we do not talk about it enough. Moreover, Capitalism is itself an unaligned AI, and understanding it through that lens clarifies a great deal.
- lowmagnet 2y ago
  oh no, it's just a real world reinforcement model
- panarky 2y ago
  People experience existential terror from AI because it feels like massive, pervasive, implacable forces that we can't understand or control, with the potential to do great harm to our personal lives and to larger social and political systems, where we have zero power to stop it or avoid it or redirect it. Forces that benefit a few at the expense of the many. What many of us are actually experiencing is existential terror about capitalism itself, but we don't have the conceptual framework or vocabulary to describe it that way. It's a cognitive shortcut to look for a definable villain to blame for our fear, and historically that's taken the form of antisemitism, anti-migrant, anti-homeless, even ironically anti-communist, and we see similar corrupted forms of blame in antivax and anti-globalist conspiracy thinking, from both the left and the right. While there are genuine x-risk hazards from AI, it seems like a lot of the current fear is really a corrupted and misplaced fear of having zero control over the foreboding and implacable forces of capitalism itself. AI is hypercapitalism and that is terrifying.
  - kridsdale1 2y ago
    Ted Chiang on the Ezra Klein podcast said basically the same thing: AI Doomerism is actually capitalist anxiety.