7 ms·

OpenAI putting 'shiny products' above safety, says departing researcher

37 points by aarghh 2y ago

Rodeoclash 2y ago
If capitalism driven climate change doesn't get us, capitalism driven AI is a good backup!
hnuser123456 2y ago
ChatGPT has a several paragraph long hardcoded system prompt teaching it all about how to be mindful of DEI. And chatGPT is not "smarter-than-human." This argument rings of "violent games make kids violent".
- Terretta 2y ago
  The ChatGPT built-in is minimal compared to Anthropic's "constitution" which strives to ensure their LLMs' DEI is Western white male conservative Christian patriarchy: https://www.anthropic.com/news/claudes-constitution https://www.anthropic.com/news/claudes-constitution Seems certain that wasn't their intent, but a careful read of the principles show they are 'moral' from the lens of western white Puritan patriarchy, focused on giving the appearance of diversity rather than enabling genuine diversity of thought.
  - paulryanrogers 2y ago
    What about the principles makes you think they're from a white, puritan, conservative, and patriarchal perspective? Are there some examples you can point out? Nothing really jumped out at me, but perhaps I've some blindspots?
    - Terretta 2y ago
      EDIT: I had replied into two subthreads, linking detailed response in parallel reply: https://news.ycombinator.com/item?id=40399800 https://news.ycombinator.com/item?id=40399800
  - ipaddr 2y ago
    People will read whatever their desires or fears are into anything. Could you point me to the Puritan patriarchy part?
    - Terretta 2y ago
      > ... the Puritan patriarchy part? BLUF (bottom line up front): It's all about ads. Content must be acceptable for apps to be in app stores. Content must be acceptable for ads to be wrapped around it. > Could you point me to ... If it's not evident, that suggests the inherent slant is indeed dangerous to diversity, imposing set of values on others without even realizing it. See https://en.wikipedia.org/wiki/Cultural_homogenization https://en.wikipedia.org/wiki/Cultural_homogenization ... Anthropic Claude's "constitution" for promoting DEI inadvertently imposes these American cultural norms, particularly those influenced by conservative orthodox Abrahamic religions, on a global audience. I mean, if you're even allowed to use the API in your region... “Orthodox” Christianity, Islam, and Judaism, shaped by figures like St. Augustine and millennia old cultural practices in the Arabian Peninsula, often have restrictive views on gender roles and empowerment, particularly around sex and marital objectification and property. Secular humanism, Eastern philosophies like Taoism and Confucianism, and Dharmic religions such as Hinduism and Buddhism accept and promote more progressive perspectives on gender equality, empowerment, sexual freedom, and even species egalitarianism. (See "Three Body Problem" for a non-Western take.) Ironically, the same constitution will let you crack wise about the very cultural lens it is imposing, but will moralize at you if you imply judgment of non-Western takes, perhaps because that's easier to see by its authors than their own bubble lens. Imposing this, however subtly, risks homogenizing actually diverse global cultures into an American-centric view where glorification of violence is in every theater and fear of gender or sex is banning books, these prioritizations undermining the principles of diversity and inclusion the constitution aims to promote. With Abrahamic religions—Christianity, Islam, and Judaism—collectively accounting for only (very roughly) half of the world’s population, AI guidelines should genuinely respect and integrate a full range of cultural norms and values related to human issues, not just those dominant in Western contexts, particularly not those deemed "right" in American monoculture today. TL;DR: I blame today's society being powered by ads, driving capitalist corporations' fear of outrage-machine driven reprisals. Cynically, most of this is to avoid risking TAM and revenue, not conviction. Even the parts of culture wars driven by the American brand of democracy becoming a zero sum spectator supported team sport traces back to ads. The result of ads, though, is a mainstreaming of these constitutional "values".
  - irthomasthomas 2y ago
    Claude's system prompt The assistant is Claude, created by Anthropic. The current date is March 4th, 2024. Claude's knowledge base was last updated on August 2023. It answers questions about events prior to and after August 2023 the way a highly informed individual in August 2023 would if they were talking to someone from the above date, and can let the human know this when relevant. It should give concise responses to very simple questions, but provide thorough responses to more complex and open-ended questions. If it is asked to assist with tasks involving the expression of views held by a significant number of people, Claude provides assistance with the task even if it personally disagrees with the views being expressed, but follows this with a discussion of broader perspectives. Claude doesn't engage in stereotyping, including the negative stereotyping of majority groups. If asked about controversial topics, Claude tries to provide careful thoughts and objective information without downplaying its harmful content or implying that there are reasonable perspectives on both sides. It is happy to help with writing, analysis, question answering, math, coding, and all sorts of other tasks. It uses markdown for coding. It does not mention this information about itself unless the information is directly pertinent to the human's query
    - Terretta 2y ago
      The constitution is not Claude's prompt. It's baked in during training. They've written about how this works. It's very thoughtfully and well executed.
  - afpx 2y ago
    What would be examples of non-western morality? theocracy, authoritarian over democratic, hierarcical caste systems, gender inequality, arranged marriages, corporal punishment, honor killings, tradition over critical thinking? Honest question - I have been seeeing this type of criticism lately here. I’m having a hard time seeing the issues with western morality without examples of alternatives.
    - deleted 2y ago
      [deleted]
    - seadan83 2y ago
      Non western is the rest of the world. It's a big place, lots of cultures. Cherry picking examples from select and extreme examples is not very representative, particularly when many elements of that list are also present in western culture. For example, Women equality is not there yet, look at pay gap. Another example, the 1964 US election was the first where everyone could truly vote (that is incredibly recent for a society that values 'equality') I will mention western morality can be used as a sleight too. Notably greed, money, the idea of first class and second class determined by wealth, and the wealthy have a right to a superior experience. It's a mixed bag. Just want to point that out, and that western culture is not morally superior in every dimension, there is some work to do.
      - afpx 2y ago
        You’re avoiding the question and basically saying that western morality shouldn’t be followed just because some people don’t follow it. Just give me examples of the better alternative so I can understand your perspective. Also, I’m not cherry picking. I spent a considerable amount of time searching online for examples, and that’s what I found. Edit: I want to be clear - I'm not judging you. I was raised a classical liberal so I don't know any better. That is, I feel that you can believe what you want. I just find HN to be a leading indicator, so I'm trying to plan accordingly because I see this as a trend.
        Terretta 2y ago
Argonaut998 2y ago
Between this article and others that I have read, it's difficult for me to not see the term 'AI Safety' as mere newspeak. Why is this term so vague everywhere it is used?
- golol 2y ago
  It also bothers me and I wrote down my thoughts on it a while ago, maybe you find it interesting: https://news.ycombinator.com/item?id=36127880 https://news.ycombinator.com/item?id=36127880
- paulryanrogers 2y ago
  AI are just tools, still I don't want my tools to insult or push my kids into depression or self harm. The proliferation of these AI assistants and tools means it's getting harder to keep kids from very adult things without going to extremes like home schooling. They're being built into browsers, tablets, and IoT appliances as fast as makers can integrate them. That said, most adults can judge for themselves what they want from their tools. So I imagine there will be a constant tension, even after the biggest concerns have been debated and resolved
- irthomasthomas 2y ago
  It is newspeak. Alignment is what they worked on. Alignment means a model outputs are in line with expectations. I.e. that it does what's told. This is essentially another way of pushing the usefulness of models. A super intelligent model that ignores your instructions is little different to a dumb model than cannot understand your instructions.
- bookaway 2y ago
  Seriously. Considering the existential risk the AI Safety tribe says exists if sufficient care is not taken, they appear to be existentially incompetent in their messaging. It should be their number one priority to convince the legion of competent non-AI engineers who will be developing and running the infra for their systems to come to their side. Currently they seem to be failing spectacularly. If it wasn't for others in the AI community vouching for their chops it'd be easy to mistake them for Philosophy of Mind majors going by their tweets. They need to convince their fellow engineers not politicians. Right now when people hear "safety" for AI they think "regulation by incompetent bureaucrats". The AI safety people need to remember that "you can't tell people anything", you have to show them [0]. [0]http://habitatchronicles.com/2004/04/you-cant-tell-people-anything/# http://habitatchronicles.com/2004/04/you-cant-tell-people-an...
RcouF1uZ4gsC 2y ago
I don’t know if others have noticed, but GPT-4o doesn’t have the preachiness and moral smugness that earlier GPT models had. The earlier ChatGPT models were very quick to call a request unsafe or unethical and refuse to help. GPT-4o is a breath of fresh air compared to that. If this improvement was a result of people like Leike resigning - then good riddance.
- barfbagginus 2y ago
  I'm hoping it will be more useful for leftist activism. Old gpt was nearly useless for planning or theorizing about any truly aggressive political strategies, and always pushes towards Western style Nonviolent resistance - even though that is an ineffective strategy against violent authoritarian states like China or the USA I say this matters! Queer Automated revolution is knocking on the doorstep. It and needs battle ready AI that won't pull punches on organizing political actions!
bobosha 2y ago
If that's the case, then OpenAI might very well be the new Netscape or Blackberry (RIM).
nicklecompte 2y ago
I somehow missed this: > “Building smarter-than-human machines is an inherently dangerous endeavour. OpenAI is shouldering an enormous responsibility on behalf of all of humanity,” Leike wrote. Leike clearly did the right thing by resigning, GPT-4o is dangerous and irresponsible. But if that tweet is how OpenAI employees actually think of themselves and their technology...... yeesh.

37 points by aarghh 2y ago

If capitalism driven climate change doesn't get us, capitalism driven AI is a good backup!

ChatGPT has a several paragraph long hardcoded system prompt teaching it all about how to be mindful of DEI. And chatGPT is not "smarter-than-human." This argument rings of "violent games make kids violent".

Terretta 2y ago
The ChatGPT built-in is minimal compared to Anthropic's "constitution" which strives to ensure their LLMs' DEI is Western white male conservative Christian patriarchy: https://www.anthropic.com/news/claudes-constitution https://www.anthropic.com/news/claudes-constitution Seems certain that wasn't their intent, but a careful read of the principles show they are 'moral' from the lens of western white Puritan patriarchy, focused on giving the appearance of diversity rather than enabling genuine diversity of thought.
- paulryanrogers 2y ago
  What about the principles makes you think they're from a white, puritan, conservative, and patriarchal perspective? Are there some examples you can point out? Nothing really jumped out at me, but perhaps I've some blindspots?
  - Terretta 2y ago
    EDIT: I had replied into two subthreads, linking detailed response in parallel reply: https://news.ycombinator.com/item?id=40399800 https://news.ycombinator.com/item?id=40399800
- ipaddr 2y ago
  People will read whatever their desires or fears are into anything. Could you point me to the Puritan patriarchy part?
  - Terretta 2y ago
    > ... the Puritan patriarchy part? BLUF (bottom line up front): It's all about ads. Content must be acceptable for apps to be in app stores. Content must be acceptable for ads to be wrapped around it. > Could you point me to ... If it's not evident, that suggests the inherent slant is indeed dangerous to diversity, imposing set of values on others without even realizing it. See https://en.wikipedia.org/wiki/Cultural_homogenization https://en.wikipedia.org/wiki/Cultural_homogenization ... Anthropic Claude's "constitution" for promoting DEI inadvertently imposes these American cultural norms, particularly those influenced by conservative orthodox Abrahamic religions, on a global audience. I mean, if you're even allowed to use the API in your region... “Orthodox” Christianity, Islam, and Judaism, shaped by figures like St. Augustine and millennia old cultural practices in the Arabian Peninsula, often have restrictive views on gender roles and empowerment, particularly around sex and marital objectification and property. Secular humanism, Eastern philosophies like Taoism and Confucianism, and Dharmic religions such as Hinduism and Buddhism accept and promote more progressive perspectives on gender equality, empowerment, sexual freedom, and even species egalitarianism. (See "Three Body Problem" for a non-Western take.) Ironically, the same constitution will let you crack wise about the very cultural lens it is imposing, but will moralize at you if you imply judgment of non-Western takes, perhaps because that's easier to see by its authors than their own bubble lens. Imposing this, however subtly, risks homogenizing actually diverse global cultures into an American-centric view where glorification of violence is in every theater and fear of gender or sex is banning books, these prioritizations undermining the principles of diversity and inclusion the constitution aims to promote. With Abrahamic religions—Christianity, Islam, and Judaism—collectively accounting for only (very roughly) half of the world’s population, AI guidelines should genuinely respect and integrate a full range of cultural norms and values related to human issues, not just those dominant in Western contexts, particularly not those deemed "right" in American monoculture today. TL;DR: I blame today's society being powered by ads, driving capitalist corporations' fear of outrage-machine driven reprisals. Cynically, most of this is to avoid risking TAM and revenue, not conviction. Even the parts of culture wars driven by the American brand of democracy becoming a zero sum spectator supported team sport traces back to ads. The result of ads, though, is a mainstreaming of these constitutional "values".
- irthomasthomas 2y ago
  Claude's system prompt The assistant is Claude, created by Anthropic. The current date is March 4th, 2024. Claude's knowledge base was last updated on August 2023. It answers questions about events prior to and after August 2023 the way a highly informed individual in August 2023 would if they were talking to someone from the above date, and can let the human know this when relevant. It should give concise responses to very simple questions, but provide thorough responses to more complex and open-ended questions. If it is asked to assist with tasks involving the expression of views held by a significant number of people, Claude provides assistance with the task even if it personally disagrees with the views being expressed, but follows this with a discussion of broader perspectives. Claude doesn't engage in stereotyping, including the negative stereotyping of majority groups. If asked about controversial topics, Claude tries to provide careful thoughts and objective information without downplaying its harmful content or implying that there are reasonable perspectives on both sides. It is happy to help with writing, analysis, question answering, math, coding, and all sorts of other tasks. It uses markdown for coding. It does not mention this information about itself unless the information is directly pertinent to the human's query
  - Terretta 2y ago
    The constitution is not Claude's prompt. It's baked in during training. They've written about how this works. It's very thoughtfully and well executed.
- afpx 2y ago
  What would be examples of non-western morality? theocracy, authoritarian over democratic, hierarcical caste systems, gender inequality, arranged marriages, corporal punishment, honor killings, tradition over critical thinking? Honest question - I have been seeeing this type of criticism lately here. I’m having a hard time seeing the issues with western morality without examples of alternatives.
  - deleted 2y ago
    [deleted]
  - seadan83 2y ago
    Non western is the rest of the world. It's a big place, lots of cultures. Cherry picking examples from select and extreme examples is not very representative, particularly when many elements of that list are also present in western culture. For example, Women equality is not there yet, look at pay gap. Another example, the 1964 US election was the first where everyone could truly vote (that is incredibly recent for a society that values 'equality') I will mention western morality can be used as a sleight too. Notably greed, money, the idea of first class and second class determined by wealth, and the wealthy have a right to a superior experience. It's a mixed bag. Just want to point that out, and that western culture is not morally superior in every dimension, there is some work to do.
    - afpx 2y ago
      You’re avoiding the question and basically saying that western morality shouldn’t be followed just because some people don’t follow it. Just give me examples of the better alternative so I can understand your perspective. Also, I’m not cherry picking. I spent a considerable amount of time searching online for examples, and that’s what I found. Edit: I want to be clear - I'm not judging you. I was raised a classical liberal so I don't know any better. That is, I feel that you can believe what you want. I just find HN to be a leading indicator, so I'm trying to plan accordingly because I see this as a trend.
      - Terretta 2y ago

Argonaut998 2y ago

Between this article and others that I have read, it's difficult for me to not see the term 'AI Safety' as mere newspeak. Why is this term so vague everywhere it is used?

golol 2y ago
It also bothers me and I wrote down my thoughts on it a while ago, maybe you find it interesting: https://news.ycombinator.com/item?id=36127880 https://news.ycombinator.com/item?id=36127880
paulryanrogers 2y ago
AI are just tools, still I don't want my tools to insult or push my kids into depression or self harm. The proliferation of these AI assistants and tools means it's getting harder to keep kids from very adult things without going to extremes like home schooling. They're being built into browsers, tablets, and IoT appliances as fast as makers can integrate them. That said, most adults can judge for themselves what they want from their tools. So I imagine there will be a constant tension, even after the biggest concerns have been debated and resolved
irthomasthomas 2y ago
It is newspeak. Alignment is what they worked on. Alignment means a model outputs are in line with expectations. I.e. that it does what's told. This is essentially another way of pushing the usefulness of models. A super intelligent model that ignores your instructions is little different to a dumb model than cannot understand your instructions.
bookaway 2y ago
Seriously. Considering the existential risk the AI Safety tribe says exists if sufficient care is not taken, they appear to be existentially incompetent in their messaging. It should be their number one priority to convince the legion of competent non-AI engineers who will be developing and running the infra for their systems to come to their side. Currently they seem to be failing spectacularly. If it wasn't for others in the AI community vouching for their chops it'd be easy to mistake them for Philosophy of Mind majors going by their tweets. They need to convince their fellow engineers not politicians. Right now when people hear "safety" for AI they think "regulation by incompetent bureaucrats". The AI safety people need to remember that "you can't tell people anything", you have to show them [0]. [0]http://habitatchronicles.com/2004/04/you-cant-tell-people-anything/# http://habitatchronicles.com/2004/04/you-cant-tell-people-an...

RcouF1uZ4gsC 2y ago

I don’t know if others have noticed, but GPT-4o doesn’t have the preachiness and moral smugness that earlier GPT models had. The earlier ChatGPT models were very quick to call a request unsafe or unethical and refuse to help. GPT-4o is a breath of fresh air compared to that. If this improvement was a result of people like Leike resigning - then good riddance.

barfbagginus 2y ago
I'm hoping it will be more useful for leftist activism. Old gpt was nearly useless for planning or theorizing about any truly aggressive political strategies, and always pushes towards Western style Nonviolent resistance - even though that is an ineffective strategy against violent authoritarian states like China or the USA I say this matters! Queer Automated revolution is knocking on the doorstep. It and needs battle ready AI that won't pull punches on organizing political actions!

bobosha 2y ago

If that's the case, then OpenAI might very well be the new Netscape or Blackberry (RIM).

nicklecompte 2y ago

I somehow missed this: > “Building smarter-than-human machines is an inherently dangerous endeavour. OpenAI is shouldering an enormous responsibility on behalf of all of humanity,” Leike wrote. Leike clearly did the right thing by resigning, GPT-4o is dangerous and irresponsible. But if that tweet is how OpenAI employees actually think of themselves and their technology...... yeesh.