6 ms·

Is there any evidence that this is a consequence of DEI rather than a deeper technical issue?

Jensson 3y ago
You get 4 images per time and are lucky to get one white person when asked for it, no other model has that issue. Other models has no problems generating black people either, so it isn't that other models only generates white people. So either it isn't a technical issue or Google failed to solve a problem everyone else easily solved. The chances of this having nothing to do with DEI is basically 0.
- ceejayoz 3y ago
  Depending on how broadly you define it, something like 10-30% of the world's population is white. Africa is about 20% of the world population; Asia is 60% of it. One in four sounds about right?
  - cm2012 3y ago
    It does the same if you ask for pictures of past popes, 1945 German soldiers, etc.
    - ceejayoz 3y ago
      It'll also add extra fingers to human hands. Presumably that's not because of DEI guardrails about polydactyly, right? The current state of the art in AI gets things wrong regularly.
      - cm2012 3y ago
        Sure, but this one is from Google adding a tag to make every image of people diverse, not AI randomness.
        ceejayoz 3y ago
        Am I missing something in the link demonstrating that, or is it conjecture?
        cm2012 3y ago
        It's been demonstrated on Twitter a few times, can't find a link handy
        perlclutcher 3y ago
        https://twitter.com/altryne/status/1760358916624719938 https://twitter.com/altryne/status/1760358916624719938 Here's some corporate-lawyer-speak straight from Google: > We are aware that Gemini is offering inaccuracies... > As part of our AI principles, we design our image generation capabilities to reflect our global user base, and we take representation and bias seriously.
        deleted 3y ago
        [deleted]
        ceejayoz 3y ago
        That doesn't back up the assertion; it's easily read as "we make sure our training sets reflect the 85% of the world that doesn't live in Europe and North America". Again, 1/4 white people is statistically what you'd expect.
        u32480932048 3y ago
        Sure, but I see three 200 responses and a 400 - not 1/4 white people as statistically expected.
        withinboredom 3y ago
        Fuck, this is going to sound fucked up... but just because you have a 1/4 chance of getting a random white person from the globe, they generally tend to clump together. For example, you generally find a shitload of Asian people in Asia, white people in Europe, and African people in Africa, and Indian people in India. Probably the only chance where you wouldn't expect this are in heavily colonized places like South Africa, Australia, and the Americas.
      - thepasswordis 3y ago
        The AI will literally scold you for asking it to make white characters, and insists that you need to be inclusive and that it is being intentionally dishonest to force the issue.
      - lmm 3y ago
        This specific thing is a much more blatant class of error, and one that has been known to occur in several previous models because of DEI systems (e.g. in cases where prompts have been leaked), and has never been known to occur for any other reason. Yes, it's conceivable that Google's newer, beter-than-ever-before AI system somehow has a fundamental technical problem that coincidentally just happens to cause the same kind of bad output as previous hamfisted DEI systems, but come on, you don't really believe that. (Or if you do, how much do you want to bet? I would absolutely stake a significant proportion of my net worth - say, $20k - on this)
        ceejayoz 3y ago
        > has never been known to occur for any other reason Of course it has. Again, these things regularly give humans extra fingers and arms. They don't even know what humans fundamentally look like. On the flip side, humans are shitty at recognizing bias. This comment thread stems from someone complaining the AI only rarely generated white people, but that's statistically accurate. It feels biased to someone in a majority-white nation with majority-white friends and coworkers, but it fundamentally isn't. I don't doubt that there are some attempts to get LLMs to go outside the "white westerner" bubble in training sets and prompts. I suspect the extent of it is also deeply exaggerated by those who like to throw around woke-this and woke-that as derogatories.
        lmm 3y ago
        > Of course it has. Again, these things regularly give humans extra fingers and arms. They don't even know what humans fundamentally look like. > This comment thread stems from someone complaining the AI only rarely generated white people, but that's statistically accurate. It feels biased to someone in a majority-white nation with majority-white friends and coworkers, but it fundamentally isn't. So the AI is simultaneously too dumb to figure out what humans look like, but also so super smart that it uses precisely accurate racial proportions when generating people (not because it's been specifically adjusted to, but naturally)? Bullshit. > I don't doubt that there are some attempts to get LLMs to go outside the "white westerner" bubble in training sets and prompts. I suspect the extent of it is also deeply exaggerated by those who like to throw around woke-this and woke-that as derogatories. You're dodging the question. Do you actually believe the reason that the last example in the article looks very much not like a man is a deep technical issue, or a DEI initiative? If the former, how much are you willing to bet? If the latter, why are you throwing out these insincere arguments?
  - wruza 3y ago
    If it does, shouldn't there be 60% asians?
123yawaworht456 3y ago
yes, there's irrefutable evidence that models are wrangled into abiding the commissars' vision rather than just do their job and output the product of their training data. https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf
nickthegreek 3y ago
I dont think so. My boss wanted me to generate a birthday image for a co-worker of a John Cena flyfishing. ChatGPT refused to do so. So I had to move to describing the type of person John Cena is instead of using his name. I kept giving me bearded people no matter what. I thought this would be the perfect time to try out Gemini for the first time. Well shit, It wont even give me a white guy. But all the black dudes are beardless. update: google agrees there is an issue. https://news.ycombinator.com/item?id=39459270 https://news.ycombinator.com/item?id=39459270
- 8f2ab37a-ed6c 3y ago
  It feels that the image generation it offers is perfect for some sort of a California-Corporate Style, e.g. you ask it for a "photo of people at the board room" or "people at the company cafeteria" and you get the corporate friendly ratio of colors, ability-levels, sizes etc. See Google's various image assets: https://www.google.com/about/careers/applications/ https://www.google.com/about/careers/applications/ . It's great for coastal and urban marketing brochures. But then then same California Corporate style makes no sense for historical images, so perhaps this is where Midjourney comes in.
minimaxir 3y ago
When DALL-E 2 was released in 2022, OpenAI published an article noting that the inclusion of guardrails was a correction for bias: https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2 https://openai.com/blog/reducing-bias-and-improving-safety-i... It was widely criticized back then: the fact that Google both brought it back and made it more prominent is weird. Notably, OpenAI's implementation is more scoped.
mike_d 3y ago
It is possible Google tried to avoid likenesses of well known people by removing any image from the training data that contained a face and then including a controlled set of people images. If you give a contractor a project that you want 200k images of people who are not famous, they will send teams to regions where you may only have to pay each person a few dollars to be photographed. Likely SE Asia and Africa.
allmadhare22 3y ago
Depending on what you ask for, it injects the word 'diverse' into the response description, so it's pretty obvious they're brute forcing diversity into it. E.g. "Generate me an image of a family" and you will get back "Here are some images of a diverse family".
sotasota 3y ago
https://pbs.twimg.com/media/GG1eyKjXQAA1FxU?format=jpg&name=large https://pbs.twimg.com/media/GG1eyKjXQAA1FxU?format=jpg&name=... https://cdn.sanity.io/images/cjtc1tnd/production/912b6b5aaccb7955fb5ce48c1f055b4d579224dd-1156x1058.png?w=1156&auto=format https://cdn.sanity.io/images/cjtc1tnd/production/912b6b5aacc... https://pbs.twimg.com/media/GG1ThfsWUAAp-SO?format=jpg&name=large https://pbs.twimg.com/media/GG1ThfsWUAAp-SO?format=jpg&name=... https://cdn.sanity.io/images/cjtc1tnd/production/e2810c02ff6ac0033bccda8e8d2a07f4935c0d7e-2232x962.png?w=2000&auto=format https://cdn.sanity.io/images/cjtc1tnd/production/e2810c02ff6... https://pbs.twimg.com/media/GG1MnepXwAAkPL6?format=jpg&name=large https://pbs.twimg.com/media/GG1MnepXwAAkPL6?format=jpg&name=... https://pbs.twimg.com/media/GG0BLVsbMAARZXr?format=jpg&name=medium https://pbs.twimg.com/media/GG0BLVsbMAARZXr?format=jpg&name=...
- flumpcakes 3y ago
  I don't understand how people could even argue that this is in any way acceptable. Fighting "bias" has become some boogyman and anything "non-white" is now beyond reproach. Shocking.
  - gedy 3y ago
    Seriously, I've basically written off using Gemini for good after this HR style nonsense. It's a shame that Google, who invented much of this tech, is so crippled by their own people's politics.
  - AnarchismIsCool 3y ago
    Fighting bias is a good thing, you'd have to be pretty...er...biased to believe otherwise. Bias is fundamentally a distortion or deviation from objective reality. This, on the other hand, is just fucking stupid political showboating that's hurting their SV white knight cause. It's just differently flavored bias
- gs17 3y ago
  "I can't generate white British royalty because they exist, but I can make up black ones" is pretty close to an actually valid reason.

You get 4 images per time and are lucky to get one white person when asked for it, no other model has that issue. Other models has no problems generating black people either, so it isn't that other models only generates white people. So either it isn't a technical issue or Google failed to solve a problem everyone else easily solved. The chances of this having nothing to do with DEI is basically 0.

ceejayoz 3y ago
Depending on how broadly you define it, something like 10-30% of the world's population is white. Africa is about 20% of the world population; Asia is 60% of it. One in four sounds about right?
- cm2012 3y ago
  It does the same if you ask for pictures of past popes, 1945 German soldiers, etc.
  - ceejayoz 3y ago
    It'll also add extra fingers to human hands. Presumably that's not because of DEI guardrails about polydactyly, right? The current state of the art in AI gets things wrong regularly.
    - cm2012 3y ago
      Sure, but this one is from Google adding a tag to make every image of people diverse, not AI randomness.
      - ceejayoz 3y ago
        Am I missing something in the link demonstrating that, or is it conjecture?
        cm2012 3y ago
        It's been demonstrated on Twitter a few times, can't find a link handy
        perlclutcher 3y ago
        https://twitter.com/altryne/status/1760358916624719938 https://twitter.com/altryne/status/1760358916624719938 Here's some corporate-lawyer-speak straight from Google: > We are aware that Gemini is offering inaccuracies... > As part of our AI principles, we design our image generation capabilities to reflect our global user base, and we take representation and bias seriously.
        deleted 3y ago
        [deleted]
        ceejayoz 3y ago
        That doesn't back up the assertion; it's easily read as "we make sure our training sets reflect the 85% of the world that doesn't live in Europe and North America". Again, 1/4 white people is statistically what you'd expect.
        u32480932048 3y ago
        Sure, but I see three 200 responses and a 400 - not 1/4 white people as statistically expected.
        withinboredom 3y ago
        Fuck, this is going to sound fucked up... but just because you have a 1/4 chance of getting a random white person from the globe, they generally tend to clump together. For example, you generally find a shitload of Asian people in Asia, white people in Europe, and African people in Africa, and Indian people in India. Probably the only chance where you wouldn't expect this are in heavily colonized places like South Africa, Australia, and the Americas.
    - thepasswordis 3y ago
      The AI will literally scold you for asking it to make white characters, and insists that you need to be inclusive and that it is being intentionally dishonest to force the issue.
    - lmm 3y ago
      This specific thing is a much more blatant class of error, and one that has been known to occur in several previous models because of DEI systems (e.g. in cases where prompts have been leaked), and has never been known to occur for any other reason. Yes, it's conceivable that Google's newer, beter-than-ever-before AI system somehow has a fundamental technical problem that coincidentally just happens to cause the same kind of bad output as previous hamfisted DEI systems, but come on, you don't really believe that. (Or if you do, how much do you want to bet? I would absolutely stake a significant proportion of my net worth - say, $20k - on this)
      - ceejayoz 3y ago
        > has never been known to occur for any other reason Of course it has. Again, these things regularly give humans extra fingers and arms. They don't even know what humans fundamentally look like. On the flip side, humans are shitty at recognizing bias. This comment thread stems from someone complaining the AI only rarely generated white people, but that's statistically accurate. It feels biased to someone in a majority-white nation with majority-white friends and coworkers, but it fundamentally isn't. I don't doubt that there are some attempts to get LLMs to go outside the "white westerner" bubble in training sets and prompts. I suspect the extent of it is also deeply exaggerated by those who like to throw around woke-this and woke-that as derogatories.
        lmm 3y ago
        > Of course it has. Again, these things regularly give humans extra fingers and arms. They don't even know what humans fundamentally look like. > This comment thread stems from someone complaining the AI only rarely generated white people, but that's statistically accurate. It feels biased to someone in a majority-white nation with majority-white friends and coworkers, but it fundamentally isn't. So the AI is simultaneously too dumb to figure out what humans look like, but also so super smart that it uses precisely accurate racial proportions when generating people (not because it's been specifically adjusted to, but naturally)? Bullshit. > I don't doubt that there are some attempts to get LLMs to go outside the "white westerner" bubble in training sets and prompts. I suspect the extent of it is also deeply exaggerated by those who like to throw around woke-this and woke-that as derogatories. You're dodging the question. Do you actually believe the reason that the last example in the article looks very much not like a man is a deep technical issue, or a DEI initiative? If the former, how much are you willing to bet? If the latter, why are you throwing out these insincere arguments?
- wruza 3y ago
  If it does, shouldn't there be 60% asians?

123yawaworht456 3y ago

yes, there's irrefutable evidence that models are wrangled into abiding the commissars' vision rather than just do their job and output the product of their training data. https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf

nickthegreek 3y ago

I dont think so. My boss wanted me to generate a birthday image for a co-worker of a John Cena flyfishing. ChatGPT refused to do so. So I had to move to describing the type of person John Cena is instead of using his name. I kept giving me bearded people no matter what. I thought this would be the perfect time to try out Gemini for the first time. Well shit, It wont even give me a white guy. But all the black dudes are beardless. update: google agrees there is an issue. https://news.ycombinator.com/item?id=39459270 https://news.ycombinator.com/item?id=39459270

8f2ab37a-ed6c 3y ago
It feels that the image generation it offers is perfect for some sort of a California-Corporate Style, e.g. you ask it for a "photo of people at the board room" or "people at the company cafeteria" and you get the corporate friendly ratio of colors, ability-levels, sizes etc. See Google's various image assets: https://www.google.com/about/careers/applications/ https://www.google.com/about/careers/applications/ . It's great for coastal and urban marketing brochures. But then then same California Corporate style makes no sense for historical images, so perhaps this is where Midjourney comes in.

minimaxir 3y ago

When DALL-E 2 was released in 2022, OpenAI published an article noting that the inclusion of guardrails was a correction for bias: https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2 https://openai.com/blog/reducing-bias-and-improving-safety-i... It was widely criticized back then: the fact that Google both brought it back and made it more prominent is weird. Notably, OpenAI's implementation is more scoped.

mike_d 3y ago

It is possible Google tried to avoid likenesses of well known people by removing any image from the training data that contained a face and then including a controlled set of people images. If you give a contractor a project that you want 200k images of people who are not famous, they will send teams to regions where you may only have to pay each person a few dollars to be photographed. Likely SE Asia and Africa.

allmadhare22 3y ago

Depending on what you ask for, it injects the word 'diverse' into the response description, so it's pretty obvious they're brute forcing diversity into it. E.g. "Generate me an image of a family" and you will get back "Here are some images of a diverse family".

sotasota 3y ago

https://pbs.twimg.com/media/GG1eyKjXQAA1FxU?format=jpg&name=large https://pbs.twimg.com/media/GG1eyKjXQAA1FxU?format=jpg&name=... https://cdn.sanity.io/images/cjtc1tnd/production/912b6b5aaccb7955fb5ce48c1f055b4d579224dd-1156x1058.png?w=1156&auto=format https://cdn.sanity.io/images/cjtc1tnd/production/912b6b5aacc... https://pbs.twimg.com/media/GG1ThfsWUAAp-SO?format=jpg&name=large https://pbs.twimg.com/media/GG1ThfsWUAAp-SO?format=jpg&name=... https://cdn.sanity.io/images/cjtc1tnd/production/e2810c02ff6ac0033bccda8e8d2a07f4935c0d7e-2232x962.png?w=2000&auto=format https://cdn.sanity.io/images/cjtc1tnd/production/e2810c02ff6... https://pbs.twimg.com/media/GG1MnepXwAAkPL6?format=jpg&name=large https://pbs.twimg.com/media/GG1MnepXwAAkPL6?format=jpg&name=... https://pbs.twimg.com/media/GG0BLVsbMAARZXr?format=jpg&name=medium https://pbs.twimg.com/media/GG0BLVsbMAARZXr?format=jpg&name=...

flumpcakes 3y ago
I don't understand how people could even argue that this is in any way acceptable. Fighting "bias" has become some boogyman and anything "non-white" is now beyond reproach. Shocking.
- gedy 3y ago
  Seriously, I've basically written off using Gemini for good after this HR style nonsense. It's a shame that Google, who invented much of this tech, is so crippled by their own people's politics.
- AnarchismIsCool 3y ago
  Fighting bias is a good thing, you'd have to be pretty...er...biased to believe otherwise. Bias is fundamentally a distortion or deviation from objective reality. This, on the other hand, is just fucking stupid political showboating that's hurting their SV white knight cause. It's just differently flavored bias
gs17 3y ago
"I can't generate white British royalty because they exist, but I can make up black ones" is pretty close to an actually valid reason.