8 ms·

This doesn't matter until someone tries suing them for it, right? And as I understand it, you don't really have a case without evidence that the hiring algorit

This doesn't matter until someone tries suing them for it, right? And as I understand it, you don't really have a case without evidence that the hiring algorithm is discriminating against people with disabilities. How would an individual even begin to gather that evidence?

marian_ivanco 4y ago
I am not sure, but if I remember correctly employer must prove they are not discriminating. And just because they are using AI they are not immune to litigation.
- danarmak 4y ago
  How can the employer prove a negative? At most I imagine the plaintiff is allowed to do discovery, and then has to prove positive discrimination based on that.
  - radu_floricica 4y ago
  - vajrabum 4y ago
    If you read the document again (?) maybe you'll see it's not about proving a negative. Instead, it's a standard of due care. Did you check whether using some particular tool illegally discriminates and document that consideration? From the document itself: "Clarifies that, when designing or choosing technological tools, employers must consider how their tools could impact different disabilities; Explains employers’ obligations under the ADA when using algorithmic decision-making tools, including when an employer must provide a reasonable accommodation;"
  - HWR_14 4y ago
    If it's a civil case, it's just the preponderance of the evidence. The jury just has to decide who they think is more likely to be correct.
- rascul 4y ago
  > I am not sure, but if I remember correctly employer must prove they are not discriminating. That seems backwards, at least in the US.
dsr_ 4y ago
The process of gathering evidence after the suit has started is called discovery. There are three major kinds of evidence that would be useful here. Most useful but least likely: email inside the company in which someone says "make sure that this doesn't select too many people with disabilities" or "it's fine that the system isn't selecting people with disabilities, carry on". Useful and very likely: prima facie evidence that the software doesn't make necessary reasonable accomodations - a video captcha without an audio alternative, things like that. Fairly useful and of moderate likelihood: statistical evidence that whatever the company said or did, it has the effect of unfairly rejecting applicants with disabilities.
- pfdietz 4y ago
  And one could go a step further: run the software itself and show that it discriminates. One doesn't just have to look at past performance of the software; it can be fed inputs tailored to bring out discriminatory performance. In this way software is more dangerous to the defendant than manual hiring practices; you can't do the same thing to an employee making hiring decisions.
  - twofornone 4y ago
    How would you make sure that the supplied version has the same weights as the production version? And wouldn't the weights and architecture be refined over time anyway?
    - dogleash 4y ago
      > How would you make sure that the supplied version has the same weights as the production version? You just run the same software (with the same state database, if applicable). Oh wait, I forgot, nobody knows or cares what software they're running. As long as the website is pretty and we can outsource the sysop burden, well then, who needs representative testing or the ability to audit?
    - emiliobumachar 4y ago
      Perjury laws. Once a judge has commanded you to give the same AI, you either give the same AI, or truthfully explain that you can't. Any deviation from that and everyone complicit is risking jail time, not just money. "this is the June 2020 version, this is the current version, we have no back ups in between" is acceptable if true. Destroying or omitting an existing version is not.
jejones3141 4y ago
These days, disparate impact is taken as evidence of discrimination, so it's easy to find "discrimination".
- mkr-hn 4y ago
  What's the difference? Discrimination is an effect more than an intent. Most people are decent and well-intentioned and don't mean to discriminate, but it still happens. If there's a disparate impact, what do you imagine causes that if not discrimination? Remembering that we all have implicit bias and it doesn't make you a mustache-twirling villain.
  - twofornone 4y ago
    >If there's a disparate impact, what do you imagine causes that if not discrimination? 20+ years of environmental differences, especially culture? The disabilities themselves? Genes? Nothing about human nature suggests that all demographics are equally competent in all fields, regardless of whether you group people by race, gender, political preferences, geography, religion, etc. To believe otherwise is fundamentally unscientific, though it's socially unacceptable to acknowledge this truth. >Remembering that we all have implicit bias This doesn't tell you anything about the direction of this bias, but the zeitgeist is such that it is nearly always assumed to go in one direction, and that's deeply problematic. It's an overcorrection that looks an awful lot like institutional discrimination. >Remembering that we all have implicit bias and it doesn't make you a mustache-twirling villain. Except pushing back against unilateral accusations of bias if you belong to one, and only one, specific demographic, you effectively are treated like a mustache-twirling villain. No one is openly complaining about "too much diversity" and keeping their job at the moment. That's bias.
    - etchalon 4y ago
      There is no scientific literature which confirms that any specific demographic quality determine's an individuals capability at any job or task. What does exist is, at best, shows mild correlation over large populations, but nothing binary or deterministic at an individual level. To whit, even if your demographic group, on average, is slightly more or less successful in a specific metric, there is no scientific basis for individualized discrimination. It's "not socially unacceptable to acknowledge this truth", it's socially unacceptable to pretend discrimination is justified.
mkr-hn 4y ago
This is what that demographic survey at the end of job applications is for. It can reveal changes in hiring trends, especially in the demographics of who doesn't get hired. I don't know how well it works in practice.
- hallway_monitor 4y ago
  I am a person, not a statistic. I always decline to answer these surveys; I encourage others to do the same.
  - mkr-hn 4y ago
    Those are for persuading people who do see you as a statistic. You can unilaterally disarm if you like, but they're going to keep discriminating until they see data that proves they're discriminating. Far too few people are persuaded by other means.
  - skrbjc 4y ago
    I’m trying to but my employer has said they will use “observer-identified” info to fill it in for me. I find it ridiculous that I can’t object to having someone guess my race and report that to the government.
    - mkr-hn 4y ago
      That sounds broken. It's supposed to be voluntary. PDF: https://www.eeoc.gov/sites/default/files/migrated_files/federal/2017-approved-Applicant-Form.pdf https://www.eeoc.gov/sites/default/files/migrated_files/fede... >> "Completion of this form is voluntary. No individual personnel selections are made based on this information. There will be no impact on your application if you choose not to answer any of these questions" Your employer shouldn't even be able to know whether or not you filled it out.
      - skrbjc 4y ago
        My experience is the reporting on current employees, which I guess is not voluntary. It's not very clear though: "Self-identification is the preferred method of identifying race/ethnicity information necessary for the EEO-1 Component 1 Report. Employers are required to attempt to allow employees to use self-identification to complete the EEO-1 Component 1 Report. However, if employees decline to self-identify their race/ethnicity, employment records or observer identification may be used. Where records are maintained, it is recommended that they be kept separately from the employee’s basic personnel file or other records available to those responsible for personnel decisions." From: https://eeocdata.org/pdfs/201%20How%20to%20get%20Ready%20to%20File.pdf https://eeocdata.org/pdfs/201%20How%20to%20get%20Ready%20to%...

marian_ivanco 4y ago

I am not sure, but if I remember correctly employer must prove they are not discriminating. And just because they are using AI they are not immune to litigation.

danarmak 4y ago
How can the employer prove a negative? At most I imagine the plaintiff is allowed to do discovery, and then has to prove positive discrimination based on that.
- radu_floricica 4y ago
- vajrabum 4y ago
  If you read the document again (?) maybe you'll see it's not about proving a negative. Instead, it's a standard of due care. Did you check whether using some particular tool illegally discriminates and document that consideration? From the document itself: "Clarifies that, when designing or choosing technological tools, employers must consider how their tools could impact different disabilities; Explains employers’ obligations under the ADA when using algorithmic decision-making tools, including when an employer must provide a reasonable accommodation;"
- HWR_14 4y ago
  If it's a civil case, it's just the preponderance of the evidence. The jury just has to decide who they think is more likely to be correct.
rascul 4y ago
> I am not sure, but if I remember correctly employer must prove they are not discriminating. That seems backwards, at least in the US.

dsr_ 4y ago

The process of gathering evidence after the suit has started is called discovery. There are three major kinds of evidence that would be useful here. Most useful but least likely: email inside the company in which someone says "make sure that this doesn't select too many people with disabilities" or "it's fine that the system isn't selecting people with disabilities, carry on". Useful and very likely: prima facie evidence that the software doesn't make necessary reasonable accomodations - a video captcha without an audio alternative, things like that. Fairly useful and of moderate likelihood: statistical evidence that whatever the company said or did, it has the effect of unfairly rejecting applicants with disabilities.

pfdietz 4y ago
And one could go a step further: run the software itself and show that it discriminates. One doesn't just have to look at past performance of the software; it can be fed inputs tailored to bring out discriminatory performance. In this way software is more dangerous to the defendant than manual hiring practices; you can't do the same thing to an employee making hiring decisions.
- twofornone 4y ago
  How would you make sure that the supplied version has the same weights as the production version? And wouldn't the weights and architecture be refined over time anyway?
  - dogleash 4y ago
    > How would you make sure that the supplied version has the same weights as the production version? You just run the same software (with the same state database, if applicable). Oh wait, I forgot, nobody knows or cares what software they're running. As long as the website is pretty and we can outsource the sysop burden, well then, who needs representative testing or the ability to audit?
  - emiliobumachar 4y ago
    Perjury laws. Once a judge has commanded you to give the same AI, you either give the same AI, or truthfully explain that you can't. Any deviation from that and everyone complicit is risking jail time, not just money. "this is the June 2020 version, this is the current version, we have no back ups in between" is acceptable if true. Destroying or omitting an existing version is not.

jejones3141 4y ago

These days, disparate impact is taken as evidence of discrimination, so it's easy to find "discrimination".

mkr-hn 4y ago
What's the difference? Discrimination is an effect more than an intent. Most people are decent and well-intentioned and don't mean to discriminate, but it still happens. If there's a disparate impact, what do you imagine causes that if not discrimination? Remembering that we all have implicit bias and it doesn't make you a mustache-twirling villain.
- twofornone 4y ago
  >If there's a disparate impact, what do you imagine causes that if not discrimination? 20+ years of environmental differences, especially culture? The disabilities themselves? Genes? Nothing about human nature suggests that all demographics are equally competent in all fields, regardless of whether you group people by race, gender, political preferences, geography, religion, etc. To believe otherwise is fundamentally unscientific, though it's socially unacceptable to acknowledge this truth. >Remembering that we all have implicit bias This doesn't tell you anything about the direction of this bias, but the zeitgeist is such that it is nearly always assumed to go in one direction, and that's deeply problematic. It's an overcorrection that looks an awful lot like institutional discrimination. >Remembering that we all have implicit bias and it doesn't make you a mustache-twirling villain. Except pushing back against unilateral accusations of bias if you belong to one, and only one, specific demographic, you effectively are treated like a mustache-twirling villain. No one is openly complaining about "too much diversity" and keeping their job at the moment. That's bias.
  - etchalon 4y ago
    There is no scientific literature which confirms that any specific demographic quality determine's an individuals capability at any job or task. What does exist is, at best, shows mild correlation over large populations, but nothing binary or deterministic at an individual level. To whit, even if your demographic group, on average, is slightly more or less successful in a specific metric, there is no scientific basis for individualized discrimination. It's "not socially unacceptable to acknowledge this truth", it's socially unacceptable to pretend discrimination is justified.

mkr-hn 4y ago

This is what that demographic survey at the end of job applications is for. It can reveal changes in hiring trends, especially in the demographics of who doesn't get hired. I don't know how well it works in practice.

hallway_monitor 4y ago
I am a person, not a statistic. I always decline to answer these surveys; I encourage others to do the same.
- mkr-hn 4y ago
  Those are for persuading people who do see you as a statistic. You can unilaterally disarm if you like, but they're going to keep discriminating until they see data that proves they're discriminating. Far too few people are persuaded by other means.
- skrbjc 4y ago
  I’m trying to but my employer has said they will use “observer-identified” info to fill it in for me. I find it ridiculous that I can’t object to having someone guess my race and report that to the government.
  - mkr-hn 4y ago
    That sounds broken. It's supposed to be voluntary. PDF: https://www.eeoc.gov/sites/default/files/migrated_files/federal/2017-approved-Applicant-Form.pdf https://www.eeoc.gov/sites/default/files/migrated_files/fede... >> "Completion of this form is voluntary. No individual personnel selections are made based on this information. There will be no impact on your application if you choose not to answer any of these questions" Your employer shouldn't even be able to know whether or not you filled it out.
    - skrbjc 4y ago
      My experience is the reporting on current employees, which I guess is not voluntary. It's not very clear though: "Self-identification is the preferred method of identifying race/ethnicity information necessary for the EEO-1 Component 1 Report. Employers are required to attempt to allow employees to use self-identification to complete the EEO-1 Component 1 Report. However, if employees decline to self-identify their race/ethnicity, employment records or observer identification may be used. Where records are maintained, it is recommended that they be kept separately from the employee’s basic personnel file or other records available to those responsible for personnel decisions." From: https://eeocdata.org/pdfs/201%20How%20to%20get%20Ready%20to%20File.pdf https://eeocdata.org/pdfs/201%20How%20to%20get%20Ready%20to%...