10 ms·

> Schneier is discussing an unpleasant fact; unbiased algorithms often discover that things we previously attributed to bias were actually unbiased predictors.

> Schneier is discussing an unpleasant fact; unbiased algorithms often discover that things we previously attributed to bias were actually unbiased predictors. That's an interesting interpretation. To me it looks more like he's discussing that bias can (probably will) be baked into algorithms even unintentionally and often subtly. And that highlights a significant difficulty with the "unbiased predictors" thing you say: can you distinguish between an unbiased algorithm discovering that something that looks like bias isn't and bias being subtly baked into the algorithm? I think that's where the "we need to understand what we expect out of the algorithms and ensure the expectations are met" bit comes in, at least in part.

yummyfajitas 11y ago
Definitely - one makes more money than the other. If a bank is biased against some group they are turning away profitable customers. This is also purely a statistical problem; once some quant discovers they can make more money by fixing the bias, they'll do it. Understanding what to expect out of the algorithms is absolutely the wrong way to determine this. The fact is that we simply don't know apriori the optimal way to allocate credit. That's why we need an algorithm in the first place.
- pron 11y ago
  Or, having realized that algorithms making predictions based on past data are politically conservative (pretty much by definition), if we really want to improve the economy, we should take political action to change the situation. This is how it's always been done, and it can work; sometimes rather well.
  - yummyfajitas 11y ago
    If you want to force banks to make suboptimal credit allocation decisions for political reasons, go ahead and openly advocate for it. And be sure to openly accept blame for it if those loans go bad and we have another banking crisis. Just don't try to claim you are attempting to eliminate bias. Eliminating bias is a job for statisticians, not politicians.
    - pron 11y ago
      > If you want to force banks to make suboptimal credit allocation decisions for political reasons, go ahead and openly advocate for it You and I both want to force banks to make suboptimal credit allocations for political reasons (or we both don't). Leaving the local optimum can only be done with coordinated action, but that doesn't mean that your algorithm (that keeps us there) is any less political. By not calling for social action, you are forcing banks to stay in their Nash equilibrium, which is suboptimal for them. What I want is for us to acknowledge that we have (possibly unintentionally) programmed our algorithms to be conservative, and then make the conscious decision whether we want conservative algorithms or progressive algorithms. Sadly, neutral algorithms are impossible[1] if based on their output we take actions that may affect the distribution of power. I'm a progressive and I want a progressive agenda. You're a conservative and want a conservative agenda. Both are perfectly fine, but you must understand that there's nothing neutral in your stance. I believe political action for social change often works and can be very helpful, so I wish to encourage it; you believe the opposite, and therefore wish to discourage it. But neither of us is neutral, and your position is not based on math; your math simply serves your position. Your own equations deliberately omit feedback that we know for a fact (through the study of history) to exist. You make assumptions that reflect your conservative values just as I make assumptions reflecting my liberal values. > Eliminating bias is a job for statisticians, not politicians. You think that when people talk of social bias they mean statistical bias? This may sometimes happen to be (also) true, but in general, when people talk of social bias they mean any human behavior that creates a dynamics for present and future unfair distributions of power. Sadly, statisticians can't eliminate that. Or, rather they can't eliminate that alone, because politics is not (just) what politicians do. Politics[2] is what we all do when we make decisions that affect the distribution of power in society. Interpreting statistical snapshots as predictors of future human behavior is not statistics but politics. Statistics tells us what is (or sometimes how things will likely be if we change nothing); certainly not what we should do. ------ [1]: Well, it's possible to pretend some are neutral if we artificially (and incorrectly) reduce the domain of our value functions. In an interconnected system like human society, it is is disingenuous to direct our action based on its outcome for a very limited sector (like a bank), without saying what we want the action to achieve for all other sectors. So, for example, if you ask me if I want bankers to be happy, then the answer is, of course I do. But that doesn't mean I'll favor any action that makes them happy if it also makes other people less happy. My point is that our value functions must be total; we can't be lazy and assign a value to a narrow look at an outcome, but we must assign a value for a complete outcome (i.e. for all of humanity). [2]: https://en.wikipedia.org/wiki/Politics https://en.wikipedia.org/wiki/Politics : Politics... is the practice and theory of influencing other people... Furthermore, politics is the study or practice of the distribution of power and resources within a given community... as well as the interrelationship(s) between communities.

yummyfajitas 11y ago

Definitely - one makes more money than the other. If a bank is biased against some group they are turning away profitable customers. This is also purely a statistical problem; once some quant discovers they can make more money by fixing the bias, they'll do it. Understanding what to expect out of the algorithms is absolutely the wrong way to determine this. The fact is that we simply don't know apriori the optimal way to allocate credit. That's why we need an algorithm in the first place.

pron 11y ago
Or, having realized that algorithms making predictions based on past data are politically conservative (pretty much by definition), if we really want to improve the economy, we should take political action to change the situation. This is how it's always been done, and it can work; sometimes rather well.
- yummyfajitas 11y ago
  If you want to force banks to make suboptimal credit allocation decisions for political reasons, go ahead and openly advocate for it. And be sure to openly accept blame for it if those loans go bad and we have another banking crisis. Just don't try to claim you are attempting to eliminate bias. Eliminating bias is a job for statisticians, not politicians.
  - pron 11y ago
    > If you want to force banks to make suboptimal credit allocation decisions for political reasons, go ahead and openly advocate for it You and I both want to force banks to make suboptimal credit allocations for political reasons (or we both don't). Leaving the local optimum can only be done with coordinated action, but that doesn't mean that your algorithm (that keeps us there) is any less political. By not calling for social action, you are forcing banks to stay in their Nash equilibrium, which is suboptimal for them. What I want is for us to acknowledge that we have (possibly unintentionally) programmed our algorithms to be conservative, and then make the conscious decision whether we want conservative algorithms or progressive algorithms. Sadly, neutral algorithms are impossible[1] if based on their output we take actions that may affect the distribution of power. I'm a progressive and I want a progressive agenda. You're a conservative and want a conservative agenda. Both are perfectly fine, but you must understand that there's nothing neutral in your stance. I believe political action for social change often works and can be very helpful, so I wish to encourage it; you believe the opposite, and therefore wish to discourage it. But neither of us is neutral, and your position is not based on math; your math simply serves your position. Your own equations deliberately omit feedback that we know for a fact (through the study of history) to exist. You make assumptions that reflect your conservative values just as I make assumptions reflecting my liberal values. > Eliminating bias is a job for statisticians, not politicians. You think that when people talk of social bias they mean statistical bias? This may sometimes happen to be (also) true, but in general, when people talk of social bias they mean any human behavior that creates a dynamics for present and future unfair distributions of power. Sadly, statisticians can't eliminate that. Or, rather they can't eliminate that alone, because politics is not (just) what politicians do. Politics[2] is what we all do when we make decisions that affect the distribution of power in society. Interpreting statistical snapshots as predictors of future human behavior is not statistics but politics. Statistics tells us what is (or sometimes how things will likely be if we change nothing); certainly not what we should do. ------ [1]: Well, it's possible to pretend some are neutral if we artificially (and incorrectly) reduce the domain of our value functions. In an interconnected system like human society, it is is disingenuous to direct our action based on its outcome for a very limited sector (like a bank), without saying what we want the action to achieve for all other sectors. So, for example, if you ask me if I want bankers to be happy, then the answer is, of course I do. But that doesn't mean I'll favor any action that makes them happy if it also makes other people less happy. My point is that our value functions must be total; we can't be lazy and assign a value to a narrow look at an outcome, but we must assign a value for a complete outcome (i.e. for all of humanity). [2]: https://en.wikipedia.org/wiki/Politics https://en.wikipedia.org/wiki/Politics : Politics... is the practice and theory of influencing other people... Furthermore, politics is the study or practice of the distribution of power and resources within a given community... as well as the interrelationship(s) between communities.