Eliminating bias from Machine Learning is a dangerous idea


This post was originally published by Andre Ye at Towards Data Science

Reframing the discussion

Everyone agrees that ethical biases in machine learning are bad, and data scientists need to do their best in ensuring that their technological creations are not disconnected from human ethics and values. For example, Amazon’s AI tool for hiring was scrapped because it showed a significant bias against female applicants. The real-world discriminatory outcomes of data science’s tinkering have led to a movement advocating for the elimination of bias from machine learning and so-called ‘responsible AI’.

While the heart is in the right place, this mantra — eliminating bias from machine learning — maybe, at the least, flawed, and at the worst, dangerous.

Usually, people have a very one-dimensional view of bias. There is the truth — an ethical place we want the model to be — and a model is biased in some way such that it deviates from the truth; hence the analyst must apply some sort of ‘anti-bias’ to cancel out the model’s bias.

Although this image is a convenient and clean portrayal of the role bias plays in machine learning, it needs to be reframed because of one simple fact: in humans, there is no objectivity; when there is no objectivity, there is no truth. There is only what we want the truth to be.

We use machine learning primarily because humans don’t know what the truth is, and we want it to unveil the truth for us. Humans doctors aren’t great at diagnosing difficult medical conditions, so we ask a machine learning model to find the truth for us. Since humans have differing opinions on what factors play what role in determining a house’s price, machine learning answers for us. No human can see all the sights and live all the experiences, and so we are biased — but machine learning can, through the medium of data, and thus mine insights no single human ever could.

In a sense, machine learning is an attempt to answer the truths that humans cannot agree upon. It is a unifying answer to divisive questions. Machine Learning simply processes the data so we can see what it is saying.

Consider, for example, a simple logistic regression machine learning model trained to predict whether someone will be granted a loan or not. Say that, when analyzing the model’s coefficients, we find that the model gives an extra 5 percent chance of getting the loan if the client is male, as opposed to being female. This violates demographic parity — a fairness metric — which says that in a truthful world, this coefficient would be zero and no advantage would be given based on gender.

In this case, the truth the data scientist seeks is a zero advantage for gender; and hence they must apply some sort of anti-bias repression distortion to the model to get it to arrive at that ethical truth. After this alteration, we’ve made a biased model ‘fair’, because it gives us the output we want it to give us.

Is this really what we want — do we want data analysts to alter the decision-maker at their discretion based on their idea of what morality and ethics is? Everyone agrees in this scenario that there should be no advantage in getting a loan based on gender, but there are countless more complex and less universal ethical decisions we grant data scientists the right to make when we go the first step. Furthermore, do we really want to repress the results of the model such that we never need to confront them?

The model is not at fault for an unethical result. AI cannot be ‘responsible’ or ‘irresponsible’, it can only be good or bad at modelling the data. The model is only the lens through which we can understand data; without the model, data is just a collection of incomprehensible matrices. Altering the model so that you see what you want to see is the same as tainting the lens through which you view the world. Nothing has changed; the model is only lying.

Information will not be honest but filtered through whatever set of values the data scientist holds.

An unethical result is not the fault of the algorithm; it simply exposes something within the data.

The idea of eliminating bias can be dangerous because, in an attempt to make the outcomes look reasonable to us, we introduce our own layer of bias. We start at universal ethical standards and slowly make our way to controversial corners of morality, and end up with a misinformation war.

Indeed, data science and the study of interpreting data has become so politicized and distorted that data scientists need to be extremely careful not to introduce their own external source of bias into a study. When an unethical result is produced, the first question should be “Why is this happening?” instead of “How can we repress it?”

Then, if it is dishonest to alter the model, how can one address unfairness in machine learning? — after all, not touching an algorithm that gives favor to a certain group of people may be more dangerous than distorting it to be more moral. The answer lies in a key realization that data is the problem. We know for a fact that women are just as deserving as men in getting loans, and when we have a complete distribution of data, the lens through which we view data — the machine learning algorithm — will reveal that truth. If the truth ends up not being as moral as humanity would have hoped, we need to investigate it and solve it.

To reframe how we think of biased outcomes: it is the partial covering of a complete distribution of data. A model trained on only part of the complete story will obviously tell a tainted tale. It is the same with humans: the primary root of prejudice is lack of exposure.

Let us consider, then, two methods of addressing this vision of bias. One option (A) is to chip away at the hidden part of the distribution such that a more complete set of data is available, by addressing disabilities in data collection and utilizing the power of randomness. The second (B) is to selectively obscure more of the data such that the remaining data is closer to what the data scientist feels is moral.

Unfortunately, the second option is the mindset most data scientists adopt when dealing with biased outcomes.

Our quest to make the world better with machine learning should be shaped through a drive to uncover more data, not to obscure even more of it. The mantra shouldn’t be ‘eliminate bias from machine learning’, it should be ‘make the data better’.

“Eliminating bias from machine learning” is the rewiring of machine learning algorithms, the lens through which we see the data. “Make the data better” is the pursuit of bringing the data closer to a universal truth. It’s the comparison of two approaches to make machine learning more ethical: one destructive, and the other restorative.

In the case of models discriminating against women, ‘make the data better’ about adding more data concerning women, not hiding information about men to artificially arrive at ethical results and suffering a corresponding decrease in performance. With face recognition failing to perform well on dark-skinned faces, append more training examples so the algorithm can better learn from them. The data and the model performance and work in tandem with ethics, if you let them.

In such a divided world, data holds the truth — only if we choose to expose and elevate it, instead of concealing it.

Spread the word

This post was originally published by Andre Ye at Towards Data Science

Related posts