PAIR AI Explorables | Is the problem in the data? Examples on Fairness, Diversity, and Bias.
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper deep learning tutorial bias in machine learning ai bias algorithmic bias bias in algorithms garbage in garbage out the problem is in the data the problem is not in the data twitter machine learning machine learning bias machine learning in society ethical ai ai ethics ai ethics bias where does bias come from google ai
In the recurring debate about bias in Machine Learning models, there is a growing argument saying that "the problem is not in the data", often citing the influence of various choices like loss functions or network architecture. In this video, we take a look at PAIR's AI Explorables through the lens of whether or not the bias problem is a data problem.

OUTLINE:
0:00 - Intro & Overview
1:45 - Recap: Bias in ML
4:25 - AI Explorables
5:40 - Measuring Fairness Explorable
11:00 - Hidden Bias Explorable
16:10 - Measuring Diversity Explorable
23:00 - Conclusion & Comments

AI Explorables: https://pair.withgoogle.com/explorables/

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/
BiliBili: https://space.bilibili.com/1824646584

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Hello, everyone. So maybe you've seen my last video about this topic. But every few months, the debate about bias in machine learning models is resurfacing. And this time, a tweet by cream car is sort of in the middle of it. And he says four things to know about race and gender bias in algorithms. First, the bias starts in the data. Second, the algorithms don't create the bias, but they do transmit it. Third, there are a huge number of other biases, race and gender bias are just the most obvious. And fourth, it's fixable. And what followed was what I thought was a pretty sensible tweet or thread about bias in machine learning and in statistics in general and what to do about it, namely the plea for understanding your data better and other suggestions. Now, there's a follow up tweet to this that is here saying, oh, this thread is doing numbers. There are a few comments disagreeing with this thread. One thing to keep in mind as you read them, as far as I can tell, they are misinterpreting what I said, because they are using a different definition of bias. And I think this really hits the nail on the head. Specifically, he got a lot of heat for saying the first thing here, the bias starts in the data. Now, every time you talk about these things, there are a number of people coming out saying, it's not the data, the problem is not the data, or the problem is not only the data. And I have to admit, I also had a little bit of a wrong impression of what that actually means. And I think the solution is in recognizing that people are using different definition of bias. And that leads to a situation where people talking past each other. So in my last video, I've pointed out, there are many different things that can go wrong with a machine learning pipeline and where bias can be introduced, and I raised the plea to not confuse them. Because what people will do is they will point to one problem, and then suggest a solution that is relevant for a different problem. Now, as far as I understand it, when Kareem talks about the bias starts in the data and is transmitted by models, what he means is statistical bias, which means that either the data set is sampled in a wrong way and doesn't represent the world as it is, which I also discussed, or that the model itself, the choices we make during training and the loss function of the choice of architecture, lead to a situation where the model output does not represent the world. This refers to statistical bias and statistical bias is in part necessary for us to build models that do generalize well. But it can be a problem. And I think everyone acknowledges that. But when people say the problem is not in the data, I think they usually mix up two different things. The first thing they mix is what I'm showing right here. There are problems with building the models itself that can amplify a bias in the data or if they are really bad models even create bias that was not present in the data set. On the other hand, I also pointed out that a lot of people actually have a problem not with the data itself, but with reality. So the bias they're talking about is bias that already exists in the world. And here the machine learning model is sort of viewed as a tool of social engineering. And very often evidence for wrong loss functions are brought up to show that there is bias that is not in the data, but then the fixes that are suggested for it are targeted towards bias that is in reality. So my plea last time was, let's not confuse the different things that go wrong and how we fix them is perfectly viable to talk about changing reality to talk about using a machine learning model to influence reality. We all know there are feedback loops and other influences that these AI systems have. And I think we should then honestly come out and say, when we talk about de-biasing, what we actually mean is we want to bias the machine learning model such that it outputs a world that we want to have and not the world that we actually have as a tool for social engineering. So today we're going to have a look at a thing that I wanted to have a look for for a while. And those are these AI explorables. They're made by Google, and they're kind of cool interactive things that give you a visual impression of what can go wrong with machine learning models. Right now they have these in the fields of privacy, and also fairness and bias. So I thought today we'd look at the ones in the fairness and bias section with special regard to people saying the problem is not in the data. Now if you actually look at who's making these arguments and who's making these explainables, there is a pretty big overlap between who is making the explainables and who is saying the problem is not in the data. So if there is good evidence for the fact that the problem is not in the data, I expect that these explainables will give us a bit of a hint about that. So my hypothesis as I go through this is going to be yes, the problem is in the data either because the data is sampled incorrectly, in which case we can simply focus on sampling a better data set. Or in the other case, because reality is not as we want it, and that is reflected in the data, in which case we're not de-biasing, we are actively biasing. But I guess you can see for yourself. So the first explorable deals with measuring fairness. And essentially, it's saying that imagine there is a disease, and if you had a perfect test for the disease, you would have no problem. So all the people in red here are sick, whereas all the people in gray are well. And the perfect test would be able to recognize all the sick people and not recognize all the well people. 100% accuracy, not a problem. This is not the case in reality, though. Usually we have tests that aren't exactly perfect. So you'll always end up with people who are sick, but not recognize the ones down here. And people who are not sick, but the test says they are sick. I'm sorry, it's really hard. I have to draw off screen and hit the region that I'm targeting. It's an experiment. Now these tests usually don't just say you're sick or you're not sick, they usually give you a probability of being sick. Now the question is, where do you cut off? Do you say a person is sick when the test is 99% sure? Do you say a person is sick when the test is 50% sure? And here is where you have to make a choice. One choice is to never miss the disease, which means that as soon as my test says this person might be sick, I already put them into the sick category, I won't ever miss anyone, or I'll just miss really few people down here. But you can see I have a large swath of people that aren't sick, but the test says they're sick just because I'm so conservative. On the other hand, I could say, I just want to be really sure. So I only classify anyone as sick if the test is really sure you can see now that very few people that aren't sick end up in the positive group. However, you have a lot of people who are sick who are not detected because you simply don't trust the test unless it's really, really sure. The aggressiveness gives you a handle on the threshold here. So full aggressiveness means that as soon as the test says there's there might be something wrong, you classify a person as sick. On the other hand of the spectrum, you just want to be really, really sure. And you can see while you miss half the sick people, you don't make any errors on healthy people. So how does this play into fairness? The fairness aspect comes in when we consider different subgroups. They say things get even more complicated when we check if the model treats different groups fairly, whatever we decide in terms of trade offs between these metrics, we probably like them to be roughly even across different groups of people. If we're trying to evenly allocate resources, having the model miss more cases in children than adults would be bad. So on the right, you can see that now we split the population into children and adults. And you can see some things going on here, namely in this fictitious world, the base rates are different. This is known as the base rate problem. And you can see that the disease seems to be more prevalent in children just from the fact that they are children. And this results in kind of a weird situation with what we had before. See, wherever you set the threshold, you're going to have a different proportion of adults and children that you misdiagnose in one way or another. So on the bottom here, you see the recall, which is right now equal for children and adults. But due to the different base rates, the children have a much higher precision than the adults. So if for example, there was some kind of worldwide pandemic, and you're an adult, you might rightfully claim that this is unfair, because just by how the threshold is set, you go to quarantine much more easily than a child, even if you are healthy. So you might plead for raising up the threshold. But again, that would not be fair to the children. And even if you allow for having different thresholds for the different groups, due to the different base rates, you'll never be able to bring both the precision and the recall to be equal for the different groups. Now I've looked at all of the different numbers. And you can see right here, I've plotted precision versus recall. For adults, it looks about like this. And for children, it looks about like this. So you can see as these curves are never intersecting, you'll never manage to find any threshold for either group that where both precision and recall match. And their conclusion to this article is somehow you cannot satisfy every single notion of fairness at the same time, which of course I agree with. But you can clearly see that the reason this whole phenomenon happens is because you have the different base rates, which draw these two curves away from one another. But let's examine our hypothesis again, is the problem here in the data? And I would argue, yes, absolutely. The problem is in reality. And reality makes it such that children are more often sick. So reality is at the cause for this problem. And this reality gets into the data. So very directly, at least in this particular problem, the problem is in the data. The next explainable is called hidden bias. And the situation is, let's pretend we're college admission officers trying to predict the GPA students will have in college. This is not real data. This is simulated data. So here we take a simple machine learning model and let it predict the college GPAs. So on the x axis, you see what we're trying to predict. And on the y axis is our model trying to predict it. So the further away we are from the middle line, the worse we're doing. And you can see here if our only input variable, and that's what it says at the top is the high school GPA, we're doing pretty badly, we can increase that performance by providing the model with more data, you can see that the points shifted towards the line, meaning we make less mistakes. Now they introduce the problem. They say if a sexist college culture has historically led to lower grades for female students is here in purple, the model will pick up on that correlation and predict lower grades for women training on historical data bakes in historical biases. And they also say here the sexist culture has improved, but the model learned from the past correlation still predicts higher grades for men. So essentially saying in the past, women were subject to sexism and therefore had lower grades. However, this is no longer the case. And now the model trained on the old data still makes that mistake. Notice that this falls pretty clearly into the skewed sampling and out of date data category. So right off the bat, the problem is in the data. So the first thing they point out here is that if we simply don't give the model access to the variable gender, the problem might still persist, because the model will simply find correlations between gender and then use that to predict. And honestly, how could the model do any different in the world that it sees and the data that it has, the purple dots are actually performing poorer. So the most accurate thing to do is to score them lower. Again, the problem here is clearly in the data and we need to get more accurate data that better reflects the real world as it is, we all agree that if we don't have the correct data, our model is going to learn all the mistakes that are in the data set. So conclusion one from this explainable is that just because you take out a protected attribute from the model, it doesn't mean that you can fix bias because the model can simply find other variables that are correlated, which is absolutely true. The next thing they're saying is that as intuitive as it might seem to exclude the protected attribute from the algorithm, it might even be beneficial to explicitly include a protected attribute. So here they have a different machine learning model. This time, they still want to predict the college GPA. However, their only input variable is the score that one alumni interviewer gives to a student. Now it just so happens that this student has a personal bias against people from low income households here in red. So here they say in our toy model, students grades don't depend on their income once they're in college. In other words, we have biased inputs and unbiased outcomes, the opposite of the previous example where the inputs weren't biased, but the toxic culture bias the outcomes. So we've completely switched frames right now, we're basically relying on this one person to interview all the people. And it is the case that you know, when this person says, yes, the GPA is probably going to be good, and vice versa. So we still have this linear relationship. However, that person has a personal bias. So necessarily, this is going to influence our decisions in a bad way. And here they argue that if we explicitly include the income, the model can compensate for this. So the model can recognize that if there is a person from a low income household, it probably shouldn't trust that assessment of the interviewer as much. So conclusion one was that if you have biased target variables, like you have this out of date data, then even excluding the protected attribute might not be enough to fix the bias. Conclusion two from this experiment, however, says that if you have accurate targets, like here we have actual data from how well people performed, then giving the model access to all the data may help. So it's not as easy as simply telling the model, don't look at this one particular variable. But again, let's look at it from the perspective of is the bias in the data. And clearly here in the second example, the problem was only there when we only relied on that biased interviewer. So again, the bias was in the data. And as soon as we acquired better data, more variables, we fix the problem, either because the data was sampled incorrectly, or because reality itself simply isn't as we want it. The third explainable is called measuring diversity. This is the most strongly worded one of the three. And I think it makes it the most explicit, which is something that I'm thankful for. So they say search ranking and recommendation systems can help find useful documents in large data sets. However, these data sets reflect the biases of the society in which they were created. And the systems risk re entrenching those biases. For example, if someone is not a white man searches for CEO pictures and sees a page of white men, they may feel that only white men can be CEOs. So the argument is one that I also made in my video, and it is that if we implement these systems, they will have an effect on society and that effect might be not what we want. But it is important to remember that this is an entirely different problem from skewed data sets or different loss functions. And when you click on the link that they cite, you get to this article, the top jobs where women are outnumbered by men named John, and it is an astounding display of the disparities that are present in some jobs. Now, while it is a valid question to ask why that is, and what might be at the cause of these problems, it's pretty clear that this is the state of the world. And any machine learning model outputting this as a search result reflects the world accurately. And the problems with these models aren't really that they don't reflect the world as is. But what the people are criticizing is that the output is not what they would like it to be. And they have their reasons there are valid feedback loops. And the reason they give here is that they may feel that only white men can be CEOs. My problems with these types of arguments is that search engines quickly cease to be search engines and are much more like wish engines. Like why use a search engine when I already know what I want to come out. But I do appreciate the honesty. So now we are truly in the field of social engineering, we're in the field of making the outputs of these models as we want. So here they have a toy data set, you can see there are squares and these squares, they come in three different colors, they come in two different sizes, and some of them have a circle and some of them don't. So here the first task is to select green boxes such that the representation of green boxes is 30%. Now given that there are three green boxes, you can just select the three green boxes and make sure that you select 10 boxes in total and you'll meet that notice that that has nothing to do with a search engine. Now, this is simply we have a target of green boxes and we're trying to meet that target. We can of course do the same thing with the number of dots and the sizes and it gets interesting once we have different intersecting targets. So we want 30% of our subset to be green 35% to have a dot and 60% to be small. And while you can almost solve this problem, the point they're making right here is that now it suddenly becomes important what difference metric you choose. If you choose the mean difference metric between your targets and the actual group you're choosing, the result will be different from when you choose for example, the absolute difference. And you can see this right here. So here they give you the best choices according to targets that you set on the left and they show you where they rank in terms of the different metrics. So the sequence that is best in terms of mean difference is only second best in terms of max difference. And as you change around the sliders, you can see that this changes and you can see how their rankings here become pretty wild. So they go into this question of which measure is best in a vacuum, they say all of these ranking methods are defensible. Picking one requires knowledge of the data set and broader societal context. For example, the doctors on the left have more variance along the shirt color attribute, but they're less diverse by gender than the doctors on the right. With the shirt color and gender targets we've picked, the two subsets have the same mean and max differences. However, in most applications, it's more important to have a representative sample of socially relevant characteristics like gender rather than something less salient like color. So the point is that if they pick the subset on the left, it might be quite diverse with respect to white or blue colored shirts, but it might not be as diverse with respect to gender. However, on the right side, you can see that everyone's wearing a white shirt. However, genders are more equally represented. So I don't really get the jump here we went from the metric you choose makes a difference in how the subgroups are represented to which attribute we choose makes the different attributes differently represented. And all of that has not really a lot to do with search engines per se, because I still don't get why I wouldn't want my search engine to just represent the world as it is. But pretty clearly you can see that if you are not satisfied with the representation of a particular shirt color of a particular gender, or other protected attributes, what you're essentially saying is that reality isn't as you want it, that reality comes into the data set. And then the data set is not as you want it. So the problem is in the data. And they go one step further and say that it's actually not as easy as simply including something like gender. So here you have stock photos for construction workers that seem to be very balanced on gender. But if you look at the feminine presenting individuals and other gender representations, they're depicted as historic nostalgia, toys, clip art, or passive. And I mean, these are certainly valid problems. But this is not truly a wish machine and not a search machine anymore. I think maybe a more accurate solution to this problem would just be to tell people that just because a search engine outputs a bunch of results that is not that per scriptive description of the world, it is rather a descriptive representation of the training data, which might or might not reflect the world as it is. I think people are in general a bit more competent than simply seeing a bunch of images on a website and thinking, oh, I'm going to now make my life decisions in accordance with what I saw here when I typed a construction worker into Google. So that was it on the pair AI explorables on the topics of fairness. And every single time, we saw that the problem is clearly in the data itself, or in the reality that then influences the data again, which is fine. But I think when we talk about these things, we should be clear about what kind of bias we mean, and then suggest solutions that are specifically for that kind of bias. Alright, that was it for me. I'll see you next time. Bye bye.
 Hello, everyone. So maybe you've seen my last video about this topic. But every few months,
 the debate about bias in machine learning models is resurfacing. And this time, a tweet by cream car
 is sort of in the middle of it. And he says four things to know about race and gender bias in
 algorithms. First, the bias starts in the data. Second, the algorithms don't create the bias,
 but they do transmit it. Third, there are a huge number of other biases, race and gender bias are
 just the most obvious. And fourth, it's fixable. And what followed was what I thought was a pretty
 sensible tweet or thread about bias in machine learning and in statistics in general and what
 to do about it, namely the plea for understanding your data better and other suggestions. Now,
 there's a follow up tweet to this that is here saying, oh, this thread is doing numbers. There
 are a few comments disagreeing with this thread. One thing to keep in mind as you read them, as far
 as I can tell, they are misinterpreting what I said, because they are using a different definition
 of bias. And I think this really hits the nail on the head. Specifically, he got a lot of heat
 for saying the first thing here, the bias starts in the data. Now, every time you talk about these
 things, there are a number of people coming out saying, it's not the data, the problem is not the
 data, or the problem is not only the data. And I have to admit, I also had a little bit of a wrong
 impression of what that actually means. And I think the solution is in recognizing that people
 are using different definition of bias. And that leads to a situation where people talking past
 each other. So in my last video, I've pointed out, there are many different things that can go wrong
 with a machine learning pipeline and where bias can be introduced, and I raised the plea to not
 confuse them. Because what people will do is they will point to one problem, and then suggest a
 solution that is relevant for a different problem. Now, as far as I understand it, when Kareem talks
 about the bias starts in the data and is transmitted by models, what he means is statistical bias,
 which means that either the data set is sampled in a wrong way and doesn't represent the world
 as it is, which I also discussed, or that the model itself, the choices we make during training
 and the loss function of the choice of architecture, lead to a situation where the model output does not
 represent the world. This refers to statistical bias and statistical bias is in part necessary for
 us to build models that do generalize well. But it can be a problem. And I think everyone acknowledges
 that. But when people say the problem is not in the data, I think they usually mix up two different
 things. The first thing they mix is what I'm showing right here. There are problems with building
 the models itself that can amplify a bias in the data or if they are really bad models even create
 bias that was not present in the data set. On the other hand, I also pointed out that a lot of people
 actually have a problem not with the data itself, but with reality. So the bias they're talking about
 is bias that already exists in the world. And here the machine learning model is sort of
 viewed as a tool of social engineering. And very often evidence for wrong loss functions are brought
 up to show that there is bias that is not in the data, but then the fixes that are suggested for it
 are targeted towards bias that is in reality. So my plea last time was, let's not confuse the
 different things that go wrong and how we fix them is perfectly viable to talk about changing
 reality to talk about using a machine learning model to influence reality. We all know there are
 feedback loops and other influences that these AI systems have. And I think we should then honestly
 come out and say, when we talk about de-biasing, what we actually mean is we want to bias the
 machine learning model such that it outputs a world that we want to have and not the world that we
 actually have as a tool for social engineering. So today we're going to have a look at a thing that
 I wanted to have a look for for a while. And those are these AI explorables. They're made by Google,
 and they're kind of cool interactive things that give you a visual impression of what can go wrong
 with machine learning models. Right now they have these in the fields of privacy, and also fairness
 and bias. So I thought today we'd look at the ones in the fairness and bias section with special
 regard to people saying the problem is not in the data. Now if you actually look at who's making
 these arguments and who's making these explainables, there is a pretty big overlap between who is making
 the explainables and who is saying the problem is not in the data. So if there is good evidence for
 the fact that the problem is not in the data, I expect that these explainables will give us a bit
 of a hint about that. So my hypothesis as I go through this is going to be yes, the problem is
 in the data either because the data is sampled incorrectly, in which case we can simply focus
 on sampling a better data set. Or in the other case, because reality is not as we want it,
 and that is reflected in the data, in which case we're not de-biasing, we are actively biasing.
 But I guess you can see for yourself. So the first explorable deals with measuring fairness.
 And essentially, it's saying that imagine there is a disease, and if you had a perfect test for the
 disease, you would have no problem. So all the people in red here are sick, whereas all the people
 in gray are well. And the perfect test would be able to recognize all the sick people and not
 recognize all the well people. 100% accuracy, not a problem. This is not the case in reality, though.
 Usually we have tests that aren't exactly perfect. So you'll always end up with people who are sick,
 but not recognize the ones down here. And people who are not sick, but the test says they are sick.
 I'm sorry, it's really hard. I have to draw off screen and hit the region that I'm targeting.
 It's an experiment. Now these tests usually don't just say you're sick or you're not sick,
 they usually give you a probability of being sick. Now the question is, where do you cut off?
 Do you say a person is sick when the test is 99% sure? Do you say a person is sick when the test
 is 50% sure? And here is where you have to make a choice. One choice is to never miss the disease,
 which means that as soon as my test says this person might be sick, I already put them into
 the sick category, I won't ever miss anyone, or I'll just miss really few people down here. But you
 can see I have a large swath of people that aren't sick, but the test says they're sick just because
 I'm so conservative. On the other hand, I could say, I just want to be really sure. So I only
 classify anyone as sick if the test is really sure you can see now that very few people that aren't
 sick end up in the positive group. However, you have a lot of people who are sick who are not
 detected because you simply don't trust the test unless it's really, really sure.
 The aggressiveness gives you a handle on the threshold here. So full aggressiveness means
 that as soon as the test says there's there might be something wrong, you classify a person as sick.
 On the other hand of the spectrum, you just want to be really, really sure. And you can see while
 you miss half the sick people, you don't make any errors on healthy people. So how does this play
 into fairness? The fairness aspect comes in when we consider different subgroups. They say things
 get even more complicated when we check if the model treats different groups fairly, whatever
 we decide in terms of trade offs between these metrics, we probably like them to be roughly even
 across different groups of people. If we're trying to evenly allocate resources, having the model
 miss more cases in children than adults would be bad. So on the right, you can see that now we split
 the population into children and adults. And you can see some things going on here, namely in this
 fictitious world, the base rates are different. This is known as the base rate problem. And you
 can see that the disease seems to be more prevalent in children just from the fact that they are
 children. And this results in kind of a weird situation with what we had before. See, wherever
 you set the threshold, you're going to have a different proportion of adults and children
 that you misdiagnose in one way or another. So on the bottom here, you see the recall,
 which is right now equal for children and adults. But due to the different base rates,
 the children have a much higher precision than the adults. So if for example, there was some kind of
 worldwide pandemic, and you're an adult, you might rightfully claim that this is unfair,
 because just by how the threshold is set, you go to quarantine much more easily than a child,
 even if you are healthy. So you might plead for raising up the threshold. But again, that would
 not be fair to the children. And even if you allow for having different thresholds for the
 different groups, due to the different base rates, you'll never be able to bring both the precision
 and the recall to be equal for the different groups. Now I've looked at all of the different
 numbers. And you can see right here, I've plotted precision versus recall. For adults, it looks
 about like this. And for children, it looks about like this. So you can see as these curves are
 never intersecting, you'll never manage to find any threshold for either group that where both
 precision and recall match. And their conclusion to this article is somehow you cannot satisfy
 every single notion of fairness at the same time, which of course I agree with. But you can clearly
 see that the reason this whole phenomenon happens is because you have the different base rates,
 which draw these two curves away from one another. But let's examine our hypothesis again,
 is the problem here in the data? And I would argue, yes, absolutely. The problem is in reality.
 And reality makes it such that children are more often sick. So reality is at the cause for this
 problem. And this reality gets into the data. So very directly, at least in this particular problem,
 the problem is in the data. The next explainable is called hidden bias. And the situation is,
 let's pretend we're college admission officers trying to predict the GPA students will have
 in college. This is not real data. This is simulated data. So here we take a simple machine
 learning model and let it predict the college GPAs. So on the x axis, you see what we're trying to
 predict. And on the y axis is our model trying to predict it. So the further away we are from the
 middle line, the worse we're doing. And you can see here if our only input variable, and that's what
 it says at the top is the high school GPA, we're doing pretty badly, we can increase that performance
 by providing the model with more data, you can see that the points shifted towards the line,
 meaning we make less mistakes. Now they introduce the problem. They say if a sexist college culture
 has historically led to lower grades for female students is here in purple, the model will pick
 up on that correlation and predict lower grades for women training on historical data bakes in
 historical biases. And they also say here the sexist culture has improved, but the model learned
 from the past correlation still predicts higher grades for men. So essentially saying in the past,
 women were subject to sexism and therefore had lower grades. However, this is no longer the case.
 And now the model trained on the old data still makes that mistake. Notice that this falls pretty
 clearly into the skewed sampling and out of date data category. So right off the bat, the problem
 is in the data. So the first thing they point out here is that if we simply don't give the model
 access to the variable gender, the problem might still persist, because the model will simply find
 correlations between gender and then use that to predict. And honestly, how could the model do any
 different in the world that it sees and the data that it has, the purple dots are actually performing
 poorer. So the most accurate thing to do is to score them lower. Again, the problem here is clearly
 in the data and we need to get more accurate data that better reflects the real world as it is,
 we all agree that if we don't have the correct data, our model is going to learn all the mistakes
 that are in the data set. So conclusion one from this explainable is that just because you take out
 a protected attribute from the model, it doesn't mean that you can fix bias because the model can
 simply find other variables that are correlated, which is absolutely true. The next thing they're
 saying is that as intuitive as it might seem to exclude the protected attribute from the algorithm,
 it might even be beneficial to explicitly include a protected attribute. So here they have a
 different machine learning model. This time, they still want to predict the college GPA. However,
 their only input variable is the score that one alumni interviewer gives to a student. Now it just
 so happens that this student has a personal bias against people from low income households here in
 red. So here they say in our toy model, students grades don't depend on their income once they're
 in college. In other words, we have biased inputs and unbiased outcomes, the opposite of the previous
 example where the inputs weren't biased, but the toxic culture bias the outcomes. So we've completely
 switched frames right now, we're basically relying on this one person to interview all the people.
 And it is the case that you know, when this person says, yes, the GPA is probably going to be good,
 and vice versa. So we still have this linear relationship. However, that person has a personal
 bias. So necessarily, this is going to influence our decisions in a bad way. And here they argue
 that if we explicitly include the income, the model can compensate for this. So the model can
 recognize that if there is a person from a low income household, it probably shouldn't trust that
 assessment of the interviewer as much. So conclusion one was that if you have biased target variables,
 like you have this out of date data, then even excluding the protected attribute might not be
 enough to fix the bias. Conclusion two from this experiment, however, says that if you have
 accurate targets, like here we have actual data from how well people performed, then giving the
 model access to all the data may help. So it's not as easy as simply telling the model, don't look at
 this one particular variable. But again, let's look at it from the perspective of is the bias
 in the data. And clearly here in the second example, the problem was only there when we only relied on
 that biased interviewer. So again, the bias was in the data. And as soon as we acquired better data,
 more variables, we fix the problem, either because the data was sampled incorrectly, or because
 reality itself simply isn't as we want it. The third explainable is called measuring diversity.
 This is the most strongly worded one of the three. And I think it makes it the most explicit,
 which is something that I'm thankful for. So they say search ranking and recommendation systems can
 help find useful documents in large data sets. However, these data sets reflect the biases of
 the society in which they were created. And the systems risk re entrenching those biases.
 For example, if someone is not a white man searches for CEO pictures and sees a page of white men,
 they may feel that only white men can be CEOs. So the argument is one that I also made in my video,
 and it is that if we implement these systems, they will have an effect on society and that effect
 might be not what we want. But it is important to remember that this is an entirely different
 problem from skewed data sets or different loss functions. And when you click on the link that
 they cite, you get to this article, the top jobs where women are outnumbered by men named John,
 and it is an astounding display of the disparities that are present in some jobs. Now, while it is a
 valid question to ask why that is, and what might be at the cause of these problems, it's pretty
 clear that this is the state of the world. And any machine learning model outputting this as a search
 result reflects the world accurately. And the problems with these models aren't really that they
 don't reflect the world as is. But what the people are criticizing is that the output is not what they
 would like it to be. And they have their reasons there are valid feedback loops. And the reason
 they give here is that they may feel that only white men can be CEOs. My problems with these types
 of arguments is that search engines quickly cease to be search engines and are much more like wish
 engines. Like why use a search engine when I already know what I want to come out. But I do
 appreciate the honesty. So now we are truly in the field of social engineering, we're in the field of
 making the outputs of these models as we want. So here they have a toy data set, you can see there
 are squares and these squares, they come in three different colors, they come in two different sizes,
 and some of them have a circle and some of them don't. So here the first task is to select green
 boxes such that the representation of green boxes is 30%. Now given that there are three green boxes,
 you can just select the three green boxes and make sure that you select 10 boxes in total and you'll
 meet that notice that that has nothing to do with a search engine. Now, this is simply we have a
 target of green boxes and we're trying to meet that target. We can of course do the same thing
 with the number of dots and the sizes and it gets interesting once we have different intersecting
 targets. So we want 30% of our subset to be green 35% to have a dot and 60% to be small. And while
 you can almost solve this problem, the point they're making right here is that now it suddenly
 becomes important what difference metric you choose. If you choose the mean difference metric
 between your targets and the actual group you're choosing, the result will be different from when
 you choose for example, the absolute difference. And you can see this right here. So here they give
 you the best choices according to targets that you set on the left and they show you where they rank
 in terms of the different metrics. So the sequence that is best in terms of mean difference is only
 second best in terms of max difference. And as you change around the sliders, you can see that this
 changes and you can see how their rankings here become pretty wild. So they go into this question
 of which measure is best in a vacuum, they say all of these ranking methods are defensible.
 Picking one requires knowledge of the data set and broader societal context. For example, the doctors
 on the left have more variance along the shirt color attribute, but they're less diverse by gender
 than the doctors on the right. With the shirt color and gender targets we've picked, the two subsets
 have the same mean and max differences. However, in most applications, it's more important to have
 a representative sample of socially relevant characteristics like gender rather than something
 less salient like color. So the point is that if they pick the subset on the left, it might be
 quite diverse with respect to white or blue colored shirts, but it might not be as diverse
 with respect to gender. However, on the right side, you can see that everyone's wearing a white shirt.
 However, genders are more equally represented. So I don't really get the jump here we went from
 the metric you choose makes a difference in how the subgroups are represented to which attribute
 we choose makes the different attributes differently represented. And all of that has
 not really a lot to do with search engines per se, because I still don't get why I wouldn't want
 my search engine to just represent the world as it is. But pretty clearly you can see that if you
 are not satisfied with the representation of a particular shirt color of a particular gender,
 or other protected attributes, what you're essentially saying is that reality isn't as you
 want it, that reality comes into the data set. And then the data set is not as you want it. So
 the problem is in the data. And they go one step further and say that it's actually not as easy as
 simply including something like gender. So here you have stock photos for construction workers
 that seem to be very balanced on gender. But if you look at the feminine presenting individuals
 and other gender representations, they're depicted as historic nostalgia, toys, clip art, or passive.
 And I mean, these are certainly valid problems. But this is not truly a wish machine and not a
 search machine anymore. I think maybe a more accurate solution to this problem would just be
 to tell people that just because a search engine outputs a bunch of results that is not that
 per scriptive description of the world, it is rather a descriptive representation of the training
 data, which might or might not reflect the world as it is. I think people are in general a bit more
 competent than simply seeing a bunch of images on a website and thinking, oh, I'm going to now make
 my life decisions in accordance with what I saw here when I typed a construction worker into Google.
 So that was it on the pair AI explorables on the topics of fairness. And every single time,
 we saw that the problem is clearly in the data itself, or in the reality that then influences
 the data again, which is fine. But I think when we talk about these things, we should be clear
 about what kind of bias we mean, and then suggest solutions that are specifically for that kind of
 bias. Alright, that was it for me. I'll see you next time. Bye bye.