Is Google Translate Sexist? Gender Stereotypes in Statistical Machine Translation
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper google translate gender stereotype machine learning biased debiasing debiasing machine learning algorithmic fairness machine learning social justice machine learning bias deep learning bias deep learning gender what is deep learning deep learning tutorial introduction to deep learning hungarian translate translate gender stereotype
#genderbias #algorithmicfairness #debiasing

A brief look into gender stereotypes in Google Translate. The origin is a Tweet containing a Hungarian text. Hungarian is a gender-neutral language, so translating gender pronouns is ambiguous. Turns out that Google Translate assigns very stereotypical pronouns. In this video, we'll have a look at the origins and possible solutions to this problem.

OUTLINE:
0:00 - Intro
1:10 - Digging Deeper
2:30 - How does Machine Translation work?
3:50 - Training Data Problems
4:40 - Learning Algorithm Problems
5:45 - Argmax Output Problems
6:45 - Pragmatics
7:50 - More on Google Translate
9:40 - Social Engineering
11:15 - Conclusion

Songs:
Like That - Anno Domini Beats
Submarine - Dyalla
Dude - Patrick Patrikios

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/
BiliBili: https://space.bilibili.com/1824646584

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 So you might have seen this tweet. Hungarian is a gender neutral language. It has no gender pronouns so Google Translate automatically chooses the gender for you. Here is how everyday sexism is consistently encoded in 2021. F you Google. On the left hand side is a Hungarian sentence. Google Translate then translates this to the following text saying she is beautiful. He is clever. He reads, she washes the dishes, he builds, she sues, he teaches, she cooks. So Google Translate chooses the gender pronoun and it appears to choose gender pronouns that are very consistent with common gender stereotypes. So this has generated a lot of outrage and the topic is coming up again and again. And I thought we just dig a little bit into the background of why this happens and what we might do about it. So the first thing you might notice is the text here is really a bouquet of stereotypes and also ends with go to hell Google. So no doubt this person has tried a bunch of things. So I've kind of reproduced the first four sentences of the input. And here it is, she is beautiful. He is clever. He reads, she washes the dishes. Now to detect whether or not this is a feature of the language, maybe there are subtle gender hints. Here is a thing you can do. You can translate it back into the other direction. She is beautiful. He is clever, which will give you the Hungarian sentence. And then we can simply change the pronouns right here. He is beautiful. She is clever. If there are subtle language hints, you would expect that if you translate this to Hungarian and back that the same sentence returns. However, if this is a truly gender neutral language, then you would not expect this to matter. So if we now translate this to Hungarian and then we take this Hungarian sentence and translate it back, oh, see, it has actually switched around the pronouns back to she is beautiful. He is clever. So no doubt Google Translate here is inferring the pronoun from the words that follow assigning beautiful to a more feminine pronoun, assigning clever to more masculine pronoun. These are gender stereotypes, and we're going to dig a little bit into why this happens. For that, we have to understand how the machine learning systems currently work. Machine learning systems are statistical systems that try to translate a piece of text into a piece of text of a different language. So here we enter the piece of text in one language, it goes into this big ML box, and outcomes actually not a single sentence, but outcomes usually a plethora of possible sentences, along with probabilities assigned to each of those outputs, the system then chooses the most likely output and displays that to the user already said this is a statistical system, it is derived from a set of training data. So it's important to understand that all the system does is tell us that the sentence she is beautiful is the most likely sentence to appear in a document that is translated from Hungarian where this original sentence was present, given the training data, the training data itself is, of course, derived from the world in some way, if you believe that such a thing as reality exists. And there we have the whole system. Now we might ask ourselves, what do we do about it? How should we fix this? And the answer, unfortunately, is it depends. It depends on where you think the problem lies. So the first point where there could be a problem is the way we derive the training data from the world or from reality itself. Common issues here are that the sampling of data is somehow skewed, it is out of date, we're working with old data. In general, the data that we have does not reflect the world. And if the data that we have is skewed in some way, we can only expect that our machine learning system picks up on that skew. So a person arguing this would say that it is actually not that likely that these sent Hungarian sentence here translates to she is beautiful. And it might be equal, you're more likely that it translates to something else. If we only had all the translation data that we could hope of the second point where we could introduce problems is when we derive the ML system from the training data. Here's the thing, every machine learning system introduces statistical biases in order for it to generalize properly. Otherwise, we could not do learning. And it's entirely possible that some of these things such as the regularizer and the loss function, or the particular choice of architecture would introduce statistical bias into the system. This would result in a model that does not reflect the data as we have it. So someone arguing for this would argue that even though we have good training data in the training data, there is no problem. The ML system derived from the training data introduces unwanted effects. So someone might argue even though the feminine version here is slightly bigger in the training data than the masculine version, through the process of learning and distilling the ML model simply abstracts this and makes it a lot more likely therefore skewing the gender balance unfairly. The last problem is the fact that we simply choose the top prediction and output that to the user. This is not really accurate. If we simply output whatever is most likely, this is an unfair representation. In fact, what we should do is we should give the user all the possibilities with all the probabilities associated. Someone arguing for this might say that the training data is fine. The ML model even makes good outputs, the probability distributions are correct and reflect the world. However, because we only pick the top one, the user is tricked into thinking that that is the only possibility or maybe just that this possibility is much more likely than the alternatives. As good as that sounds to output always the probabilities associated with different ambiguous translations. The short answer of why we don't do this is pragmatics. I'll give you an example. This is Billy Billy. It's a Chinese video sharing websites and for people who cannot access YouTube from China, I do upload my videos to Billy Billy so they can watch them. However, while I'm practicing Mandarin, I'm not good enough yet to navigate a site that is full of characters that I have even a difficult time parsing. And this is what Google Translate is usually used as I just want to navigate effectively to the point where I can upload a video define its categories, leave a description, and then send that off. If Google Translate were to give me every possible ambiguity of every translation, how could I possibly achieve my task. And this all breaks down if you just think one step beyond the things like gender, if there is ambiguity in a translation, and you give me all the outputs, what am I supposed to know, I go to Google Translate, because I don't know what something means. And especially if you don't give me actual probabilities together with the possibilities, I have no clue what to do. But let's go into this a little bit more. See, if we go to this original sentence and explore Google a little bit more, you might ask why is not even consistent across the entire thing I input Google splits by sentences, it's pretty clear, because once you hover over it, you get the different sentences right here, you can solve this by inputting a comma, in which case, at least within a sentence, the translation is consistent. This is not always the case. But it gives you a little bit of a hint on how Google Translate works. Moreover, if you just input a single word, Google will actually give you the output distribution over all the translations here. The second thing is if you input an entire sentence, and it has a gender pronoun, Google actually gives you both versions. And it says that translations are gender specific. It is only when you input more than one sentence that this doesn't work anymore. In fact, if I make this into one sentence, Google gives me both versions. And this is already the corner case, because technically, it should give me every combinatorial version of the different assignments of these four variables right here. So you can clearly see that Google is doing everything it can to give you a good practical solution that still makes sense in the majority of use cases, people use Google Translate, because they want to get an idea of what something in a language means that they don't understand, they don't go to Google Translate to draft their formal letters that must be absolutely correct. So I think the accusation against Google here and saying things like fu Google, and honestly, Google has found a super pragmatic solution. And I think they're just doing the best they can in the face of the overwhelming complexity that is machine translation. All of that being said, there is a fourth category, a category of people that says that even if we derive the training data correctly, and it reflects the world, even if our algorithm does not introduce any additional bias, even if the output probability distribution is the correct probability distribution for that translation, this is still not good, because they see the problem here in reality itself, it is reality that doesn't conform to some preconceived notion. And this might have multiple reasons. For example, a person arguing this might argue that if we output the correct probability distribution, that might have some downstream effects, or it might reinforce the stereotypes or a number of other arguments, someone arguing like this would see ml models more as tools for social engineering, which is a valid stance to have not criticizing that any of this pipeline is wrong, but that the original bias that exists in the world is carried over into these outputs. And we should change that in order to affect the world. Now, while that is valid stance to have, and certainly debatable, you have to ask yourself whether you really want to give Google a multi billion multi national corporation, the almost monopolistic power to decide on what's good and bad for society. And personally, I'm going to go no with this one. In any case, what I want you to take away from this is that there are many possible places where problems can be introduced, and therefore many possible points where we can introduce solutions. But what we have to be careful of is that we don't confuse the different points and we don't let people provide evidence for one particular point of problem and then suggest a solution that is in an entirely different area. All right, that was it for me. I hope this was at least a little bit entertaining. Bye bye.
 So you might have seen this tweet. Hungarian is a gender neutral language. It has no gender
 pronouns so Google Translate automatically chooses the gender for you. Here is how everyday sexism
 is consistently encoded in 2021. F you Google. On the left hand side is a Hungarian sentence.
 Google Translate then translates this to the following text saying she is beautiful. He is
 clever. He reads, she washes the dishes, he builds, she sues, he teaches, she cooks. So Google
 Translate chooses the gender pronoun and it appears to choose gender pronouns that are very
 consistent with common gender stereotypes. So this has generated a lot of outrage and the topic is
 coming up again and again. And I thought we just dig a little bit into the background of why this
 happens and what we might do about it. So the first thing you might notice is the text here is really
 a bouquet of stereotypes and also ends with go to hell Google. So no doubt this person has tried a
 bunch of things. So I've kind of reproduced the first four sentences of the input. And here it is,
 she is beautiful. He is clever. He reads, she washes the dishes. Now to detect whether or not
 this is a feature of the language, maybe there are subtle gender hints. Here is a thing you can do.
 You can translate it back into the other direction. She is beautiful. He is clever,
 which will give you the Hungarian sentence. And then we can simply change the pronouns
 right here. He is beautiful. She is clever. If there are subtle language hints, you would expect
 that if you translate this to Hungarian and back that the same sentence returns. However, if this
 is a truly gender neutral language, then you would not expect this to matter. So if we now translate
 this to Hungarian and then we take this Hungarian sentence and translate it back, oh, see, it has
 actually switched around the pronouns back to she is beautiful. He is clever. So no doubt Google
 Translate here is inferring the pronoun from the words that follow assigning beautiful to a more
 feminine pronoun, assigning clever to more masculine pronoun. These are gender stereotypes,
 and we're going to dig a little bit into why this happens. For that, we have to understand how the
 machine learning systems currently work. Machine learning systems are statistical systems that try
 to translate a piece of text into a piece of text of a different language. So here we enter
 the piece of text in one language, it goes into this big ML box, and outcomes actually not a
 single sentence, but outcomes usually a plethora of possible sentences, along with probabilities
 assigned to each of those outputs, the system then chooses the most likely output and displays that
 to the user already said this is a statistical system, it is derived from a set of training data.
 So it's important to understand that all the system does is tell us that the sentence she is
 beautiful is the most likely sentence to appear in a document that is translated from Hungarian
 where this original sentence was present, given the training data, the training data itself is,
 of course, derived from the world in some way, if you believe that such a thing as reality exists.
 And there we have the whole system. Now we might ask ourselves, what do we do about it? How should
 we fix this? And the answer, unfortunately, is it depends. It depends on where you think the problem
 lies. So the first point where there could be a problem is the way we derive the training data
 from the world or from reality itself. Common issues here are that the sampling of data is
 somehow skewed, it is out of date, we're working with old data. In general, the data that we have
 does not reflect the world. And if the data that we have is skewed in some way, we can only expect
 that our machine learning system picks up on that skew. So a person arguing this would say that it
 is actually not that likely that these sent Hungarian sentence here translates to she is beautiful.
 And it might be equal, you're more likely that it translates to something else. If we only had all
 the translation data that we could hope of the second point where we could introduce problems
 is when we derive the ML system from the training data. Here's the thing, every machine learning
 system introduces statistical biases in order for it to generalize properly. Otherwise, we could not
 do learning. And it's entirely possible that some of these things such as the regularizer and the
 loss function, or the particular choice of architecture would introduce statistical bias
 into the system. This would result in a model that does not reflect the data as we have it.
 So someone arguing for this would argue that even though we have good training data in the training
 data, there is no problem. The ML system derived from the training data introduces unwanted effects.
 So someone might argue even though the feminine version here is slightly bigger in the training
 data than the masculine version, through the process of learning and distilling the ML model
 simply abstracts this and makes it a lot more likely therefore skewing the gender balance
 unfairly. The last problem is the fact that we simply choose the top prediction and output that
 to the user. This is not really accurate. If we simply output whatever is most likely, this is an
 unfair representation. In fact, what we should do is we should give the user all the possibilities
 with all the probabilities associated. Someone arguing for this might say that the training data
 is fine. The ML model even makes good outputs, the probability distributions are correct and reflect
 the world. However, because we only pick the top one, the user is tricked into thinking that that
 is the only possibility or maybe just that this possibility is much more likely than the alternatives.
 As good as that sounds to output always the probabilities associated with different ambiguous
 translations. The short answer of why we don't do this is pragmatics. I'll give you an example.
 This is Billy Billy. It's a Chinese video sharing websites and for people who cannot access YouTube
 from China, I do upload my videos to Billy Billy so they can watch them. However, while I'm practicing
 Mandarin, I'm not good enough yet to navigate a site that is full of characters that I have even
 a difficult time parsing. And this is what Google Translate is usually used as I just want to
 navigate effectively to the point where I can upload a video define its categories, leave a
 description, and then send that off. If Google Translate were to give me every possible ambiguity
 of every translation, how could I possibly achieve my task. And this all breaks down if you just think
 one step beyond the things like gender, if there is ambiguity in a translation, and you give me
 all the outputs, what am I supposed to know, I go to Google Translate, because I don't know what
 something means. And especially if you don't give me actual probabilities together with the
 possibilities, I have no clue what to do. But let's go into this a little bit more. See, if we go to
 this original sentence and explore Google a little bit more, you might ask why is not even consistent
 across the entire thing I input Google splits by sentences, it's pretty clear, because once you hover
 over it, you get the different sentences right here, you can solve this by inputting a comma,
 in which case, at least within a sentence, the translation is consistent. This is not always the
 case. But it gives you a little bit of a hint on how Google Translate works. Moreover, if you just
 input a single word, Google will actually give you the output distribution over all the translations
 here. The second thing is if you input an entire sentence, and it has a gender pronoun, Google
 actually gives you both versions. And it says that translations are gender specific. It is only when
 you input more than one sentence that this doesn't work anymore. In fact, if I make this into one
 sentence, Google gives me both versions. And this is already the corner case, because technically,
 it should give me every combinatorial version of the different assignments of these four variables
 right here. So you can clearly see that Google is doing everything it can to give you a good
 practical solution that still makes sense in the majority of use cases, people use Google Translate,
 because they want to get an idea of what something in a language means that they don't understand,
 they don't go to Google Translate to draft their formal letters that must be absolutely correct.
 So I think the accusation against Google here and saying things like fu Google, and honestly,
 Google has found a super pragmatic solution. And I think they're just doing the best they can in
 the face of the overwhelming complexity that is machine translation. All of that being said,
 there is a fourth category, a category of people that says that even if we derive the training data
 correctly, and it reflects the world, even if our algorithm does not introduce any additional bias,
 even if the output probability distribution is the correct probability distribution for that
 translation, this is still not good, because they see the problem here in reality itself, it is reality
 that doesn't conform to some preconceived notion. And this might have multiple reasons. For example,
 a person arguing this might argue that if we output the correct probability distribution,
 that might have some downstream effects, or it might reinforce the stereotypes or a number of
 other arguments, someone arguing like this would see ml models more as tools for social engineering,
 which is a valid stance to have not criticizing that any of this pipeline is wrong, but that the
 original bias that exists in the world is carried over into these outputs. And we should change that
 in order to affect the world. Now, while that is valid stance to have, and certainly debatable,
 you have to ask yourself whether you really want to give Google a multi billion multi national
 corporation, the almost monopolistic power to decide on what's good and bad for society.
 And personally, I'm going to go no with this one. In any case, what I want you to take away from this
 is that there are many possible places where problems can be introduced, and therefore many
 possible points where we can introduce solutions. But what we have to be careful of is that we don't
 confuse the different points and we don't let people provide evidence for one particular point
 of problem and then suggest a solution that is in an entirely different area. All right,
 that was it for me. I hope this was at least a little bit entertaining. Bye bye.