[Rant] The Male Only History of Deep Learning
deep learning machine learning neural networks history groups ideology
This casting of our field in terms of ideological narrow-sighted group-think is disgusting. Keep Science about ideas!

https://twitter.com/timnitGebru/status/1252752743942328321

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
 Alright, so instead of reviewing a paper today, I thought I might review this thing. So this person on Twitter posted this link to an article called Brief History of Deep Learning from 1943 to 2019 of Machine Learning Knowledge.ai. So let's look at this. Actually let's look at the tweet first. Because this is... I just saw this. The male-only history of deep learning, where you say AlexNet makes history, but ImageNet doesn't, because women's contributions don't count. And contributions from anyone except for white and white-adjacent people for that matter. That is the tweet, and it has 109 retweets, over 400 likes, and people generally agreeing with this sentiment. So the person is expressing concerns that this article is only going over one particular group of people. So let's look at the article. They basically go over the history of neural networks, of deep learning in an algorithmic sense. So let's check it out. So first we go into neurons, starting at 1943. And the Perceptron paper right here. The first backpropagation algorithm from Kelly. This actually, I think people like Schmidhuber would be proud. As far as I can tell, this is kind of more of a forgotten history, or some of these things are more of a forgotten history. Of course, Minsky's paper, very famous. But here backpropagation attributed to this paper, and so on. And you can see things people like Hinton only coming up later here, the Boltzmann machine, backpropagation in neural networks now. So this, as far as I can tell, it's just a take on kind of the history of algorithmic development. And you can see here, it really is about algorithms. The algorithms behind deep learning. So here is the vanishing gradient problem, the LSTM as an architectural component, deep belief networks. Then you have GPUs for training. Again, vanishing gradients, AlexNet, then GANs, AlphaGo. So we're now going a bit faster. And then the end, it says, the godfathers win the Turing Award for their immense contribution in advancements in area of deep learning and artificial intelligence. This is a defining moment for those who had worked relentlessly on neural networks when the entire machine learning community had moved away from it in the 1970s. So the article clearly is focused on algorithmic developments in deep learning. And that's why AlexNet is here. Now this person rags that AlexNet is here, but ImageNet isn't. And clearly you can see from the article, ImageNet is a data set. It was not made with deep learning in mind. It was simply made as a data set. It's not an algorithmic development. So GANs are here as well, right? But CelebA isn't. C410 isn't. MNIST isn't. The PantryBank isn't. So I think we've skipped a lot of architectural advancements here, like transformers or all kinds of things here. But the history is clearly about the algorithmic developments. And to reframe this, it clearly states ImageNet doesn't because women's contributions don't count. The insinuation here, absolutely, I find this to be absolutely intellectually dishonest. And they say, and contributions from anyone except for white and white-adjacent people for that matter. At this point you just have to laugh. Because of course the narrative that the person wanted to tell was that it's only white people that count. But then you scroll and you're like, arrrrr, it doesn't fit my narrative, right? This GPU is not a white person. So to make it fit your narrative you have to call white-adjacent... What is white-adjacent? It's like, whatever I don't like I now call white. But people just agreeing with this, I find this absolutely disgusting. And I find the article to be okay. I don't know better. But if you have a problem with... I definitely think there is misattribution in science throughout, even systematic. But to say that ImageNet wasn't included because women's contributions don't count, that is just a straight out lie. And to call people white-adjacent is like, how do you not have a bell in your head that goes ding ding ding ding ding when you do something like this? So I find this to be dishonest, either willfully or just because people have so become used to seeing the world in one particular frame. I think these calls only get big whenever there is money and attention going into a field, right? If you look at any field where it's just a bunch of weirdos doing their thing, the weirdos don't care who's there. They just care about the ideas that people have. And I believe we should take that view in science in general. I don't care who has the idea. And these people do. And I disagree. All right, that was it. Keep pushing back on these things if you agree as well. And keep science for ideas. Thanks.
 Alright, so instead of reviewing a paper today, I thought I might review this thing.
 So this person on Twitter posted this link to an article called Brief History of Deep
 Learning from 1943 to 2019 of Machine Learning Knowledge.ai.
 So let's look at this.
 Actually let's look at the tweet first.
 Because this is...
 I just saw this.
 The male-only history of deep learning, where you say AlexNet makes history, but ImageNet
 doesn't, because women's contributions don't count.
 And contributions from anyone except for white and white-adjacent people for that matter.
 That is the tweet, and it has 109 retweets, over 400 likes, and people generally agreeing
 with this sentiment.
 So the person is expressing concerns that this article is only going over one particular
 group of people.
 So let's look at the article.
 They basically go over the history of neural networks, of deep learning in an algorithmic
 sense.
 So let's check it out.
 So first we go into neurons, starting at 1943.
 And the Perceptron paper right here.
 The first backpropagation algorithm from Kelly.
 This actually, I think people like Schmidhuber would be proud.
 As far as I can tell, this is kind of more of a forgotten history, or some of these things
 are more of a forgotten history.
 Of course, Minsky's paper, very famous.
 But here backpropagation attributed to this paper, and so on.
 And you can see things people like Hinton only coming up later here, the Boltzmann machine,
 backpropagation in neural networks now.
 So this, as far as I can tell, it's just a take on kind of the history of algorithmic
 development.
 And you can see here, it really is about algorithms.
 The algorithms behind deep learning.
 So here is the vanishing gradient problem, the LSTM as an architectural component, deep
 belief networks.
 Then you have GPUs for training.
 Again, vanishing gradients, AlexNet, then GANs, AlphaGo.
 So we're now going a bit faster.
 And then the end, it says, the godfathers win the Turing Award for their immense contribution
 in advancements in area of deep learning and artificial intelligence.
 This is a defining moment for those who had worked relentlessly on neural networks when
 the entire machine learning community had moved away from it in the 1970s.
 So the article clearly is focused on algorithmic developments in deep learning.
 And that's why AlexNet is here.
 Now this person rags that AlexNet is here, but ImageNet isn't.
 And clearly you can see from the article, ImageNet is a data set.
 It was not made with deep learning in mind.
 It was simply made as a data set.
 It's not an algorithmic development.
 So GANs are here as well, right?
 But CelebA isn't.
 C410 isn't.
 MNIST isn't.
 The PantryBank isn't.
 So I think we've skipped a lot of architectural advancements here, like transformers or all
 kinds of things here.
 But the history is clearly about the algorithmic developments.
 And to reframe this, it clearly states ImageNet doesn't because women's contributions don't
 count.
 The insinuation here, absolutely, I find this to be absolutely intellectually dishonest.
 And they say, and contributions from anyone except for white and white-adjacent people
 for that matter.
 At this point you just have to laugh.
 Because of course the narrative that the person wanted to tell was that it's only white people
 that count.
 But then you scroll and you're like, arrrrr, it doesn't fit my narrative, right?
 This GPU is not a white person.
 So to make it fit your narrative you have to call white-adjacent...
 What is white-adjacent?
 It's like, whatever I don't like I now call white.
 But people just agreeing with this, I find this absolutely disgusting.
 And I find the article to be okay.
 I don't know better.
 But if you have a problem with...
 I definitely think there is misattribution in science throughout, even systematic.
 But to say that ImageNet wasn't included because women's contributions don't count, that is
 just a straight out lie.
 And to call people white-adjacent is like, how do you not have a bell in your head that
 goes ding ding ding ding ding when you do something like this?
 So I find this to be dishonest, either willfully or just because people have so become used
 to seeing the world in one particular frame.
 I think these calls only get big whenever there is money and attention going into a
 field, right?
 If you look at any field where it's just a bunch of weirdos doing their thing, the weirdos
 don't care who's there.
 They just care about the ideas that people have.
 And I believe we should take that view in science in general.
 I don't care who has the idea.
 And these people do.
 And I disagree.
 All right, that was it.
 Keep pushing back on these things if you agree as well.
 And keep science for ideas.
 Thanks.