STOCHASTIC MEME DESCENT - Deep Learning Meme Review - Episode 2 (Part 2 of 2)
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper funny meme memes meme review gpt-3 google deepmind haha deep neural networks christmas sunglasses transformers neurips gathertown pytorch tensorflow paddlepaddle review rebuttal proof theory analysis is all you need captcha stock market state of the art attention
#memes #science #ai

Part 2 of Antonio and me examining the latest and greatest of deep learning memes.

Music:
Sunshower - LATASHÁ
Papov - Yung Logos
Sunny Days - Anno Domini Beats
Trinity - Jeremy Blake

More memes:
facebook.com/convolutionalmemes


Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/
BiliBili: https://space.bilibili.com/1824646584

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 At some point I will be able to code you, Janik. You will be able to? To code you. To code me? Yes, so that finally you will release videos in time. ["Random Guessing"] Random guessing, Michael Asperger. 47% accuracy. Nice. Yes. Nice. Yes. If you change the seed, you can get 48. Ha ha, you'll never reach me. Yes, I will. How? Are you coming up with a better algorithm? No, but using a weaker baseline. Ha ha. Getting published is so easy. It's a job. Yeah. It's a job. Janik, do you even, sometimes I realize that, you know, my life, every three months is gonna be like a deadline. Is this real life? This is it. It doesn't get better. Is this the peak? This is it. Like some years ago I thought it was gonna be fun, you know, you just enjoy the life. You just, you know, have nice conversations. Like you try your best. You think about things like for a long time. Then you find, no. That does not sound like machine learning research. Okay. Two things we don't have, long times and thinking. Model overfits on training data. Or, word, new data. I got one paper rejected because the review was like, where is Cypher? That was the review. Where is Cypher? Where is it? Where is it, Antonio? If there's no Cypher, how should I know? How does any paper get accepted without C-Phar? It's called C-Phar. I don't know. Maybe it's called Cypher. I don't know. It's like an abbreviation of something. People who study Latin will call it C-Phar. Social distancing guidelines. COVID-19, 1.5 meters. Minion outlier. That's very true. I'm having something like that to deal with right now. I think I forgot something. If you forgot, it wasn't that important. Yeah, you're right. This could actually work, you know? Like there are these, aren't there these proofs that some of these algorithms only converge if you average over gradients? Yeah. So if you accumulate your gradients technically with a decreasing learning rate, this might work. Yannick, it's all wrong. So. Yeah, that's exactly how it's done. But what's the story behind this? There's no story. There's no story. No. I'll just give you a minute. I didn't get it. Should I really, I should really calculate the, yeah. It's true, right? It's true, yeah. It's actually true. Okay, okay. This actually works. I thought like, okay, Yannick, it's Saturday. I woke up two hours ago. Yeah, it's actually true. It's actually true. Dig move now. Weaver process. Yeah. Beautiful, beautiful. Douchiness. Douchiness, it's a word. I didn't know. Epsilon is expected to grow very large over the next 48 hours. No, no. No. No. You're not. No. It has to be small enough. Enough, small enough. Abstract, introduction results. I was, did I tell you this? Maybe it was also in the other meme review, where it was a paper. It's mine, that's my paper. Well, I remember it was like, in this paper, in this specific paper, that was this, okay, we prove that this is true. And in the introduction, it was like, sometimes. It was like the same thing, but with a sometimes. We show that sometimes, under some assumption, and then you read the paper, it's actually just an example. Not everyone should go, recommended for you. I'm surprised that sometimes I look at the thing, I don't, I will never enjoy it. And then I do. And then I do. Us YouTubers, we have to regularly sacrifice GPUs to the algorithm and recommendation. Yeah, it really likes GPUs. Does it, do you have to burn them? Do you have to make them burn? You have to, like. You have to take like some cooler liquid and sprinkle it on top, and then you have to dance around it and some flowers on top of it. And then you have to eat it. OMG, I love all this water-cooled CPUs. New toothpaste exists, dentists. I didn't get the machine learning thing. There's no machine. Okay, okay, yeah, perfect, perfect. I love this. I don't know why, but it's so good. Yannick, that's the big surprise. At the end of this video, there's going to be a big surprise. What? It's a citation from the office. Okay, but yeah, seriously, for each one of you, Yannick is going to make a good video. I'm going to make a good video. For each one of you, Yannick is going to make a gift. Is it the MATLAB license? Damn it, don't spoil... Forms of birth control, tens of... I should just put machine learning. When your model improves from 5% accuracy to 7% accuracy. Machine learning! Machine learning, finding global minima. Machine learning, finding local minima. Yeah. That's so damn true. Fury people are weird. Fury people are the worst. That's even true, like I'm completely serious, 100% serious. Like they get excited about infinitely wide neural networks. Oh yeah, or what if you take the step size to be infinitely small? Yeah. That's how you do things. I mean, the only thing that's infinitely wide is your mom. Self-driving cars aren't even hard to make lol. Just program it, not to hit stuff. Don't... You know, in all of my code, true story, in all of my code, I write in a line. And it's usually like a common doubt, but I write in a line that says If target equals Yannick, then don't fire. Really, just I anticipate that some of my code will be used in the robot overlord army. Yeah, that's such a smart move. I know. You gotta think ahead. For some reason, they will shoot everything except the traffic lights. How? Interviewer, what's your biggest strength? I'm an expert in machine learning. Now good that we did this this way, because the other way would have been a bit strange. What's 9 plus 10? It's 3. Not even close, it's 19. It's 16. Wrong, it's still 19. It's 18. No, it's 19. It's 19. You're fired. I wonder what GPT-3 would say to this. Should we try it out? We should try it out. When you drop the learning rate. Everyone is freaking out what happened here, but they dropped the learning rate. So clear. It's like, that's what you do. You stagnate, you divide it by 10, shag-a-boom. I'll give you 10 seconds to copy what's on the whiteboard. The whiteboard. It's actually from my video. Yeah, I kind of remember something similar to that. What was this? I have no idea. Not a slightest clue. So this actually is also on my video. They really tried. They really tried. But sometimes I mean, if I make a mistake on a video or something, I'll put like a comment. You never make mistakes. Before I set the video to visible, it's just so mean to the people who want to do this. Mom, if your friends jumped off a bridge, would you jump too? How much time? I needed this meme and I didn't know I needed that. No, you can't just add more parameters and data to model GPT-3 is no different from Elisa since it's just glorified pattern matching and curve fitting not true intelligence which requires a symbolic representation of the input which connectionist models will never be able to do. Also the data needed is almost an entire percent of the total possible data we can collect from common ground alone and the hardware needed to train GPT-3 is unfit with new ass-witching-crasters. The guy is like... Do you think GPT is intelligent? I think he's aware. And that he... Oh my God, no, no, no. Oh, we're going to leave this in Elkrochus. Do you think GPT-3 is intelligent though? I think, well, I like the colors. I like the colors of the GPU there. I think that anybody with best colors, it's like slightly, you know, funny. So it can be funny, you know, but not intelligent. Do you think? I think it is... It's not? Is... I think it is. It is... I'll be canceled for like the 50th time. Researchers hate him. Local man discovers one weird trick to general intelligence. Turns out you just weren't using enough layers. Learn the secret to a stunning result. Learn the truth now. Yeah. Yes, that's again me. That's again me. Own it. The stickers, the stickers, they own it. Own it. And that is probably the Adam paper. Do you know that Adam proof is famously wrong? I very much know. Oh yeah, yeah, I do. I just heard it. I just repeat it to sound smart. No, I know it. I know it. It's like there are at least four mistakes in that proof. And the thing that it got probably like 30,000 citations before, before realizing that it was... It's still getting citations, no? No, you know, the second part of a story, well, now it's 60,000. The other paper, the paper that fixes the mistake introduces AMS grad. The proof, the mistake, basically the V variable. Yeah. Then it's a problem for the proof. OK. And AMS grad fixes the mistake. But now there's another paper that tells that actually Adam does converge. So we go back to the fact, no, no, guys, no. It just did it wrong. It just did it wrong. But yeah. It's like when you don't use the method your teacher wants you to use. Exactly. Yeah. But nobody used AMS grad. Yeah. Nobody ever used it. No. I spit on AMS grad. I really don't like it. Albert Einstein. Insanity is doing the same thing over and over again and expecting different results. That's how I make papers. Come on. Seed equals to. Or maybe like resubmission. How it started. Jaleco. Against the mob. This is a very dark period. How it's going? In the channels. Jaleco versus Twitter. Yeah. Yeah. We have a superstar right here. We don't. We don't. We don't talk about this. No, no, we don't. We don't talk about this. That's nothing happened. Nothing happened. Nvidia new AI be like. That's what they do now. You're like how many millions of dollars are going into just making your eyes go. Crazy. Mew for God. Free loop. All right. That was it for in review. Thank you so much for watching. Thank you. Thank you. I want to thank Yannick for having me here. It is always a pleasure. Yeah. And hopefully 2021 will have also cake Yannick. Where the hell is the cake? More cake. Yeah. Bye bye. Bye.
 At some point I will be able to code you, Janik.
 You will be able to?
 To code you.
 To code me?
 Yes, so that finally you will release videos in time.
 ["Random Guessing"]
 Random guessing, Michael Asperger.
 47% accuracy.
 Nice.
 Yes.
 Nice.
 Yes.
 If you change the seed, you can get 48.
 Ha ha, you'll never reach me.
 Yes, I will.
 How?
 Are you coming up with a better algorithm?
 No, but using a weaker baseline.
 Ha ha.
 Getting published is so easy.
 It's a job.
 Yeah.
 It's a job.
 Janik, do you even, sometimes I realize that, you know,
 my life, every three months is gonna be like a deadline.
 Is this real life?
 This is it.
 It doesn't get better.
 Is this the peak?
 This is it.
 Like some years ago I thought it was gonna be fun,
 you know, you just enjoy the life.
 You just, you know, have nice conversations.
 Like you try your best.
 You think about things like for a long time.
 Then you find, no.
 That does not sound like machine learning research.
 Okay.
 Two things we don't have, long times and thinking.
 Model overfits on training data.
 Or, word, new data.
 I got one paper rejected because the review was like,
 where is Cypher?
 That was the review.
 Where is Cypher?
 Where is it?
 Where is it, Antonio?
 If there's no Cypher, how should I know?
 How does any paper get accepted without C-Phar?
 It's called C-Phar.
 I don't know.
 Maybe it's called Cypher.
 I don't know.
 It's like an abbreviation of something.
 People who study Latin will call it C-Phar.
 Social distancing guidelines.
 COVID-19, 1.5 meters.
 Minion outlier.
 That's very true.
 I'm having something like that to deal with right now.
 I think I forgot something.
 If you forgot, it wasn't that important.
 Yeah, you're right.
 This could actually work, you know?
 Like there are these,
 aren't there these proofs that some of these algorithms
 only converge if you average over gradients?
 Yeah.
 So if you accumulate your gradients technically
 with a decreasing learning rate, this might work.
 Yannick, it's all wrong.
 So.
 Yeah, that's exactly how it's done.
 But what's the story behind this?
 There's no story.
 There's no story.
 No.
 I'll just give you a minute.
 I didn't get it.
 Should I really, I should really calculate the,
 yeah.
 It's true, right?
 It's true, yeah.
 It's actually true.
 Okay, okay.
 This actually works.
 I thought like, okay, Yannick, it's Saturday.
 I woke up two hours ago.
 Yeah, it's actually true.
 It's actually true.
 Dig move now.
 Weaver process.
 Yeah.
 Beautiful, beautiful.
 Douchiness.
 Douchiness, it's a word.
 I didn't know.
 Epsilon is expected to grow very large
 over the next 48 hours.
 No, no.
 No.
 No.
 You're not.
 No.
 It has to be small enough.
 Enough, small enough.
 Abstract, introduction results.
 I was, did I tell you this?
 Maybe it was also in the other meme review,
 where it was a paper.
 It's mine, that's my paper.
 Well, I remember it was like, in this paper,
 in this specific paper, that was this,
 okay, we prove that this is true.
 And in the introduction, it was like, sometimes.
 It was like the same thing, but with a sometimes.
 We show that sometimes, under some assumption,
 and then you read the paper, it's actually just an example.
 Not everyone should go, recommended for you.
 I'm surprised that sometimes I look at the thing,
 I don't, I will never enjoy it.
 And then I do.
 And then I do.
 Us YouTubers, we have to regularly sacrifice GPUs
 to the algorithm and recommendation.
 Yeah, it really likes GPUs.
 Does it, do you have to burn them?
 Do you have to make them burn?
 You have to, like.
 You have to take like some cooler liquid
 and sprinkle it on top, and then you have to dance around it
 and some flowers on top of it.
 And then you have to eat it.
 OMG, I love all this water-cooled CPUs.
 New toothpaste exists, dentists.
 I didn't get the machine learning thing.
 There's no machine.
 Okay, okay, yeah, perfect, perfect.
 I love this.
 I don't know why, but it's so good.
 Yannick, that's the big surprise.
 At the end of this video, there's going to be a big surprise.
 What?
 It's a citation from the office.
 Okay, but yeah, seriously, for each one of you,
 Yannick is going to make a good video.
 I'm going to make a good video.
 For each one of you, Yannick is going to make a gift.
 Is it the MATLAB license?
 Damn it, don't spoil...
 Forms of birth control, tens of...
 I should just put machine learning.
 When your model improves from 5% accuracy to 7% accuracy.
 Machine learning!
 Machine learning, finding global minima.
 Machine learning, finding local minima.
 Yeah.
 That's so damn true.
 Fury people are weird.
 Fury people are the worst.
 That's even true, like I'm completely serious, 100% serious.
 Like they get excited about infinitely wide neural networks.
 Oh yeah, or what if you take the step size to be infinitely small?
 Yeah.
 That's how you do things.
 I mean, the only thing that's infinitely wide is your mom.
 Self-driving cars aren't even hard to make lol.
 Just program it, not to hit stuff.
 Don't...
 You know, in all of my code, true story, in all of my code, I write in a line.
 And it's usually like a common doubt, but I write in a line that says
 If target equals Yannick, then don't fire.
 Really, just I anticipate that some of my code will be used in the robot overlord army.
 Yeah, that's such a smart move.
 I know.
 You gotta think ahead.
 For some reason, they will shoot everything except the traffic lights.
 How?
 Interviewer, what's your biggest strength?
 I'm an expert in machine learning.
 Now good that we did this this way, because the other way would have been a bit strange.
 What's 9 plus 10?
 It's 3.
 Not even close, it's 19.
 It's 16.
 Wrong, it's still 19.
 It's 18.
 No, it's 19.
 It's 19.
 You're fired.
 I wonder what GPT-3 would say to this.
 Should we try it out?
 We should try it out.
 When you drop the learning rate.
 Everyone is freaking out what happened here, but they dropped the learning rate.
 So clear.
 It's like, that's what you do.
 You stagnate, you divide it by 10, shag-a-boom.
 I'll give you 10 seconds to copy what's on the whiteboard.
 The whiteboard.
 It's actually from my video.
 Yeah, I kind of remember something similar to that.
 What was this?
 I have no idea.
 Not a slightest clue.
 So this actually is also on my video.
 They really tried.
 They really tried.
 But sometimes I mean, if I make a mistake on a video or something, I'll put like a comment.
 You never make mistakes.
 Before I set the video to visible, it's just so mean to the people who want to do this.
 Mom, if your friends jumped off a bridge, would you jump too?
 How much time?
 I needed this meme and I didn't know I needed that.
 No, you can't just add more parameters and data to model GPT-3 is no different from Elisa
 since it's just glorified pattern matching and curve fitting not true intelligence which requires a symbolic representation of the input which connectionist models will never be able to do.
 Also the data needed is almost an entire percent of the total possible data we can collect from common ground alone
 and the hardware needed to train GPT-3 is unfit with new ass-witching-crasters.
 The guy is like...
 Do you think GPT is intelligent?
 I think he's aware.
 And that he... Oh my God, no, no, no.
 Oh, we're going to leave this in Elkrochus.
 Do you think GPT-3 is intelligent though?
 I think, well, I like the colors.
 I like the colors of the GPU there.
 I think that anybody with best colors, it's like slightly, you know, funny.
 So it can be funny, you know, but not intelligent.
 Do you think?
 I think it is...
 It's not?
 Is...
 I think it is.
 It is...
 I'll be canceled for like the 50th time.
 Researchers hate him.
 Local man discovers one weird trick to general intelligence.
 Turns out you just weren't using enough layers.
 Learn the secret to a stunning result.
 Learn the truth now.
 Yeah.
 Yes, that's again me.
 That's again me.
 Own it.
 The stickers, the stickers, they own it.
 Own it.
 And that is probably the Adam paper.
 Do you know that Adam proof is famously wrong?
 I very much know.
 Oh yeah, yeah, I do.
 I just heard it. I just repeat it to sound smart.
 No, I know it. I know it.
 It's like there are at least four mistakes in that proof.
 And the thing that it got probably like 30,000 citations before, before realizing that it was...
 It's still getting citations, no?
 No, you know, the second part of a story, well, now it's 60,000.
 The other paper, the paper that fixes the mistake introduces AMS grad.
 The proof, the mistake, basically the V variable.
 Yeah.
 Then it's a problem for the proof.
 OK.
 And AMS grad fixes the mistake.
 But now there's another paper that tells that actually Adam does converge.
 So we go back to the fact, no, no, guys, no.
 It just did it wrong.
 It just did it wrong. But yeah.
 It's like when you don't use the method your teacher wants you to use.
 Exactly. Yeah.
 But nobody used AMS grad.
 Yeah.
 Nobody ever used it.
 No.
 I spit on AMS grad.
 I really don't like it.
 Albert Einstein.
 Insanity is doing the same thing over and over again and expecting different results.
 That's how I make papers. Come on.
 Seed equals to.
 Or maybe like resubmission.
 How it started.
 Jaleco.
 Against the mob.
 This is a very dark period.
 How it's going?
 In the channels.
 Jaleco versus Twitter.
 Yeah.
 Yeah.
 We have a superstar right here.
 We don't.
 We don't.
 We don't talk about this.
 No, no, we don't.
 We don't talk about this.
 That's nothing happened.
 Nothing happened.
 Nvidia new AI be like.
 That's what they do now.
 You're like how many millions of dollars are going into just making your eyes go.
 Crazy.
 Mew for God.
 Free loop.
 All right.
 That was it for in review.
 Thank you so much for watching.
 Thank you.
 Thank you.
 I want to thank Yannick for having me here.
 It is always a pleasure.
 Yeah.
 And hopefully 2021 will have also cake Yannick.
 Where the hell is the cake?
 More cake.
 Yeah.
 Bye bye.
 Bye.