When BERT Plays the Lottery, All Tickets Are Winning (Paper Explained)
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper bert nlp lottery ticket good bad winning pruning weights attention transformer heads multi-head fine-tuning glue benchmark
BERT is a giant model. Turns out you can prune away many of its components and it still works. This paper analyzes BERT pruning in light of the Lottery Ticket Hypothesis and finds that even the "bad" lottery tickets can be fine-tuned to good accuracy.

OUTLINE:
0:00 - Overview
1:20 - BERT
3:20 - Lottery Ticket Hypothesis
13:00 - Paper Abstract
18:00 - Pruning BERT
23:00 - Experiments
50:00 - Conclusion

https://arxiv.org/abs/2005.00561

ML Street Talk Channel: https://www.youtube.com/channel/UCMLtBahI5DMrt0NPvDSoIRQ

Abstract:
Much of the recent success in NLP is due to the large Transformer-based models such as BERT (Devlin et al, 2019). However, these models have been shown to be reducible to a smaller number of self-attention heads and layers. We consider this phenomenon from the perspective of the lottery ticket hypothesis. For fine-tuned BERT, we show that (a) it is possible to find a subnetwork of elements that achieves performance comparable with that of the full model, and (b) similarly-sized subnetworks sampled from the rest of the model perform worse. However, the "bad" subnetworks can be fine-tuned separately to achieve only slightly worse performance than the "good" ones, indicating that most weights in the pre-trained BERT are potentially useful. We also show that the "good" subnetworks vary considerably across GLUE tasks, opening up the possibilities to learn what knowledge BERT actually uses at inference time.

Authors: Sai Prasanna, Anna Rogers, Anna Rumshisky

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
 Hi there. Today we're looking at when BERT plays the lottery. All tickets are winning by Sy, Prasanna, Anna Rogers and Anna Rumschiski. So a high level overview of this paper is the following. The paper basically looks at BERT in terms of the lottery ticket hypothesis and it says that if you fine tune BERT on different downstream tasks, then the lottery ticket winners you're going to find are different between the tasks. And also the claim all tickets are winning refers to the fact that if you remove the winning tickets, then you can still train the rest to relatively good performance. Therefore, all tickets are winning, not just the sub network. So that was the high level overview for those of you who just want to be interested, if you want to continue watching this video. If you do like videos like this, consider sharing, liking, subscribing, telling your mother, father, brother and friends about it. Alright, let's dive in. So BERT, it is a language model. Basically, if you don't know what BERT is, I've done a video on BERT, but really quickly, what you can do with BERT is you can take a sentence, something like hello there, and you can put it through this multi-layer neural network. And what you'll get out is basically an embedding of that sentence. So a vector embedding of it. We'll make it really easy. And what is usually done is this is pre-trained on a task called masked language modeling. This is unsupervised training. And then you take this embedding and you fine tune. Basically, you put on a classifier head. Basically, say, let's take sentiment classification. So you have two output classes. And you want to say, is this sentence I put in positive or negative sentiment? So you would train this classifier by basically taking this part from the pre-trained masked language modeling and then training the part here that does the sentiment classification. You would sort of add that on top and then fine tune the entire network to solve this task. So that is basically BERT fine tuning on different tasks. And there is this benchmark called GLUE, where it has a number of tasks. In this case, I think they look at nine tasks of GLUE. It has nine of these tasks. One is the, an example is the sentiment classification. And it basically gets a score for each one. And thereby, you can sort of estimate how good your language model is by how well it is performing on each of those individual tasks. But the notable difference here, too, let's say a computer vision, like an ImageNet classifier is the fact that first it is pre-trained, right? This part here is pre-trained on a large corpus. And second, there are different downstream tasks that you fine tune on. So the second part that is important is the lottery ticket hypothesis. So I've also done a video on the lottery ticket hypothesis. And if you very quickly what the lottery ticket hypothesis is the following. So let's say you have an image classifier. And I have a bunch of layers in my neural network. I'm going to draw them like this. And at the end, right, I can classify it into like 10 different or a thousand different classes, whatever. And the input here is an image. So my neural network is going to have weights. So every one of these neurons is connected to each other. Now this can be a convolutional network or a MLP. So all is connected to all and as well here, right, everything is connected to pretty much everything. So we know, first of all, we know that we can train these big networks to relatively good accuracy. And then second of all, so first, we can train them. Second, we know we can prune them after training. What pruning means is the fact that after I have trained such a thing, I can then go and I can figure out which one which of these connections that I have learned are the important ones. And maybe I'll say, ah, these these here, actually these these five, I don't need more than those. Let's actually connect them to the end. These seven or so. I don't need all of the other ones. I just need those. And I can pretty much get the same accuracy as the full network. Now, the important part here is that you can only do pruning after you've trained a network. If you try to prune at the beginning, it doesn't work. So what the lottery ticket hypothesis says is basically, so how does training work? First you have your parameters. Let's let's tell them as a list. So each of these weights is an entry in the list here. First, you initialize these randomly, then through training, training, you get to your train state, right? You get each of the ones into your these are now trained. And in the train state, you can select the ones you think are important. The lottery ticket hypothesis says if I take those that are important and basically go back to the beginning, like here, here, and here, and I basically roll them back to that state that they were in when they were initialized. So I put the same random number there that I got at initialization. I can then make a network where I only have those. I can train that network and I can get a good accuracy. So this basically wasn't possible in the pruning framework because we said we can't prune, we can only prune after training because only then do we know which ones are the important ones. In fact, the lottery ticket hypothesis or the paper shows that you can train the smaller neural network from the beginning, but the catch of course is you have to know which ones those are and you have to know what value to set them on at the beginning. And you only know that after you've trained the full network. But still it kind of gives the, it tells you that you don't need all these connections for training. You basically only need so many connections such that somewhere in there, there's going to be the good ones, right? And if you knew the good ones from the beginning, you could just train those and then only train the smaller sub network. So that's the lottery ticket hypothesis. Naturally these connections here, this small network is called the winning, a winning ticket because if you knew what it was, you could basically train a much smaller network and reach the same accuracy. So this paper looks at BERT in terms of this lottery ticket hypothesis. Now it's a bit more complicated than just in these feed forward networks because BERT is not a feed forward network. BERT is a transformer. So what does that mean? A transformer consists of many layers and each layer, let's expand the layer here. So each layer consists of, let's go over there, need some space. So again, we have our layers of BERT and it goes, the signal goes like this. So each layer consists of two things. First of all, of many attention heads, that's called. Now I'm going to draw these as blocks right here. So four, let's call them four attention heads. Individual attention heads are all parameterized by individual matrices. And then on top of that, there is an MLP. So this is the multi-layer perceptron. This is one, basically a feed forward layer, residual actually. And so there is a skip connection right here. And these are the attention heads. Okay. And then the next layer would again have the same structure, four of these and then one of these and so on with the skip connection. All right. So the pruning in BERT is different than pruning in the feed forward or convolutional layer that we looked at. Pruning in BERT to what this paper looks at is pruning either an entire attention head like this. So kind of leaving out in the entire head away, which is, this is an entire matrix. This is not a single weight, right? This is many, many weights or even more drastically leaving away an entire MLP and basically only relying on the skip connection. All right. So this, you have these two things you can do. You can leave away heads or you can leave away entire MLPs or you can combine these things in some way. Right. So the notable difference here to the lottery ticket hypothesis pruning is the fact that here over up here, what we prune are connections. So prune connections, individual connections, individual connections. And here we prune entire modules. Now this is a, in my opinion, this is a qualitative difference, a very large qualitative difference actually. Why would you do this? So this paper basically doesn't invent this kind of pruning. They go after already existing literature. So what's the advantage in pruning modules? Well, you have to see what's the advantage in pruning per se. So in pruning, what you're trying to do is obtain a smaller network that gives you the same accuracy, but that you can run faster, right? That uses less memory and you can run it faster. And if you prune like this, like we did in the lottery ticket hypothesis paper, you don't really gain anything because if you have a matrix, if you have a matrix, matrix multiply, right? I have two matrices and I multiply them right here. If I cut out one weight here or one weight here, it doesn't help me because I have GPUs and those will parallelize these matrix procedures. And it doesn't really help me because we don't really have good hardware for sparse or matrix, matrix multiplication or matrix, matrix multiplication with holes or things like this. So it almost gains me nothing. The lottery ticket hypothesis paper is very much a kind of more of a scientific curiosity paper. And once we have sparse matrix multiply hardware, which I think already exists, but is not super widely distributed, once we have that, we will be able to make use of this. Whereas the people that prune BERT, so these are more, let's say, industry people. If you prune an entire module, well, that's an entire matrix that falls away. So I have to, I can basically save an entire matrix, matrix multiply in the forward pass here and the backward pass. Well, okay, I don't prune during training, but I can basically save an entire matrix multiply here by pruning an entire module. So I'm not sure if I were an author and I say I want to look at BERT in terms of the lottery ticket hypothesis, I would find a way, I would go away from this literature and find a way to also just prune here individual weights. It's not going to be faster, but the lottery ticket investigations aren't supposed to be faster, they're supposed to tell you something about the nature of the things you're investigating. And of course how you do this is simply by masking, right? You simply force these entries to be zero and therefore you don't have forward signal, you don't have gradient. Interestingly, they actually do the masking, but they do it on the whole entire module level. Okay, so this was BERT and the lottery ticket hypothesis and the all tickets are winning, we're going to investigate later. Let's see what they say in the abstract. Say much of the recent success in NLP is due to the large transformer based models such as BERT. Okay, they say however these models have been shown to be reducible to a smaller number of self-attention heads and layers. So this would be pruning. We consider this phenomenon from the perspective of the lottery ticket hypothesis. For fine-tuned BERT we show that, here's the contributions, A, it is possible to find a sub network of elements that achieves performance comparable with that of the full model. So basically this is the pruning objective, right? You want to prune it such that the performance holds and in terms of the lottery ticket hypothesis you want to prune, reset to the beginning and then also and then train again and that will give you, actually in the lottery ticket hypothesis you can gain performance if you prune by a certain amount. In this case here they always lose performance but yeah. So second of all similarly sized sub networks sampled from the rest of the model so the non-winning ticket perform worse. So if you just prune away the good parts then the bad parts perform worse of course. However the bad sub networks can be fine-tuned separately to achieve only slightly worse performance than the good ones indicating that most weights in the pre-trained BERT are potentially useful. So this is interesting. If they be fine-tuned separately this is exactly what the lottery ticket hypothesis is doing, right? It's basically fine-tuning only a sub part of the network and here they say even if we take the parts of the network that have low scores for pruning and we retrain those then we can achieve a good performance. So further they say we also show that the good sub networks vary considerably across glue tasks. This is this benchmark opening up the possibilities to learn what knowledge BERT actually uses at inference time. Alright so this is the overview of the paper. So a last thing to say which I've already kind of alluded to is the fact that in the original lottery ticket hypothesis as I said you had a graph and you had some sort of here was 100% accuracy and here was how much you prune. Of course you start at 100% if you prune nothing but then as you prune the interesting thing is it kind of goes up and then it goes down. So this is the first thing here it goes up to a certain amount if you don't prune and in the original lottery ticket hypothesis here somewhere here would be 50% of the network I think. And then once you go down let's say here to 90% of performance you are at something like 5% of the network size or 3%. So you can prune away most things and still be like extremely extremely powerful. Now we're going to see what these essentially what these people do here is here is 100% and they simply prune until they reach 90%. So we don't necessarily know what happens in the middle we just know they start here and somehow they get to 90% and what they end up with is something like 50% of the network still remaining. So again see the qualitative difference here between the 5% of the lottery tickets in the original paper and the 50-ish or so percent or considerable amount more in this paper right here and I'm pretty sure that is due to the fact that they prune entire modules here so they don't prune on a fine-grained enough level to investigate these phenomenon because as I said we don't know but I'm pretty sure this just goes down here and not up first. So qualitatively it seems different. Alright so here they introduce what they do again ERT is made up of these attention heads and MLPs. The MLPs have a skip connection as you can see here and the attention head attention layers are basically made up each of N of these attention heads. What they will do is they will look at 12 layer networks. Each layer will have 12 of these attention heads and one of the MLPs. So you have in total 144 heads and 12 MLP layers. The way they determine which ones to prune is pretty easy. In front of each attention head and in front of each MLP they put one of these binary variables right here. These variables can take values 0 or 1. If they are 0 the layers or the head is basically inactive, no propagation. If they are 1 they are active. And they determine what value to set them to by computing important scores. Basically determining how important is a head or a layer for the network. And that's pretty simple. You simply take the gradient of the loss. I think they go after this paper right here that's supposed to be the following. You derive the loss by these variables right here and therefore you get these important scores. And then you can simply prune the layers with the lowest important scores because that means that the gradient with respect to them is the smallest. That means your loss changes the least if you were to leave them away. So they here determine their pruning strategy. Their constraint here is as I said 90% of the performance of the full model. So they train the full model, fine tune the full model on this task and then they set themselves a budget of 90% and they simply prune until the model reaches 90%. Once it goes lower they stop. So they have three methods of pruning. One is heads only where they only cut away these attention heads. As I said there are 144 of them. They have the pruning strategy of MLPs only where they only prune the MLPs, leave all the attention heads alone. Then they have this heads and MLPs. They say we compute head and MLP important scores in a single backward pass, pruning 10% heads and one MLP with the smallest scores until the performance on the dev set is within 90%. Then we continue pruning heads alone and then MLPs alone. This I guess until again they are no longer in the 90% so until they reach their budget. So this is a combined strategy. This strategy results in a larger number of total components pruned within our performance threshold. So this is the thing we should focus on right because in pruning the name of the game is how much can you take away and still be within your budget. This strategy seems to be the viable strategy here. A last thing here is fine tuning. So the other difference between this paper and the lottery ticket hypothesis is that we said that in the original paper here these are randomly initialized weights. Like you train a class for an ImageNet or something, you start from randomly initialized weights and the lottery ticket papers they all kind of presuppose random initializations. Whereas BERT, when you do the same thing for BERT, these are not random initializations. We said in BERT what you usually do is you train the encoder part here. You pre-train with masked language modeling first and then second you train the entire thing. Let's skip the color here. Second you train the entire thing. You fine tune the entire thing. So if we talk about initializations in the BERT task then the initialization would be at this point right here after the masked language modeling would be the initialization. So the weights are not random. The weights are actually pre-trained on the masked language modeling task which is also a qualitative difference and sort of lets us inspect. So the authors say that since we trained with masked language modeling and people sort of claim that masked language modeling learned something about the language, we can now investigate kind of which attention heads, which modules in BERT are encoding which parts of the language. And this is going to be interesting once we look at which attention heads and which modules survive in the individual tasks, we can sort of compare tasks across each other by seeing which of the heads they share in their winning tickets. Alright, so they produce these graphs here. These are sort of one of the central graphs here and the way to read this is on the left side here you have the layer, you have the layer index and on the x-axis you simply have the index of the head. There are 144 boxes here. Each one corresponds to one of the attention heads. The top number is always the mean number of glue tasks that this head survived in. So what they do is they take the pre-trained BERT, they fine tune it on these nine tasks and for each of the nine tasks they determine the winning tickets. And the number here says how many, in how many of these nine tasks is this particular attention head a part of the winning ticket. Now they repeat it for different random seats, that's why you have floating point numbers and the lower part is the standard deviation across that. So you can see quite a number of heads make it into a lot of these tasks. So you can say this part, this thing right here, red on red, this head right here survives in seven out of the nine tasks. So it should be fairly, it should probably encode something fairly substantial about language that is shared across these seven tasks. You can see some of the heads like this one here doesn't survive in almost any task which basically means that it's, you know, that one's not really super important for these tasks. It might have been, you know, important for the pre-training but not for these particular tasks. What's interesting, so what you can see is that the mean or median or so is like three, four or five. And that means that a lot of the heads are sort of somewhat important for some of the tasks. And you can see the qualitative difference. If this were the like original lottery ticket paper, most of these numbers would be at zero because the lottery ticket size is just so much smaller. Here you can directly see that you are going to retain a large number of things in your network in order to get 90% of the performance. And that's probably because you prune entire modules again. So they have this for two variants here. First for the strategy of masking heads only. And the right one is for masking heads and MLPs. And the same here on the bottom. These are the same numbers but not for attention heads but for MLP layers. And you see again this is masking MLPs only. This is masking heads and MLPs. So if you compare the two, you see that for example this here and this here are substantially darker which means more of this stuff survives. Now we can't really... It seems like here for example, it's darker than here. So on the right side more stuff survives but also you have more things to prune, right? You can prune the heads and the MLPs. And they claim before that the masking heads and MLP strategy results in more things being pruned which isn't really congruent with here generally more things surviving. But it could be because of the fact maybe the sum of the two is still lower than the sum of each individual thing here. Though it doesn't really look like it. So I'm a bit confused about this but I'm just going to assume that the sum of the two is lower. Does that make sense if both are darker? Well it shouldn't be the sum... It should be the sum of this plus a completely dark this in terms of masking heads only or vice versa versus the sum of those two, right? So that should be the measure. But it just seems a bit doesn't work out too much. But okay that's what they say. So by the way if the authors are here you have... This is cut off. Haha. Yeah this is annoying. This is like trying to get LoTeC to do things and it doesn't comply. Alright so what you can... Another thing you can see in the authors point out here is that if you mask heads and MLPs you sort of shift more things to the back of the network to the higher up layers. And they reason now because you also mask the heads basically they can't do as much work so the heads would be masked somewhere here. So all that work is going to shift upon the MLPs and mostly to the back of the network because this thing here cannot take over work that this attention head here is now not performing anymore because it was pruned because the signal travels this way. So the authors kind of interpret these results right here and I think the most important thing to see is simply the variance of things. So most heads are actually important for at least two or three tasks and no head is important for all the tasks consistently. I think that's the take home message right here. Okay and they contrast this to previous research that has basically said this experiment falls up on a study by this that showed that only a few transformer heads in machine translation tasks did the heavy lifting while the rest could be pruned. And this paper similarly showed that most of BERT self-attention head in MNLI task could be pruned and that the good heads were mostly shared between the MNLI matched and mismatched. And they basically say yeah that's correct but that is only within one task right if you go beyond if you go to several tasks then the heads that are important differ quite a bit. Okay so let's continue and go here. They ask how task independent are the good self-nerve works and they basically look at these kinds of graphs right here which are pretty interesting. So we've got this. This is heads shared between tasks. So what this measures is these are the different tasks in the glue benchmark and they basically look at each task look at its winning lottery ticket and look at which heads survive in the winning ticket. And then they put that here on the diagonal. So if in QNLI a head survives it gets a 1 here and if it doesn't survive it gets a 0. So on average 85 out of the 144 heads survive right 85 heads survive. That's pretty as I said this is somewhat like over 50% of the network. It's entirely different than the original lottery ticket hypothesis paper. So 85% not percent 85 of the 144 heads survive. Now they look at the other tasks so for QNLI they would look at maybe MNLI task here and ask which of the heads that survived in QNLI also survives in MNLI. So that gets you the shared heads and again the lower numbers is standard deviation. So 62 heads survive in QNLI and MNLI and the authors here are sort of arguing that from these sort of numbers you should be able to see which of the tasks share different linguistic knowledge. So different linguistic knowledge could be relevant for different tasks but if some tasks share a lot of the attention heads that survive in the winning tickets that basically means that the model is using that information that is in that head for both tasks. This could be good in that you say oh yeah these tasks really are used similar linguistic features or it could be something that you don't expect and then you might be able to investigate maybe the model is doing something shady here because it really shouldn't, these tasks don't really have much in common. So they do this for the heads and the MLPs here. Now if you ask why the WNLI here has a bunch of zeros that's because it's a wonky task and basically the best thing you can do is predict the most frequent class. So you can prune just about anything away on these MLPs because they have the skip connections you don't need them to predict the most frequent class. What I want to go into is the following statement right here. So, note that figure one, so the figure before, shows very few heads or MLPs that are universally useless. Only seven heads that survived in less than two tasks. 86% of heads and 67% of MLPs survived in two to seven tasks with relatively high standard deviation. They say this means that the good sub networks for different tasks have relatively little in common. So they make this sort of statement again here that the good sub network have little in common and it might seem like that for the figure initially. But if you look at this figure it actually shows something pretty interesting I think. So if you look at a number, let's say for example this here, this 74 and I haven't actually tried. Yeah let's look at the 74 and this here. So let's look at these tasks QQP and RTE. Okay, so if you look at QQP and RTE you could see that these are tasks that already they don't have a lot of heads in common right and you might be able to say well if what they're saying is true that the tasks share relatively little you would expect them to be relatively independent. But if I look at this 78 here it means that 78 out of 144 heads survive and here it means that 74 out of 144 heads survive. So if I now would think that okay generally for different tasks things are different how many heads would I expect there to be surviving in both if the tasks are independent. So that's these two things multiplied right times 144. So I can scratch this here and the 7 times 7 is whatever 49. Let's go 7 times 8 about this so that's 5, 6. Do I need to get out the calculator? I want to do this calculator. I'm going to do this the right way. Okay I hope you can see that so that's 78 times 74 divided by 144. Did I do it right? I probably did it wrong. 78 times 74 divided by 144. Alright so that's 40 heads and you see that there's 43 heads and I've actually gone through a bunch of these numbers before not these ones but generally the shared number of heads is higher than what one would expect if you assume that the tasks are independent. And I'm sort of missing sort of an analysis of that here because that I find to be a pretty interesting finding of these things and sort of I mean I get the fact that they say based on the graphics up here that the tasks are sort of seem to be relatively independent with respect to the heads that survive and of course relatively independent is a relative term but it's sort of an investigation into why we see considerable difference between tasks here in terms of that. So these numbers are always over what you would assume for independence. That seems to be pretty interesting. Alright so they say they here go into this figure two and this pairwise comparison and they analyze a couple of the different tasks here and what you would expect and I don't want to go too much into these tasks because honestly I also don't know all of these tasks. I don't know which tasks should share a lot of things, which ones shouldn't but it is a good way. Like it is a very smart way to investigate if the model really learns similar tasks to use similar information. Alright the last thing they do right here is the good and the bad subnetworks in BERT fine tuning. So they say our final experiment puts the above evidence of good subnetworks in BERT fine tuned from the perspective of lottery ticket hypothesis which predicts that the lucky subnetworks can be retrained from scratch to match the performance of the full network. To test this hypothesis we experiment with the following subnetworks. So that means I wasn't really sure when I read it the first time but now I'm fairly sure that all of the results so far were just pruning and maybe not retraining. So just sort of doing the pruning thing and not doing this lottery ticket retraining which shouldn't make a lot of the difference as we're going to see but just for the understanding. Because it seems like pruning and retraining doesn't do that much for the winning tickets as you'll see right now. But yeah so now they actually retrain from scratch. So good networks the elements selected from the full model by importance scores as described in the previous section. So here they're going to evaluate these good networks. First of all they're going to evaluate them pruned and they're going to evaluate them retrained in the lottery ticket style. Then they're also going to evaluate bad subnetworks. The elements sampled from those that did not survive the pruning plus a random sample of elements with high importance score so as to match the size of the good subnetworks. So because the good subnetworks are 50% or more of the network they want to sample from the things that did not survive so from the bad ones and they plus a random sample of the good subnetworks to just match the size. So we would expect these to perform maybe worse but maybe we can also train them to achieve good performance. And then they investigate bad subnetworks. Simple inversion of the good subnetworks. So these would be just anything but the good. They are 5 to 18% smaller in size than the sampled bad subnetworks but they do not contain any elements with high importance scores. And they say okay for all of them they evaluate their performance on all tasks simply after pruning and with fine tuning the same subnetwork with the same random seeds and with the rest of the model of masks. So this is really what the lottery ticket hypothesis does except they of course mask entire modules and not individual weights. And here you can see the general results. So the general results look like something like this. This is a typical example. So this is the, let's go out, oh yeah, this here is simply the dumb classifier that always tells the highest probability class. This is the, like this is sort of the idiot's baseline. Okay. This here is the full model. This here is the good pruned and this here is the good after it's retrained again. Okay. So you see by retraining you can basically gain. Now the original lottery ticket this would sometimes even go up here depending on how much was pruned but you can see that there is a slight gain after you retrain the pruned part. Okay. And the other thing to note here is that you don't lose much. Basically you only drop a little bit by pruning which that's what makes it the good part. You only drop a little bit. However, if you have the bad part which are these and let's say the good plus bad. These are the bad plus some of the good ones. You see that the performance drops pretty heavily almost to the baseline of the most frequent class and also here. So I would actually, I would go with this one right here if, because that's just the bad ones. You see the performance drops considerably but then, and that's what the authors claim is pretty interesting. If you retrain that part, the bad part, so to say, you can achieve sort of a very comparable performance to what you can achieve with the good parts. And this appears to be true for most of the results right here. There are some outliers like this one but there the score is also, so this is the Matthews correlation and not the accuracy. So the score is a bit different there. But you can see here the good plus bad also gets a fairly high accuracy. So the authors claim this is pretty surprising which I guess it is if you look at this. But what I want to do is I actually want, I have asked the author of the lottery ticket hypothesis this question. So this is from our machine learning street talk with Jonathan Franco and this is another channel that I am a part of and I would like to show you this right here when I ask them this question. Another question from Reddit, Imnemo asks, suppose you try to construct a lottery ticket by taking all the weights that were not part of a winning ticket and retraining from those, will that model be unable to learn the task or might there be another winning ticket hiding among them or one that was not originally used? So this is the most common question I get by people who read the original paper and I hope that by answering it here in a public forum I can answer it once and for all. The challenge in doing this experiment is let us take the MNIST example. So suppose that we find a winning ticket on MNIST. It is going to be about 3% of the original size of the network. So that means that if you remove it you still get 97% of the weights left. And so my guess is that if you were to train those 97% of weights you will get to the same accuracy as you got with 100% of weights because you have barely pruned the network at all. You could randomly prune by 3% and it would not affect it. And then you could go and find another lottery ticket that is mutually exclusive with the first. You still have 94% of the weights and you could probably iterate this for a very long time. You could probably this way find 10, 15 lottery tickets like this, maybe more, that are all mutually exclusive and still leave you with a remaining residual that is capable of training to full accuracy. So the challenge with this experiment is that the lottery tickets are small, which is great, but it means that whatever is left is large enough that I am sure there is another lottery ticket in there and another lottery ticket in there and so on and so on and so on. So it is an interesting idea in principle, but once you kind of look at the sizes of things you still got so much over-parameterization left that I think you just find more lottery tickets. You can even probably, I am guessing, swap out one weight from a lottery ticket with another weight and it would not matter or swap out a handful of weights. And so combinatorially the number of lottery tickets is massive and we are just finding one. All right, so as you saw this is kind of the most common question that Jonathan gets here. And as you can see the difference here of course is that our original tickets are already sort of 50% of the network, so what is left is only 50%. So this is substantially different. Now two things I have to remark here. First of all, because we are pruning modules and not individual weights for the good one, it is the reason that we do get these big winning tickets, right? But also what I think is happening is that because we are pruning these entire modules we are actually not fine-grained enough. So that means every time we eliminate a module we actually kill some good ones and some bad ones. So in here I am going to guess there are some good ones and there are some bad ones. But since we can only kill entire modules, we sort of, we simply kill the one that on average has the most good ones. But I am guessing that in the thing we kill there are simply, sorry, we kill the one that has on average the least good ones. But there are still some good weights in there. And if you believe the original lottery ticket hypothesis that means that these, actually these very few weights in those modules can still train to full accuracy. So actually what these authors claim is surprising in light of the original lottery ticket hypothesis. I think if you look at it from the perspective of the actual hypothesis which considers individual weight and a very small subset of them, the original hypothesis would pretty much predict that you could train something where you pruned away a bunch of modules entirely. Or you could train these bad modules because they are still going to contain a small-sized lottery ticket that is going to be responsible for the good performance. So that is kind of the first thing. And the second thing, in general, you heard Jonathan, I do not think that is actually even a question of the size of the tickets. Nothing in the original hypothesis forbids the non-winning ticket from also being trained to good accuracy. It simply says something about the winning ticket. It does not say anything about the non-winning ticket. So those are the two comments. And I think the question and the investigation, even though it is interesting, I think it is sort of maybe not thought through, at least in the perspective of what they go for here. The result is very interesting. But again, I think they claim the original hypothesis would sort of say these are the bad parts and you could not train them. And then they say it is surprising that you can. But I would say that the original hypothesis would in fact predict that you could train those things because you have pruned away these entire modules, which is very coarse-grained and that leaves still good weights in the bad parts. Okay. So they conclude. However, we can see that both good and bad networks can be retrained with comparable performance for many tasks. The inverted bad networks perform worse than the sampled ones, but that could be due to them being smaller in size. Performance of all inverted bad networks on call is almost zero. Okay. Yeah, okay. Very little remains when that mask is inverted. That is the task we looked at because they claim that is so small, which makes sense, right? So discussion. Say, does BERT have bad subnetworks? The key result of this study is that as far as fine-tuning is concerned, BERT does not seem to have bad subnetworks that cannot be retrained to relatively good performance level, suggesting that the weight that do not survive pruning are not just inactive. However, it is important to remember that we consider elements of BERT architecture as atomic units, while the original lottery ticket work relied on magnitude pruning of individual weights. So they're well aware here of these differences, and they can see which, and they can see to that right here. So that's good. On that level, BERT probably does have bad subnetworks, and they show that can be found in the transformer model with global iterative pruning. We'll leave it to future research to find out to what extent the effective subnetworks overlap with the effective architectural blocks, and what that says about the architecture of BERT and the other transformers. So as you see, they're well aware that all of what I said is the case. So it's not like I'm criticizing and saying they're wrong. It's just that if you read it, you sort of get the impression that this is what they're saying. And I think the light of which a reader goes through it is just a bit such that you come off, if you don't read until here, you come off thinking something else. Our results suggest that most architecture blocks of BERT are potentially usable in fine tuning. This should not be interpreted as proof that they all encode potentially irrelevant linguistic information. That's absolutely true. It is also possible that pre-training somehow simply made them more amenable to optimization, which is another question for future research. And they go into what do different BERT components do in the different things. So again, I think this work here is actually most relevant for investigating this question, what do BERT components, the different BERT components do for the different tasks, to look which tasks use which things. And the actual recognition that none of these modules is useless, I would consider pretty pretty cool finding. Okay, so in conclusion, they say prior work shows that it was possible to prune most self-attention ads. We extend this to the fully connected layers. We show fine tune purses good and bad top networks, where the good heads and MOPs alone reach performance comparable with the full network, and the bad ones do not perform well. However, this pattern does not quite conform to lottery ticket hypothesis. Both good and bad networks can be fine tuned separately to reach comparable performance. We also show that 86% of heads and 57% of MOPs and good sub networks are not universally useful. Cross-glue tasks and overlap between good and sub networks do not necessarily correspond to task types. So that's where we didn't go into. This raises questions about the degree to which fine tune BERT relies on task specific or general linguistic knowledge and opens up the possibility of studying the good sub networks to see what types of knowledge BERT actually relies on at inference time. So this is sort of future research direction. And with that, I think we've gone through the paper. I hope you got something useful out of this. I think it's a pretty cool paper. It's a pretty cool methodology, and I think a lot of work can build upon this to do interesting analysis of these language models. Again, if you like this video, consider sharing it, subscribing, liking, and bye bye.
 Hi there. Today we're looking at when BERT plays the lottery. All tickets are winning
 by Sy, Prasanna, Anna Rogers and Anna Rumschiski. So a high level overview of this paper is
 the following. The paper basically looks at BERT in terms of the lottery ticket hypothesis
 and it says that if you fine tune BERT on different downstream tasks, then the lottery
 ticket winners you're going to find are different between the tasks. And also the claim all
 tickets are winning refers to the fact that if you remove the winning tickets, then you
 can still train the rest to relatively good performance. Therefore, all tickets are winning,
 not just the sub network. So that was the high level overview for those of you who just
 want to be interested, if you want to continue watching this video. If you do like videos
 like this, consider sharing, liking, subscribing, telling your mother, father, brother and friends
 about it. Alright, let's dive in. So BERT, it is a language model. Basically, if you
 don't know what BERT is, I've done a video on BERT, but really quickly, what you can
 do with BERT is you can take a sentence, something like hello there, and you can put it through
 this multi-layer neural network. And what you'll get out is basically an embedding of
 that sentence. So a vector embedding of it. We'll make it really easy. And what is usually
 done is this is pre-trained on a task called masked language modeling. This is unsupervised
 training. And then you take this embedding and you fine tune. Basically, you put on a
 classifier head. Basically, say, let's take sentiment classification. So you have two
 output classes. And you want to say, is this sentence I put in positive or negative sentiment?
 So you would train this classifier by basically taking this part from the pre-trained masked
 language modeling and then training the part here that does the sentiment classification.
 You would sort of add that on top and then fine tune the entire network to solve this
 task. So that is basically BERT fine tuning on different tasks. And there is this benchmark
 called GLUE, where it has a number of tasks. In this case, I think they look at nine tasks
 of GLUE. It has nine of these tasks. One is the, an example is the sentiment classification.
 And it basically gets a score for each one. And thereby, you can sort of estimate how
 good your language model is by how well it is performing on each of those individual
 tasks. But the notable difference here, too, let's say a computer vision, like an ImageNet
 classifier is the fact that first it is pre-trained, right? This part here is pre-trained on a
 large corpus. And second, there are different downstream tasks that you fine tune on. So
 the second part that is important is the lottery ticket hypothesis. So I've also done a video
 on the lottery ticket hypothesis. And if you very quickly what the lottery ticket hypothesis
 is the following. So let's say you have an image classifier. And I have a bunch of layers
 in my neural network. I'm going to draw them like this. And at the end, right, I can classify
 it into like 10 different or a thousand different classes, whatever. And the input here is an
 image. So my neural network is going to have weights. So every one of these neurons is
 connected to each other. Now this can be a convolutional network or a MLP. So all is
 connected to all and as well here, right, everything is connected to pretty much everything.
 So we know, first of all, we know that we can train these big networks to relatively
 good accuracy. And then second of all, so first, we can train them. Second, we know
 we can prune them after training. What pruning means is the fact that after I have trained
 such a thing, I can then go and I can figure out which one which of these connections that
 I have learned are the important ones. And maybe I'll say, ah, these these here, actually
 these these five, I don't need more than those. Let's actually connect them to the end. These
 seven or so. I don't need all of the other ones. I just need those. And I can pretty
 much get the same accuracy as the full network. Now, the important part here is that you can
 only do pruning after you've trained a network. If you try to prune at the beginning, it doesn't
 work. So what the lottery ticket hypothesis says is basically, so how does training work?
 First you have your parameters. Let's let's tell them as a list. So each of these weights
 is an entry in the list here. First, you initialize these randomly, then through training, training,
 you get to your train state, right? You get each of the ones into your these are now trained.
 And in the train state, you can select the ones you think are important. The lottery
 ticket hypothesis says if I take those that are important and basically go back to the
 beginning, like here, here, and here, and I basically roll them back to that state that
 they were in when they were initialized. So I put the same random number there that I
 got at initialization. I can then make a network where I only have those. I can train that
 network and I can get a good accuracy. So this basically wasn't possible in the pruning
 framework because we said we can't prune, we can only prune after training because only
 then do we know which ones are the important ones. In fact, the lottery ticket hypothesis
 or the paper shows that you can train the smaller neural network from the beginning,
 but the catch of course is you have to know which ones those are and you have to know
 what value to set them on at the beginning. And you only know that after you've trained
 the full network. But still it kind of gives the, it tells you that you don't need all
 these connections for training. You basically only need so many connections such that somewhere
 in there, there's going to be the good ones, right? And if you knew the good ones from
 the beginning, you could just train those and then only train the smaller sub network.
 So that's the lottery ticket hypothesis. Naturally these connections here, this small network
 is called the winning, a winning ticket because if you knew what it was, you could basically
 train a much smaller network and reach the same accuracy. So this paper looks at BERT
 in terms of this lottery ticket hypothesis. Now it's a bit more complicated than just
 in these feed forward networks because BERT is not a feed forward network. BERT is a transformer.
 So what does that mean? A transformer consists of many layers and each layer, let's expand
 the layer here. So each layer consists of, let's go over there, need some space. So again,
 we have our layers of BERT and it goes, the signal goes like this. So each layer consists
 of two things. First of all, of many attention heads, that's called. Now I'm going to draw
 these as blocks right here. So four, let's call them four attention heads. Individual
 attention heads are all parameterized by individual matrices. And then on top of that, there is
 an MLP. So this is the multi-layer perceptron. This is one, basically a feed forward layer,
 residual actually. And so there is a skip connection right here. And these are the attention
 heads. Okay. And then the next layer would again have the same structure, four of these
 and then one of these and so on with the skip connection. All right. So the pruning in BERT
 is different than pruning in the feed forward or convolutional layer that we looked at.
 Pruning in BERT to what this paper looks at is pruning either an entire attention head
 like this. So kind of leaving out in the entire head away, which is, this is an entire matrix.
 This is not a single weight, right? This is many, many weights or even more drastically
 leaving away an entire MLP and basically only relying on the skip connection. All right.
 So this, you have these two things you can do. You can leave away heads or you can leave
 away entire MLPs or you can combine these things in some way. Right. So the notable
 difference here to the lottery ticket hypothesis pruning is the fact that here over up here,
 what we prune are connections. So prune connections, individual connections, individual connections.
 And here we prune entire modules. Now this is a, in my opinion, this is a qualitative
 difference, a very large qualitative difference actually. Why would you do this? So this paper
 basically doesn't invent this kind of pruning. They go after already existing literature.
 So what's the advantage in pruning modules? Well, you have to see what's the advantage
 in pruning per se. So in pruning, what you're trying to do is obtain a smaller network that
 gives you the same accuracy, but that you can run faster, right? That uses less memory
 and you can run it faster. And if you prune like this, like we did in the lottery ticket
 hypothesis paper, you don't really gain anything because if you have a matrix, if you have
 a matrix, matrix multiply, right? I have two matrices and I multiply them right here. If
 I cut out one weight here or one weight here, it doesn't help me because I have GPUs and
 those will parallelize these matrix procedures. And it doesn't really help me because we don't
 really have good hardware for sparse or matrix, matrix multiplication or matrix, matrix multiplication
 with holes or things like this. So it almost gains me nothing. The lottery ticket hypothesis
 paper is very much a kind of more of a scientific curiosity paper. And once we have sparse matrix
 multiply hardware, which I think already exists, but is not super widely distributed, once
 we have that, we will be able to make use of this. Whereas the people that prune BERT,
 so these are more, let's say, industry people. If you prune an entire module, well, that's
 an entire matrix that falls away. So I have to, I can basically save an entire matrix,
 matrix multiply in the forward pass here and the backward pass. Well, okay, I don't prune
 during training, but I can basically save an entire matrix multiply here by pruning
 an entire module. So I'm not sure if I were an author and I say I want to look at BERT
 in terms of the lottery ticket hypothesis, I would find a way, I would go away from this
 literature and find a way to also just prune here individual weights. It's not going to
 be faster, but the lottery ticket investigations aren't supposed to be faster, they're supposed
 to tell you something about the nature of the things you're investigating. And of course
 how you do this is simply by masking, right? You simply force these entries to be zero
 and therefore you don't have forward signal, you don't have gradient. Interestingly, they
 actually do the masking, but they do it on the whole entire module level. Okay, so this
 was BERT and the lottery ticket hypothesis and the all tickets are winning, we're going
 to investigate later. Let's see what they say in the abstract. Say much of the recent
 success in NLP is due to the large transformer based models such as BERT. Okay, they say
 however these models have been shown to be reducible to a smaller number of self-attention
 heads and layers. So this would be pruning. We consider this phenomenon from the perspective
 of the lottery ticket hypothesis. For fine-tuned BERT we show that, here's the contributions,
 A, it is possible to find a sub network of elements that achieves performance comparable
 with that of the full model. So basically this is the pruning objective, right? You
 want to prune it such that the performance holds and in terms of the lottery ticket hypothesis
 you want to prune, reset to the beginning and then also and then train again and that
 will give you, actually in the lottery ticket hypothesis you can gain performance if you
 prune by a certain amount. In this case here they always lose performance but yeah. So
 second of all similarly sized sub networks sampled from the rest of the model so the
 non-winning ticket perform worse. So if you just prune away the good parts then the bad
 parts perform worse of course. However the bad sub networks can be fine-tuned separately
 to achieve only slightly worse performance than the good ones indicating that most weights
 in the pre-trained BERT are potentially useful. So this is interesting. If they be fine-tuned
 separately this is exactly what the lottery ticket hypothesis is doing, right? It's basically
 fine-tuning only a sub part of the network and here they say even if we take the parts
 of the network that have low scores for pruning and we retrain those then we can achieve a
 good performance. So further they say we also show that the good sub networks vary considerably
 across glue tasks. This is this benchmark opening up the possibilities to learn what
 knowledge BERT actually uses at inference time. Alright so this is the overview of the
 paper. So a last thing to say which I've already kind of alluded to is the fact that in the
 original lottery ticket hypothesis as I said you had a graph and you had some sort of here
 was 100% accuracy and here was how much you prune. Of course you start at 100% if you
 prune nothing but then as you prune the interesting thing is it kind of goes up and then it goes
 down. So this is the first thing here it goes up to a certain amount if you don't prune
 and in the original lottery ticket hypothesis here somewhere here would be 50% of the network
 I think. And then once you go down let's say here to 90% of performance you are at something
 like 5% of the network size or 3%. So you can prune away most things and still be like
 extremely extremely powerful. Now we're going to see what these essentially what these people
 do here is here is 100% and they simply prune until they reach 90%. So we don't necessarily
 know what happens in the middle we just know they start here and somehow they get to 90%
 and what they end up with is something like 50% of the network still remaining. So again
 see the qualitative difference here between the 5% of the lottery tickets in the original
 paper and the 50-ish or so percent or considerable amount more in this paper right here and I'm
 pretty sure that is due to the fact that they prune entire modules here so they don't prune
 on a fine-grained enough level to investigate these phenomenon because as I said we don't
 know but I'm pretty sure this just goes down here and not up first. So qualitatively it
 seems different. Alright so here they introduce what they do again ERT is made up of these
 attention heads and MLPs. The MLPs have a skip connection as you can see here and the
 attention head attention layers are basically made up each of N of these attention heads.
 What they will do is they will look at 12 layer networks. Each layer will have 12 of
 these attention heads and one of the MLPs. So you have in total 144 heads and 12 MLP
 layers. The way they determine which ones to prune is pretty easy. In front of each
 attention head and in front of each MLP they put one of these binary variables right here.
 These variables can take values 0 or 1. If they are 0 the layers or the head is basically
 inactive, no propagation. If they are 1 they are active. And they determine what value
 to set them to by computing important scores. Basically determining how important is a head
 or a layer for the network. And that's pretty simple. You simply take the gradient of the
 loss. I think they go after this paper right here that's supposed to be the following.
 You derive the loss by these variables right here and therefore you get these important
 scores. And then you can simply prune the layers with the lowest important scores because
 that means that the gradient with respect to them is the smallest. That means your loss
 changes the least if you were to leave them away. So they here determine their pruning
 strategy. Their constraint here is as I said 90% of the performance of the full model.
 So they train the full model, fine tune the full model on this task and then they set
 themselves a budget of 90% and they simply prune until the model reaches 90%. Once it
 goes lower they stop. So they have three methods of pruning. One is heads only where they only
 cut away these attention heads. As I said there are 144 of them. They have the pruning
 strategy of MLPs only where they only prune the MLPs, leave all the attention heads alone.
 Then they have this heads and MLPs. They say we compute head and MLP important scores in
 a single backward pass, pruning 10% heads and one MLP with the smallest scores until
 the performance on the dev set is within 90%. Then we continue pruning heads alone and then
 MLPs alone. This I guess until again they are no longer in the 90% so until they reach
 their budget. So this is a combined strategy. This strategy results in a larger number of
 total components pruned within our performance threshold. So this is the thing we should
 focus on right because in pruning the name of the game is how much can you take away
 and still be within your budget. This strategy seems to be the viable strategy here.
 A last thing here is fine tuning. So the other difference between this paper and the lottery
 ticket hypothesis is that we said that in the original paper here these are randomly
 initialized weights. Like you train a class for an ImageNet or something, you start from
 randomly initialized weights and the lottery ticket papers they all kind of presuppose
 random initializations. Whereas BERT, when you do the same thing for BERT, these are
 not random initializations. We said in BERT what you usually do is you train the encoder
 part here. You pre-train with masked language modeling first and then second you train the
 entire thing. Let's skip the color here. Second you train the entire thing. You fine tune
 the entire thing. So if we talk about initializations in the BERT task then the initialization would
 be at this point right here after the masked language modeling would be the initialization.
 So the weights are not random. The weights are actually pre-trained on the masked language
 modeling task which is also a qualitative difference and sort of lets us inspect. So
 the authors say that since we trained with masked language modeling and people sort of
 claim that masked language modeling learned something about the language, we can now investigate
 kind of which attention heads, which modules in BERT are encoding which parts of the language.
 And this is going to be interesting once we look at which attention heads and which modules
 survive in the individual tasks, we can sort of compare tasks across each other by seeing
 which of the heads they share in their winning tickets. Alright, so they produce these graphs
 here. These are sort of one of the central graphs here and the way to read this is on
 the left side here you have the layer, you have the layer index and on the x-axis you
 simply have the index of the head. There are 144 boxes here. Each one corresponds to one
 of the attention heads. The top number is always the mean number of glue tasks that
 this head survived in. So what they do is they take the pre-trained BERT, they fine
 tune it on these nine tasks and for each of the nine tasks they determine the winning
 tickets. And the number here says how many, in how many of these nine tasks is this particular
 attention head a part of the winning ticket. Now they repeat it for different random seats,
 that's why you have floating point numbers and the lower part is the standard deviation
 across that. So you can see quite a number of heads make it into a lot of these tasks.
 So you can say this part, this thing right here, red on red, this head right here survives
 in seven out of the nine tasks. So it should be fairly, it should probably encode something
 fairly substantial about language that is shared across these seven tasks. You can see
 some of the heads like this one here doesn't survive in almost any task which basically
 means that it's, you know, that one's not really super important for these tasks. It
 might have been, you know, important for the pre-training but not for these particular
 tasks. What's interesting, so what you can see is that the mean or median or so is like
 three, four or five. And that means that a lot of the heads are sort of somewhat important
 for some of the tasks. And you can see the qualitative difference. If this were the like
 original lottery ticket paper, most of these numbers would be at zero because the lottery
 ticket size is just so much smaller. Here you can directly see that you are going to
 retain a large number of things in your network in order to get 90% of the performance. And
 that's probably because you prune entire modules again. So they have this for two variants
 here. First for the strategy of masking heads only. And the right one is for masking heads
 and MLPs. And the same here on the bottom. These are the same numbers but not for attention
 heads but for MLP layers. And you see again this is masking MLPs only. This is masking
 heads and MLPs. So if you compare the two, you see that for example this here and this
 here are substantially darker which means more of this stuff survives. Now we can't
 really... It seems like here for example, it's darker than here. So on the right side
 more stuff survives but also you have more things to prune, right? You can prune the
 heads and the MLPs. And they claim before that the masking heads and MLP strategy results
 in more things being pruned which isn't really congruent with here generally more things
 surviving. But it could be because of the fact maybe the sum of the two is still lower
 than the sum of each individual thing here. Though it doesn't really look like it. So
 I'm a bit confused about this but I'm just going to assume that the sum of the two is
 lower. Does that make sense if both are darker? Well it shouldn't be the sum... It should
 be the sum of this plus a completely dark this in terms of masking heads only or vice
 versa versus the sum of those two, right? So that should be the measure. But it just seems
 a bit doesn't work out too much. But okay that's what they say. So by the way if the
 authors are here you have... This is cut off. Haha. Yeah this is annoying. This is like trying
 to get LoTeC to do things and it doesn't comply. Alright so what you can... Another thing you
 can see in the authors point out here is that if you mask heads and MLPs you sort of shift
 more things to the back of the network to the higher up layers. And they reason now
 because you also mask the heads basically they can't do as much work so the heads would
 be masked somewhere here. So all that work is going to shift upon the MLPs and mostly
 to the back of the network because this thing here cannot take over work that this attention
 head here is now not performing anymore because it was pruned because the signal travels this
 way. So the authors kind of interpret these results right here and I think the most important
 thing to see is simply the variance of things. So most heads are actually important for at
 least two or three tasks and no head is important for all the tasks consistently. I think that's
 the take home message right here. Okay and they contrast this to previous research that
 has basically said this experiment falls up on a study by this that showed that only a
 few transformer heads in machine translation tasks did the heavy lifting while the rest
 could be pruned. And this paper similarly showed that most of BERT self-attention head
 in MNLI task could be pruned and that the good heads were mostly shared between the
 MNLI matched and mismatched. And they basically say yeah that's correct but that is only within
 one task right if you go beyond if you go to several tasks then the heads that are important
 differ quite a bit. Okay so let's continue and go here. They ask how task independent
 are the good self-nerve works and they basically look at these kinds of graphs right here which
 are pretty interesting. So we've got this. This is heads shared between tasks. So what
 this measures is these are the different tasks in the glue benchmark and they basically look
 at each task look at its winning lottery ticket and look at which heads survive in the winning
 ticket. And then they put that here on the diagonal. So if in QNLI a head survives it
 gets a 1 here and if it doesn't survive it gets a 0. So on average 85 out of the 144
 heads survive right 85 heads survive. That's pretty as I said this is somewhat like over
 50% of the network. It's entirely different than the original lottery ticket hypothesis
 paper. So 85% not percent 85 of the 144 heads survive. Now they look at the other tasks
 so for QNLI they would look at maybe MNLI task here and ask which of the heads that
 survived in QNLI also survives in MNLI. So that gets you the shared heads and again the
 lower numbers is standard deviation. So 62 heads survive in QNLI and MNLI and the authors
 here are sort of arguing that from these sort of numbers you should be able to see which
 of the tasks share different linguistic knowledge. So different linguistic knowledge could be
 relevant for different tasks but if some tasks share a lot of the attention heads that survive
 in the winning tickets that basically means that the model is using that information that
 is in that head for both tasks. This could be good in that you say oh yeah these tasks
 really are used similar linguistic features or it could be something that you don't expect
 and then you might be able to investigate maybe the model is doing something shady here
 because it really shouldn't, these tasks don't really have much in common. So they do this
 for the heads and the MLPs here. Now if you ask why the WNLI here has a bunch of zeros
 that's because it's a wonky task and basically the best thing you can do is predict the most
 frequent class. So you can prune just about anything away on these MLPs because they have
 the skip connections you don't need them to predict the most frequent class. What I want
 to go into is the following statement right here. So, note that figure one, so the figure
 before, shows very few heads or MLPs that are universally useless. Only seven heads
 that survived in less than two tasks. 86% of heads and 67% of MLPs survived in two to
 seven tasks with relatively high standard deviation. They say this means that the good
 sub networks for different tasks have relatively little in common. So they make this sort of
 statement again here that the good sub network have little in common and it might seem like
 that for the figure initially. But if you look at this figure it actually shows something
 pretty interesting I think. So if you look at a number, let's say for example this here,
 this 74 and I haven't actually tried. Yeah let's look at the 74 and this here. So let's
 look at these tasks QQP and RTE. Okay, so if you look at QQP and RTE you could see that
 these are tasks that already they don't have a lot of heads in common right and you might
 be able to say well if what they're saying is true that the tasks share relatively little
 you would expect them to be relatively independent. But if I look at this 78 here it means that
 78 out of 144 heads survive and here it means that 74 out of 144 heads survive. So if I
 now would think that okay generally for different tasks things are different how many heads
 would I expect there to be surviving in both if the tasks are independent. So that's these
 two things multiplied right times 144. So I can scratch this here and the 7 times 7 is
 whatever 49. Let's go 7 times 8 about this so that's 5, 6. Do I need to get out the calculator?
 I want to do this calculator. I'm going to do this the right way. Okay I hope you can
 see that so that's 78 times 74 divided by 144. Did I do it right? I probably did it
 wrong. 78 times 74 divided by 144. Alright so that's 40 heads and you see that there's
 43 heads and I've actually gone through a bunch of these numbers before not these ones
 but generally the shared number of heads is higher than what one would expect if you assume
 that the tasks are independent. And I'm sort of missing sort of an analysis of that here
 because that I find to be a pretty interesting finding of these things and sort of I mean
 I get the fact that they say based on the graphics up here that the tasks are sort of
 seem to be relatively independent with respect to the heads that survive and of course relatively
 independent is a relative term but it's sort of an investigation into why we see considerable
 difference between tasks here in terms of that. So these numbers are always over what
 you would assume for independence. That seems to be pretty interesting. Alright so they
 say they here go into this figure two and this pairwise comparison and they analyze
 a couple of the different tasks here and what you would expect and I don't want to go too
 much into these tasks because honestly I also don't know all of these tasks. I don't know
 which tasks should share a lot of things, which ones shouldn't but it is a good way.
 Like it is a very smart way to investigate if the model really learns similar tasks to
 use similar information. Alright the last thing they do right here is the good and the
 bad subnetworks in BERT fine tuning. So they say our final experiment puts the above evidence
 of good subnetworks in BERT fine tuned from the perspective of lottery ticket hypothesis
 which predicts that the lucky subnetworks can be retrained from scratch to match the
 performance of the full network. To test this hypothesis we experiment with the following
 subnetworks. So that means I wasn't really sure when I read it the first time but now
 I'm fairly sure that all of the results so far were just pruning and maybe not retraining.
 So just sort of doing the pruning thing and not doing this lottery ticket retraining which
 shouldn't make a lot of the difference as we're going to see but just for the understanding.
 Because it seems like pruning and retraining doesn't do that much for the winning tickets
 as you'll see right now. But yeah so now they actually retrain from scratch. So good networks
 the elements selected from the full model by importance scores as described in the previous
 section. So here they're going to evaluate these good networks. First of all they're
 going to evaluate them pruned and they're going to evaluate them retrained in the lottery
 ticket style. Then they're also going to evaluate bad subnetworks. The elements sampled from
 those that did not survive the pruning plus a random sample of elements with high importance
 score so as to match the size of the good subnetworks. So because the good subnetworks
 are 50% or more of the network they want to sample from the things that did not survive
 so from the bad ones and they plus a random sample of the good subnetworks to just match
 the size. So we would expect these to perform maybe worse but maybe we can also train them
 to achieve good performance. And then they investigate bad subnetworks. Simple inversion
 of the good subnetworks. So these would be just anything but the good. They are 5 to
 18% smaller in size than the sampled bad subnetworks but they do not contain any elements with
 high importance scores. And they say okay for all of them they evaluate their performance
 on all tasks simply after pruning and with fine tuning the same subnetwork with the same
 random seeds and with the rest of the model of masks. So this is really what the lottery
 ticket hypothesis does except they of course mask entire modules and not individual weights.
 And here you can see the general results. So the general results look like something
 like this. This is a typical example. So this is the, let's go out, oh yeah, this here is
 simply the dumb classifier that always tells the highest probability class. This is the,
 like this is sort of the idiot's baseline. Okay. This here is the full model. This here
 is the good pruned and this here is the good after it's retrained again. Okay. So you see
 by retraining you can basically gain. Now the original lottery ticket this would sometimes
 even go up here depending on how much was pruned but you can see that there is a slight
 gain after you retrain the pruned part. Okay. And the other thing to note here is that you
 don't lose much. Basically you only drop a little bit by pruning which that's what makes
 it the good part. You only drop a little bit. However, if you have the bad part which are
 these and let's say the good plus bad. These are the bad plus some of the good ones. You
 see that the performance drops pretty heavily almost to the baseline of the most frequent
 class and also here. So I would actually, I would go with this one right here if, because
 that's just the bad ones. You see the performance drops considerably but then, and that's what
 the authors claim is pretty interesting. If you retrain that part, the bad part, so to
 say, you can achieve sort of a very comparable performance to what you can achieve with the
 good parts. And this appears to be true for most of the results right here. There are
 some outliers like this one but there the score is also, so this is the Matthews correlation
 and not the accuracy. So the score is a bit different there. But you can see here the
 good plus bad also gets a fairly high accuracy. So the authors claim this is pretty surprising
 which I guess it is if you look at this. But what I want to do is I actually want, I have
 asked the author of the lottery ticket hypothesis this question. So this is from our machine
 learning street talk with Jonathan Franco and this is another channel that I am a part
 of and I would like to show you this right here when I ask them this question.
 Another question from Reddit, Imnemo asks, suppose you try to construct a lottery ticket
 by taking all the weights that were not part of a winning ticket and retraining from those,
 will that model be unable to learn the task or might there be another winning ticket hiding
 among them or one that was not originally used?
 So this is the most common question I get by people who read the original paper and
 I hope that by answering it here in a public forum I can answer it once and for all. The
 challenge in doing this experiment is let us take the MNIST example. So suppose that
 we find a winning ticket on MNIST. It is going to be about 3% of the original size of the
 network. So that means that if you remove it you still get 97% of the weights left.
 And so my guess is that if you were to train those 97% of weights you will get to the same
 accuracy as you got with 100% of weights because you have barely pruned the network at all.
 You could randomly prune by 3% and it would not affect it. And then you could go and find
 another lottery ticket that is mutually exclusive with the first. You still have 94% of the
 weights and you could probably iterate this for a very long time. You could probably this
 way find 10, 15 lottery tickets like this, maybe more, that are all mutually exclusive
 and still leave you with a remaining residual that is capable of training to full accuracy.
 So the challenge with this experiment is that the lottery tickets are small, which is great,
 but it means that whatever is left is large enough that I am sure there is another lottery
 ticket in there and another lottery ticket in there and so on and so on and so on. So
 it is an interesting idea in principle, but once you kind of look at the sizes of things
 you still got so much over-parameterization left that I think you just find more lottery
 tickets. You can even probably, I am guessing, swap out one weight from a lottery ticket
 with another weight and it would not matter or swap out a handful of weights. And so combinatorially
 the number of lottery tickets is massive and we are just finding one.
 All right, so as you saw this is kind of the most common question that Jonathan gets here.
 And as you can see the difference here of course is that our original tickets are already
 sort of 50% of the network, so what is left is only 50%. So this is substantially different.
 Now two things I have to remark here. First of all, because we are pruning modules and
 not individual weights for the good one, it is the reason that we do get these big winning
 tickets, right? But also what I think is happening is that because we are pruning these entire
 modules we are actually not fine-grained enough. So that means every time we eliminate a module
 we actually kill some good ones and some bad ones. So in here I am going to guess there
 are some good ones and there are some bad ones. But since we can only kill entire modules,
 we sort of, we simply kill the one that on average has the most good ones. But I am guessing
 that in the thing we kill there are simply, sorry, we kill the one that has on average
 the least good ones. But there are still some good weights in there. And if you believe
 the original lottery ticket hypothesis that means that these, actually these very few
 weights in those modules can still train to full accuracy. So actually what these authors
 claim is surprising in light of the original lottery ticket hypothesis. I think if you
 look at it from the perspective of the actual hypothesis which considers individual weight
 and a very small subset of them, the original hypothesis would pretty much predict that
 you could train something where you pruned away a bunch of modules entirely. Or you could
 train these bad modules because they are still going to contain a small-sized lottery ticket
 that is going to be responsible for the good performance. So that is kind of the first
 thing. And the second thing, in general, you heard Jonathan, I do not think that is actually
 even a question of the size of the tickets. Nothing in the original hypothesis forbids
 the non-winning ticket from also being trained to good accuracy. It simply says something
 about the winning ticket. It does not say anything about the non-winning ticket. So
 those are the two comments. And I think the question and the investigation, even though
 it is interesting, I think it is sort of maybe not thought through, at least in the perspective
 of what they go for here. The result is very interesting. But again, I think they claim
 the original hypothesis would sort of say these are the bad parts and you could not
 train them. And then they say it is surprising that you can. But I would say that the original
 hypothesis would in fact predict that you could train those things because you have
 pruned away these entire modules, which is very coarse-grained and that leaves still
 good weights in the bad parts. Okay. So they conclude. However, we can see that both good
 and bad networks can be retrained with comparable performance for many tasks. The inverted bad
 networks perform worse than the sampled ones, but that could be due to them being smaller
 in size. Performance of all inverted bad networks on call is almost zero. Okay. Yeah, okay. Very
 little remains when that mask is inverted. That is the task we looked at because they
 claim that is so small, which makes sense, right? So discussion. Say, does BERT have
 bad subnetworks? The key result of this study is that as far as fine-tuning is concerned,
 BERT does not seem to have bad subnetworks that cannot be retrained to relatively good
 performance level, suggesting that the weight that do not survive pruning are not just inactive.
 However, it is important to remember that we consider elements of BERT architecture
 as atomic units, while the original lottery ticket work relied on magnitude pruning of
 individual weights. So they're well aware here of these differences, and they can see
 which, and they can see to that right here. So that's good. On that level, BERT probably
 does have bad subnetworks, and they show that can be found in the transformer model with global
 iterative pruning. We'll leave it to future research to find out to what extent the effective
 subnetworks overlap with the effective architectural blocks, and what that says about the architecture
 of BERT and the other transformers. So as you see, they're well aware that all of what I
 said is the case. So it's not like I'm criticizing and saying they're wrong. It's just that if you
 read it, you sort of get the impression that this is what they're saying. And I think the light of
 which a reader goes through it is just a bit such that you come off, if you don't read until here,
 you come off thinking something else. Our results suggest that most architecture blocks of BERT are
 potentially usable in fine tuning. This should not be interpreted as proof that they all encode
 potentially irrelevant linguistic information. That's absolutely true. It is also possible that
 pre-training somehow simply made them more amenable to optimization, which is another question for
 future research. And they go into what do different BERT components do in the different things. So
 again, I think this work here is actually most relevant for investigating this question, what do
 BERT components, the different BERT components do for the different tasks, to look which tasks use
 which things. And the actual recognition that none of these modules is useless, I would consider pretty
 pretty cool finding. Okay, so in conclusion, they say prior work shows that it was possible to prune
 most self-attention ads. We extend this to the fully connected layers. We show fine tune purses
 good and bad top networks, where the good heads and MOPs alone reach performance comparable with
 the full network, and the bad ones do not perform well. However, this pattern does not quite conform
 to lottery ticket hypothesis. Both good and bad networks can be fine tuned separately to reach
 comparable performance. We also show that 86% of heads and 57% of MOPs and good sub networks are
 not universally useful. Cross-glue tasks and overlap between good and sub networks do not
 necessarily correspond to task types. So that's where we didn't go into. This raises questions
 about the degree to which fine tune BERT relies on task specific or general linguistic knowledge
 and opens up the possibility of studying the good sub networks to see what types of knowledge BERT
 actually relies on at inference time. So this is sort of future research direction. And with that,
 I think we've gone through the paper. I hope you got something useful out of this. I think it's a
 pretty cool paper. It's a pretty cool methodology, and I think a lot of work can build upon this to
 do interesting analysis of these language models. Again, if you like this video, consider sharing it,
 subscribing, liking, and bye bye.