Movement Pruning: Adaptive Sparsity by Fine-Tuning (Paper Explained)
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper prune pruning transfer learning weights magnitude gradient moving small importance huggingface nlp natural language processing squad mnli bert transformer attention cnn distillation teacher sparse sparsity question answering mobile edge tune fine-tune
Deep neural networks are large models and pruning has become an important part of ML product pipelines, making models small while keeping their performance high. However, the classic pruning method, Magnitude Pruning, is suboptimal in models that are obtained by transfer learning. This paper proposes a solution, called Movement Pruning and shows its superior performance.

OUTLINE:
0:00 - Intro & High-Level Overview
0:55 - Magnitude Pruning
4:25 - Transfer Learning
7:25 - The Problem with Magnitude Pruning in Transfer Learning
9:20 - Movement Pruning
22:20 - Experiments
24:20 - Improvements via Distillation
26:40 - Analysis of the Learned Weights

Paper: https://arxiv.org/abs/2005.07683
Code: https://github.com/huggingface/transformers/tree/master/examples/movement-pruning

Abstract:
Magnitude pruning is a widely used strategy for reducing model size in pure supervised learning; however, it is less effective in the transfer learning regime that has become standard for state-of-the-art natural language processing applications. We propose the use of movement pruning, a simple, deterministic first-order weight pruning method that is more adaptive to pretrained model fine-tuning. We give mathematical foundations to the method and compare it to existing zeroth- and first-order pruning methods. Experiments show that when pruning large pretrained language models, movement pruning shows significant improvements in high-sparsity regimes. When combined with distillation, the approach achieves minimal accuracy loss with down to only 3% of the model parameters.

Authors: Victor Sanh, Thomas Wolf, Alexander M. Rush

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
 Hi there, today we're looking at movement pruning, adaptive sparsity by fine tuning by Victor Sun, Thomas Wolff and Alexander M. Rush of Hugging Face and Cornell University. On a high level, this paper proposes that if you have a transfer learning objective and you want to do pruning, you should not do pruning by weight magnitude, you should do pruning by how much the weights move during the transfer learning. This yields better results in the very sparse model regimes and is specifically relevant to current NLP transfer learning tasks such as BERT models. So if you like content like this, consider subscribing and sharing it to your friends and as always leave a comment if you have anything to say on this. Alright let's dive in. So they say magnitude pruning is a widely used strategy for reducing model size in pure supervised learning. So what is magnitude pruning? Now if I have a neural network, let's say I have a convolutional neural network and I input my little cat right here and I have a bunch of layers right and now if we look at these layers, each of these layers is going to be made up of these units of the neurons and the next layer is also made up of these neurons. Now what kind of neural network that is, it's not that important but what is important is that you have these connections from neuron to neuron and in let's say a fully connected network every neuron is connected to every other neuron. In a CNN that would be slightly different but in essence you have a lot of connections here and these are usually called weights. So these are the weights. Now the problem is if I train like these giant neural networks and I want to ship them for example to mobile devices to my customers then they won't be able to download gigabytes of models or even like hundreds of megabytes of models, it's just not possible. So what we want to do is we want to prune this model which means we want to remove parts of these weights, a lot of these weights but we don't want to lose accuracy of the network. So imagine I have a network and that's trained, it's an image classifier, it's here it's cats or dogs and I have it trained to a good accuracy. I want to delete these weights but I want to retain the performance and these methods are called pruning. Now what people do is usually they sort of go in stepwise fashion, they say well first of all I don't need some of these and then they delete some and then they sort of retrain the pruned network and after that they go again and they say well I don't really need that one and they don't really need that one so they do it in this stepwise fashion until the network is of the size that they want and the hope is that you don't lose too much accuracy. So the question is how do you select which weights you need and which ones you don't need and usually this is done by so called magnitude pruning which means that you look at the weights and the weights they'll have some distribution, there will be very negative so here is very negative weights and here is very large positive weights and what you'll say is that okay probably the weights that are very large they contribute a lot to the signal of the network within the network and the weights that are quite small they're you know since there's all this noise and stuff they're probably not that important so I'm going to cut off basically right here and everything that's in here I'm going to delete those are the non-important weights whereas on the outside those are the important weights. This is called magnitude pruning because it goes by the magnitude of the weight the absolute value of the weight. So you don't actually need so there's not one threshold here you don't need a threshold you simply need a method to order the weights right and then you keep removing them until you're satisfied with the size. So this is magnitude pruning. Now what's the problem with the magnitude pruning in these kinds of tasks? They say however it is less effective in the transfer learning regime that has become standard for state-of-the-art natural language processing applications. So what do you do in these transfer learning regimes? In the transfer learning regime and actually let's go with the image example right here even though it's mostly used in NLP we can do the same thing. So let's say we have a classifier here for cats and dogs our classifier and we had a big big database of cats and dogs images right so we were able to train that fairly well and we don't prune it yet we have this full network. Now we want to adapt this to a task where we want to recognize whether or not the animal is sick. So we developed this app for a veterinarian and it's like a short screening for a particular disease that a cat might have and we already have this cats and dogs classifier so it's reasonable to assume that this classifier has some good features to work with cats and dog images. So what we can do instead of because let's assume for this other task we just have this tiny little data set which is not enough to train a neural network of this size right but so in the first step we'll train this big neural network on the cats versus dogs and then we what we do is we transfer learning so we transfer all the weights right here and here we have a different task now sick or not sick right this is cat this is dog and here is sick or not sick not sick and of course we can't transfer these particular weights but we hope that the features here will sort of be the same so we transfer them and then we train these weights including the head right here this part we train it on this little data set and we hope that we already have this good starting point we only need to you know learn the basically the specifics of what makes these two data sets different and we won't have to learn entire task of dealing with cat and dog images from the get-go okay so this is called transfer learning now in this case we combine the two so first we want to transfer learn like if we build this app for vets and then we might say oh this is not you know this is not only for vets this is actually for anyone you know who has a cat or a dog at home so what we could do is build an app where anyone at home could scan their their cat and it would output like a probability of the cat having that disease so we want this neural network is still the same size as this neural network so now we want to do the pruning we want this neural network to become sparse to only have a couple of connections left such that it's a few kilobytes large but retain performance now they say when you do this step you can't just do the magnitude pruning like you did right here and why not because this is not this model right here is not the result of a training step like of a regular training process but is the result of a transfer learning process where first you do the big training and then second you adapt it and why is that the case well ultimately what you want to do is you want to prove the non-important weights now there could be a weight right here this one that is very important for the cat versus dog task but that is not important for the sick versus non-sick task and we also we know that in these transfer learning settings the weights they don't tend to move that much in general the research shows that once you've trained a neural network basically the beginning is important but then once you did it like if you adapt it or transfer learn it and so on the weights they won't move that much so in essence this weight maybe starts out right here and it will sort of stay around this place it will maybe go a little bit down because it's not important but it won't move much during transfer learning that's just a property of transfer learning so this paper here says we can't just use magnitude pruning when we transfer learn because what will basically go by what will basically say is the will assign the importance based on based on the original neural network task on the cat versus dog we we will miss specify the importance of the weights what we should do is actually measure the importance with respect to this task and how do they achieve it so on a high level they're basically saying okay if we start out well this was fatal if we start out with a point over here let's make that red red i want the color red well it's blue now so if we ah if we start out with a point over here what we should do what we should do is we should observe how it moves during transfer learning if it moves towards zero then it's probably not that important for the new task and if it moves to the to be even larger then it's probably important for that new task okay so that's that's a high level now how do you measure how it moves and what exactly how exactly do you do all of this during training such that you don't make mistakes that's the point of this paper they say we propose movement pruning a simple deterministic first-order weight pruning method that is more adaptive to pre-trained model fine-tuning we give mathematical foundations to the method and compare it to existing zero and first-order pruning methods okay so um yeah we said so that's basically on a high level that's that now how do they actually do it what they do right here is the following they say what we can define we can define each each network layer basically as a matrix multiplication by a weight now you can express pretty much any neural network as such a multiplication with a weight so you have x the in the signal in each layer and you multiply that by the weight matrix w now if you prune the neural network you can see that right here what you're saying is i basically in here i have the matrix m which is a mask so the mask is either zero or one for if a weight is active or if a weight is not active now this is not a matrix multiply actually this is like a hadamard product um but you have this mask matrix and what decides on this mask this mask is decided as you can see right here by this s so s s is a matrix that for each entry in w it will decide how important it is now in the classic sense in the magnitude pruning you already saw that this is just going to be the absolute value of w i j okay and then the top v simply means that you take the whoever are the most important the most magnitude that those are going to be one in the mask and everything else is going to be zero in the mask okay that's how this s this the w determines the s and the s determines the m okay so what you ultimately use is the m right here um but in now what we want to do is we want to actually make the s based on the movement and the movement is not really a defined concept because it goes over steps and so on so how do you do the movement in in a kind of dynamic way and this paper says you should do it by gradient so um you you should observe the gradient of your loss function with respect to this s matrix to this importance matrix right what does it mean so the what does it mean it means let's consider this quantity right here if s is the importance of a particular connection and if the gradient is large that means this this connection moves a lot right it like the loss pulls it into a particular direction okay so we're not talking about yet which direction actually the gradient has a sign it's either positive or negative right so by this quantity you can decide how much does this new task want this particular importance score to move so this is a direct direct measure of how much basically the loss function pulls on that importance score how much and now you can simply decide if and they have these they have i think they have a diagram yes so but i don't like that let's go so we have right here we have what's the value of this gradient of l with respect to s and here is w so if the gradient is positive and w is already positive that means the gradient goes into the positive direction so you increase the loss function in that let's put the negative gradient here because you do gradient descent right so so if the negative gradient is positive and the weight is already positive in in this case that means you're all the weight is already high but now the loss function wants to push it even higher so that must be a very very important weight right like it's like very good the same goes if the gradient the negative gradient is negative and the weight is already negative the weight being negative already means the weight you know it has a negative sign and then the gradient wants it to go even more negative the the optimization procedure says this thing should become even more negative and also we say that's probably a good weight now the other two cases means basically the weight's already positive but the gradient wants it to go negative which means it's pulled towards zero now it's entirely possible that it's going to cross zero and go like if you're here going from over here gonna here cross zero and become like super large but that violates our basic assumptions that the transfer learning doesn't move the weights too much right what you're caring for is basically this local neighborhood right here okay so you can make the fair assumption that these weights are not that important in the case where the negative gradient goes against the sign of the weight so this this is of course discrete right now but we can actually assign a number by how large the gradient is and by how large the weight already is and therefore we can make a score so the important score right here as you can see is the weight multiplied by the gradient of the weight and they can actually show mathematically that if you do this over multiple steps so you optimize while you do this pruning and they do some sort of a soft pruning so you can kind of correct your mistakes later on I mean they have hard and soft pruning but in any case they can correct their mistakes later on this will actually result in these important scores being an accumulation over the training over the entire training of this quantity and that's pretty cool because that means eventually you'll sort of have a consistent estimator of these important scores across your training procedure okay because the main fear with something like this of course is that it's very brittle and very much depends on the training dynamics and who knows if in step one something bad happens and so on but the the math behind this here gives sort of more evidence that this it can be like a self-correcting mechanism it is actually not too dependent on the particular training dynamics so they do this experimental setup now they have some they have some quirks here actually let's first go to the the the actual different methods they compare different methods right here where they say okay there's magnitude pruning it's a zero with order which basically just means you just look at the weight magnitude that that's it it's top v which means you just pick the top whatever and the objective is just the loss and the scores are just the absolute value we've seen this now movement pruning on the other hand is first order which means you look at the movement in our case the gradient as you can see here those are the importance scores and you use this straight through estimator which is basically just a way of saying that even though you're masking some things in the forward step you shouldn't mask them in the in the gradient backward step because you still want gradient signal to reach so if you have layers and you have a weight right here at least that's how I understand it I have not read that paper but if you mask this one here you still want the gradient to sort of flow backwards because you still need the actual importance scores for the weights that here below connect to this weight I think that's what is meant not entirely sure in this though so but you can see that the objective function is also the actual loss function now this is contrasted to a baseline called l0 regularization which is quite similar but is also first order but has this sort of regularizer right here and uses the gumball softmax in order to determine the scores and as you can see it also has a different score function and it has this continuous hard concrete importance masking function sorry masking function and they have a variant of movement pruning that tends to perform a little bit better which is soft movement pruning where instead of just going by the loss function optimizing the loss function they optimize the loss function plus something now they have as you can see here they have a thresholding masking function and the threshold is actually dynamic or it's determined by the importance scores and they have then a regularizer that make the importance scores sparse so instead of saying we just want the top 5% of weights they now just put a lot of mass on this lambda right here which will cause the s to be sparse and you know they think if they are not happy with how many weights they can simply increase or decrease this lambda such that they get to their desired sparsity of course you see there's the direct trade-off with the loss function right so the more you put weight on this lambda the less weight is you put basically on the loss function itself so you have the trade-off here is very explicit whereas in basic movement pruning it's just given by you masking away completely the bottom 1-v of percent of the weights but you see the score function is the same oh well the score function here the score function is the same okay now there are quite a number of tricks here like there's this sparsity scheduling function and so on so as always in NLP and with any big models there are a bunch of engineering tricks that make everything work better and you can never exactly tell how much is due to that and how much is due to the the actual technique but if you know you can sort of assess whether or not these it's done well and this here actually the rationale makes sense and that's why I tend to think that it is actually a better method and the experiments are very very convincing let's say okay this is just a pictorial comparison where you can see movement pruning sorry magnitude pruning all it does is it looks at after you fine-tune what are the weights and it just cuts away everything in the middle doesn't care about how the weights were when before however movement pruning looks at the combination of what were the weights before and what are the weights now and it cuts away everything where the weights moved towards zero which are these quadrants right here and it leaves in everything where it moved away from zero or that's that's the ordering let's say that how much it moved okay experiments now as you might already have figured out by now in the machine learning and especially the NLP community the methods presented always outperform the previous methods in this case it's pretty special so they test this on these number of tasks quad and NLMI and QQP and these are quite hard tasks from an NLP perspective like squad is question answering MNLIs like language inference so these would be on the I would guess these are on the foreign NLP system on the harder side of tasks that's it's fairly cool and as you can see here now just focus on first of all focus on this MAP which is the magnitude pruning so that's the baseline if you will and the purple one the SMVP which is the soft movement pruning okay now you can also focus on the MVP right here but they're approximately the same now the RPP you can maybe see in the graph is performing fairly well even compared to the full model it's another baseline basically but we just want to compare those to and you can see that in this regime that the magnitude pruning outperforms the movement pruning but however in this regime the movement pruning is much better and that's the percent where the percent of remaining weights is very very low so this is kind of the extreme sparse case where you only have 10% or you only have 3% of the weights left and you can see that the movement pruning is outperforming the magnitude pruning by a lot okay now they do discover that so this this happens in all in all of these tasks as you can see right here and they do they do discover that if you then distill the model further so in distillation this is yet another technique that you can use to boost the performance of the transfer learned model so in distillation you would not only train the model on you so you have you now you have your this model that you transfer learn and you have the pruned version and in the pruned version what you would do is you would simply also train it on this data set but what you can also do is you can distill this model right here the one that you trained on the same task right that's that's presumably better because it still has all the weights okay you can run a data point through both so the same data point goes through this and you get an output which are logits which is like representing a distribution saying yeah it's about this high now instead of assigning the hard labels so here we also get the label right it's a supervised learning task like one and zero you also put the data point through this model right here obtain whatever that model would have said and let's say it's about this and now presumably this model is better already so you say well the label here says that it's this class but the model that's really good says you shouldn't be too sure about it so you can sort of mix the two losses and this this process of transferring the knowledge of this model to here is called distillation with the lower model being the teacher model now if you do distillation you can actually improve your performance even more and they show that in the experiments here especially again in the low in the low parameter regime but you can see for example in squad here that the distilled movement pruned method now catches up with the magnitude pruned method in the also in the high in the not so sparse regime okay and they analyze these weights and as you can see that expected the magnitude pruned method it will simply cut out anything right here that's not not a surprise whereas the movement pruned method it will leave a lot of these weights alive so as you can as you can see basically it's it's very much the case since you can outperform the red the yellow one can outperform the red one it is almost warranted to say that the this magnitude pruning wasn't the best choice it's actually a better choice to leave some of those weights in and actually cut some of the weights that are large out just based on their movement now the V here in the middle is of course due to the fact that if a weight is here it was probably not super important in the first place and since since this thing removes anything that moves towards zero any point starting let's say around here moving towards zero would end up here so all the points that end up in this region probably moved towards zero during training and therefore are cut away so there's there's not just like for there to be points there would have been points that started even more in the middle and then moved out to here right and there's just not as many so that's why you have that the V shape is very natural to expect right here so they they analyze this then in terms of the where the model cuts the weights out now they experiment on a BERT base thing that which is a transformer with 12 layers and if you don't know what BERT is you can go look at my video on BERT but you can see that the magnitude pruning will sort of cut all the weights on the layers equally so it will sort of go through the layers and take away let's say 90 percent of each here you can see 10 percent of weights remaining whereas the movement pruning especially the soft movement pruning will actually make a large difference it will remove much much more of the later layers weights and keep the lower layer weights which i think if you do transfer learning from these language models it tends to be that the lower layers maybe pick up if you if you think of a CNN the lower layers might pick up you know on these essential image features like corners and so on and the higher layers will pick up on the task specific things now if you do like a big pre-training tasks you might have a lot of information that you need there but then if you distill it and transfer it down to like a small set small task where only a single thing is important like in squad it's only important what's the answer to the question then you can probably remove a lot of that superfluous information that was there like high level features from the pre-training task i mean that's my my guess here but they also have explained that so yeah that was this paper if you're still here and you enjoyed it leave a like tell me in the comments what you think and i'll see you next time bye bye
 Hi there, today we're looking at movement pruning, adaptive sparsity by fine tuning
 by Victor Sun, Thomas Wolff and Alexander M. Rush of Hugging Face and Cornell University.
 On a high level, this paper proposes that if you have a transfer learning objective
 and you want to do pruning, you should not do pruning by weight magnitude, you should
 do pruning by how much the weights move during the transfer learning.
 This yields better results in the very sparse model regimes and is specifically relevant
 to current NLP transfer learning tasks such as BERT models.
 So if you like content like this, consider subscribing and sharing it to your friends
 and as always leave a comment if you have anything to say on this.
 Alright let's dive in.
 So they say magnitude pruning is a widely used strategy for reducing model size in pure
 supervised learning.
 So what is magnitude pruning?
 Now if I have a neural network, let's say I have a convolutional neural network and
 I input my little cat right here and I have a bunch of layers right and now if we look
 at these layers, each of these layers is going to be made up of these units of the neurons
 and the next layer is also made up of these neurons.
 Now what kind of neural network that is, it's not that important but what is important is
 that you have these connections from neuron to neuron and in let's say a fully connected
 network every neuron is connected to every other neuron.
 In a CNN that would be slightly different but in essence you have a lot of connections
 here and these are usually called weights.
 So these are the weights.
 Now the problem is if I train like these giant neural networks and I want to ship them for
 example to mobile devices to my customers then they won't be able to download gigabytes
 of models or even like hundreds of megabytes of models, it's just not possible.
 So what we want to do is we want to prune this model which means we want to remove parts
 of these weights, a lot of these weights but we don't want to lose accuracy of the network.
 So imagine I have a network and that's trained, it's an image classifier, it's here it's cats
 or dogs and I have it trained to a good accuracy.
 I want to delete these weights but I want to retain the performance and these methods
 are called pruning.
 Now what people do is usually they sort of go in stepwise fashion, they say well first
 of all I don't need some of these and then they delete some and then they sort of retrain
 the pruned network and after that they go again and they say well I don't really need
 that one and they don't really need that one so they do it in this stepwise fashion until
 the network is of the size that they want and the hope is that you don't lose too much
 accuracy.
 So the question is how do you select which weights you need and which ones you don't
 need and usually this is done by so called magnitude pruning which means that you look
 at the weights and the weights they'll have some distribution, there will be very negative
 so here is very negative weights and here is very large positive weights and what you'll
 say is that okay probably the weights that are very large they contribute a lot to the
 signal of the network within the network and the weights that are quite small they're you
 know since there's all this noise and stuff they're probably not that important so I'm
 going to cut off basically right here and everything that's in here I'm going to delete
 those are the non-important weights whereas on the outside those are the important weights.
 This is called magnitude pruning because it goes by the magnitude of the weight the absolute
 value of the weight.
 So you don't actually need so there's not one threshold here you don't need a threshold
 you simply need a method to order the weights right and then you keep removing them until
 you're satisfied with the size.
 So this is magnitude pruning.
 Now what's the problem with the magnitude pruning in these kinds of tasks?
 They say however it is less effective in the transfer learning regime that has become standard
 for state-of-the-art natural language processing applications.
 So what do you do in these transfer learning regimes?
 In the transfer learning regime and actually let's go with the image example right here
 even though it's mostly used in NLP we can do the same thing.
 So let's say we have a classifier here for cats and dogs our classifier and we had a
 big big database of cats and dogs images right so we were able to train that fairly well
 and we don't prune it yet we have this full network.
 Now we want to adapt this to a task where we want to recognize whether or not the animal
 is sick.
 So we developed this app for a veterinarian and it's like a short screening for a particular
 disease that a cat might have and we already have this cats and dogs classifier so it's
 reasonable to assume that this classifier has some good features to work with cats and
 dog images.
 So what we can do instead of because let's assume for this other task we just have this
 tiny little data set which is not enough to train a neural network of this size right
 but so in the first step we'll train this big neural network on the cats versus dogs
 and then we what we do is we transfer learning so we transfer all the weights right here
 and here we have a different task now sick or not sick right this is cat this is dog
 and here is sick or not sick not sick and of course we can't transfer these particular
 weights but we hope that the features here will sort of be the same so we transfer them
 and then we train these weights including the head right here this part we train it
 on this little data set and we hope that we already have this good starting point we only
 need to you know learn the basically the specifics of what makes these two data sets different
 and we won't have to learn entire task of dealing with cat and dog images from the get-go
 okay so this is called transfer learning now in this case we combine the two so first we
 want to transfer learn like if we build this app for vets and then we might say oh this
 is not you know this is not only for vets this is actually for anyone you know who has
 a cat or a dog at home so what we could do is build an app where anyone at home could
 scan their their cat and it would output like a probability of the cat having that disease
 so we want this neural network is still the same size as this neural network so now we
 want to do the pruning we want this neural network to become sparse to only have a couple
 of connections left such that it's a few kilobytes large but retain performance now they say
 when you do this step you can't just do the magnitude pruning like you did right here
 and why not because this is not this model right here is not the result of a training
 step like of a regular training process but is the result of a transfer learning process
 where first you do the big training and then second you adapt it and why is that the case
 well ultimately what you want to do is you want to prove the non-important weights now
 there could be a weight right here this one that is very important for the cat versus
 dog task but that is not important for the sick versus non-sick task and we also we know
 that in these transfer learning settings the weights they don't tend to move that much
 in general the research shows that once you've trained a neural network basically the beginning
 is important but then once you did it like if you adapt it or transfer learn it and so
 on the weights they won't move that much so in essence this weight maybe starts out right
 here and it will sort of stay around this place it will maybe go a little bit down because
 it's not important but it won't move much during transfer learning that's just a property
 of transfer learning so this paper here says we can't just use magnitude pruning when we
 transfer learn because what will basically go by what will basically say is the will
 assign the importance based on based on the original neural network task on the cat versus
 dog we we will miss specify the importance of the weights what we should do is actually
 measure the importance with respect to this task and how do they achieve it so on a high
 level they're basically saying okay if we start out well this was fatal if we start
 out with a point over here let's make that red red i want the color red well it's blue
 now so if we ah if we start out with a point over here what we should do what we should
 do is we should observe how it moves during transfer learning if it moves towards zero
 then it's probably not that important for the new task and if it moves to the to be
 even larger then it's probably important for that new task okay so that's that's a high
 level now how do you measure how it moves and what exactly how exactly do you do all
 of this during training such that you don't make mistakes that's the point of this paper
 they say we propose movement pruning a simple deterministic first-order weight pruning method
 that is more adaptive to pre-trained model fine-tuning we give mathematical foundations
 to the method and compare it to existing zero and first-order pruning methods okay so um
 yeah we said so that's basically on a high level that's that now how do they actually
 do it what they do right here is the following they say what we can define we can define
 each each network layer basically as a matrix multiplication by a weight now you can express
 pretty much any neural network as such a multiplication with a weight so you have x the in the signal
 in each layer and you multiply that by the weight matrix w now if you prune the neural
 network you can see that right here what you're saying is i basically in here i have the matrix
 m which is a mask so the mask is either zero or one for if a weight is active or if a weight
 is not active now this is not a matrix multiply actually this is like a hadamard product um
 but you have this mask matrix and what decides on this mask this mask is decided as you can
 see right here by this s so s s is a matrix that for each entry in w it will decide how
 important it is now in the classic sense in the magnitude pruning you already saw that
 this is just going to be the absolute value of w i j okay and then the top v simply means
 that you take the whoever are the most important the most magnitude that those are going to
 be one in the mask and everything else is going to be zero in the mask okay that's how
 this s this the w determines the s and the s determines the m okay so what you ultimately
 use is the m right here um but in now what we want to do is we want to actually make
 the s based on the movement and the movement is not really a defined concept because it
 goes over steps and so on so how do you do the movement in in a kind of dynamic way and
 this paper says you should do it by gradient so um you you should observe the gradient
 of your loss function with respect to this s matrix to this importance matrix right what
 does it mean so the what does it mean it means let's consider this quantity right here if
 s is the importance of a particular connection and if the gradient is large that means this
 this connection moves a lot right it like the loss pulls it into a particular direction
 okay so we're not talking about yet which direction actually the gradient has a sign
 it's either positive or negative right so by this quantity you can decide how much does
 this new task want this particular importance score to move so this is a direct direct measure
 of how much basically the loss function pulls on that importance score how much and now
 you can simply decide if and they have these they have i think they have a diagram yes
 so but i don't like that let's go so we have right here we have what's the value of this
 gradient of l with respect to s and here is w so if the gradient is positive and w is
 already positive that means the gradient goes into the positive direction so you increase
 the loss function in that let's put the negative gradient here because you do gradient descent
 right so so if the negative gradient is positive and the weight is already positive in in this
 case that means you're all the weight is already high but now the loss function wants to push
 it even higher so that must be a very very important weight right like it's like very
 good the same goes if the gradient the negative gradient is negative and the weight is already
 negative the weight being negative already means the weight you know it has a negative
 sign and then the gradient wants it to go even more negative the the optimization procedure
 says this thing should become even more negative and also we say that's probably a good weight
 now the other two cases means basically the weight's already positive but the gradient
 wants it to go negative which means it's pulled towards zero now it's entirely possible that
 it's going to cross zero and go like if you're here going from over here gonna here cross
 zero and become like super large but that violates our basic assumptions that the transfer
 learning doesn't move the weights too much right what you're caring for is basically
 this local neighborhood right here okay so you can make the fair assumption that these
 weights are not that important in the case where the negative gradient goes against the
 sign of the weight so this this is of course discrete right now but we can actually assign
 a number by how large the gradient is and by how large the weight already is and therefore
 we can make a score so the important score right here as you can see is the weight multiplied
 by the gradient of the weight and they can actually show mathematically that if you do
 this over multiple steps so you optimize while you do this pruning and they do some sort
 of a soft pruning so you can kind of correct your mistakes later on I mean they have hard
 and soft pruning but in any case they can correct their mistakes later on this will
 actually result in these important scores being an accumulation over the training over
 the entire training of this quantity and that's pretty cool because that means eventually
 you'll sort of have a consistent estimator of these important scores across your training
 procedure okay because the main fear with something like this of course is that it's
 very brittle and very much depends on the training dynamics and who knows if in step
 one something bad happens and so on but the the math behind this here gives sort of more
 evidence that this it can be like a self-correcting mechanism it is actually not too dependent
 on the particular training dynamics so they do this experimental setup now they have some
 they have some quirks here actually let's first go to the the the actual different methods
 they compare different methods right here where they say okay there's magnitude pruning
 it's a zero with order which basically just means you just look at the weight magnitude
 that that's it it's top v which means you just pick the top whatever and the objective
 is just the loss and the scores are just the absolute value we've seen this now movement
 pruning on the other hand is first order which means you look at the movement in our case
 the gradient as you can see here those are the importance scores and you use this straight
 through estimator which is basically just a way of saying that even though you're masking
 some things in the forward step you shouldn't mask them in the in the gradient backward
 step because you still want gradient signal to reach so if you have layers and you have
 a weight right here at least that's how I understand it I have not read that paper but
 if you mask this one here you still want the gradient to sort of flow backwards because
 you still need the actual importance scores for the weights that here below connect to
 this weight I think that's what is meant not entirely sure in this though so but you can
 see that the objective function is also the actual loss function now this is contrasted
 to a baseline called l0 regularization which is quite similar but is also first order but
 has this sort of regularizer right here and uses the gumball softmax in order to determine
 the scores and as you can see it also has a different score function and it has this
 continuous hard concrete importance masking function sorry masking function and they have
 a variant of movement pruning that tends to perform a little bit better which is soft
 movement pruning where instead of just going by the loss function optimizing the loss function
 they optimize the loss function plus something now they have as you can see here they have
 a thresholding masking function and the threshold is actually dynamic or it's determined by
 the importance scores and they have then a regularizer that make the importance scores
 sparse so instead of saying we just want the top 5% of weights they now just put a lot
 of mass on this lambda right here which will cause the s to be sparse and you know they
 think if they are not happy with how many weights they can simply increase or decrease
 this lambda such that they get to their desired sparsity of course you see there's the direct
 trade-off with the loss function right so the more you put weight on this lambda the
 less weight is you put basically on the loss function itself so you have the trade-off
 here is very explicit whereas in basic movement pruning it's just given by you masking away
 completely the bottom 1-v of percent of the weights but you see the score function is
 the same oh well the score function here the score function is the same okay now there
 are quite a number of tricks here like there's this sparsity scheduling function and so on
 so as always in NLP and with any big models there are a bunch of engineering tricks that
 make everything work better and you can never exactly tell how much is due to that and how
 much is due to the the actual technique but if you know you can sort of assess whether
 or not these it's done well and this here actually the rationale makes sense and that's
 why I tend to think that it is actually a better method and the experiments are very
 very convincing let's say okay this is just a pictorial comparison where you can see movement
 pruning sorry magnitude pruning all it does is it looks at after you fine-tune what are
 the weights and it just cuts away everything in the middle doesn't care about how the weights
 were when before however movement pruning looks at the combination of what were the
 weights before and what are the weights now and it cuts away everything where the weights
 moved towards zero which are these quadrants right here and it leaves in everything where
 it moved away from zero or that's that's the ordering let's say that how much it moved
 okay experiments now as you might already have figured out by now in the machine learning
 and especially the NLP community the methods presented always outperform the previous methods
 in this case it's pretty special so they test this on these number of tasks quad and NLMI
 and QQP and these are quite hard tasks from an NLP perspective like squad is question
 answering MNLIs like language inference so these would be on the I would guess these
 are on the foreign NLP system on the harder side of tasks that's it's fairly cool and
 as you can see here now just focus on first of all focus on this MAP which is the magnitude
 pruning so that's the baseline if you will and the purple one the SMVP which is the soft
 movement pruning okay now you can also focus on the MVP right here but they're approximately
 the same now the RPP you can maybe see in the graph is performing fairly well even compared
 to the full model it's another baseline basically but we just want to compare those to and
 you can see that in this regime that the magnitude pruning outperforms the movement pruning but
 however in this regime the movement pruning is much better and that's the percent where
 the percent of remaining weights is very very low so this is kind of the extreme sparse
 case where you only have 10% or you only have 3% of the weights left and you can see that
 the movement pruning is outperforming the magnitude pruning by a lot okay now they do
 discover that so this this happens in all in all of these tasks as you can see right
 here and they do they do discover that if you then distill the model further so in distillation
 this is yet another technique that you can use to boost the performance of the transfer
 learned model so in distillation you would not only train the model on you so you have
 you now you have your this model that you transfer learn and you have the pruned version
 and in the pruned version what you would do is you would simply also train it on this
 data set but what you can also do is you can distill this model right here the one that
 you trained on the same task right that's that's presumably better because it still
 has all the weights okay you can run a data point through both so the same data point
 goes through this and you get an output which are logits which is like representing a distribution
 saying yeah it's about this high now instead of assigning the hard labels so here we also
 get the label right it's a supervised learning task like one and zero you also put the data
 point through this model right here obtain whatever that model would have said and let's
 say it's about this and now presumably this model is better already so you say well the
 label here says that it's this class but the model that's really good says you shouldn't
 be too sure about it so you can sort of mix the two losses and this this process of transferring
 the knowledge of this model to here is called distillation with the lower model being the
 teacher model now if you do distillation you can actually improve your performance even
 more and they show that in the experiments here especially again in the low in the low
 parameter regime but you can see for example in squad here that the distilled movement
 pruned method now catches up with the magnitude pruned method in the also in the high in the
 not so sparse regime okay and they analyze these weights and as you can see that expected
 the magnitude pruned method it will simply cut out anything right here that's not not
 a surprise whereas the movement pruned method it will leave a lot of these weights alive
 so as you can as you can see basically it's it's very much the case since you can outperform
 the red the yellow one can outperform the red one it is almost warranted to say that
 the this magnitude pruning wasn't the best choice it's actually a better choice to leave
 some of those weights in and actually cut some of the weights that are large out just
 based on their movement now the V here in the middle is of course due to the fact that
 if a weight is here it was probably not super important in the first place and since since
 this thing removes anything that moves towards zero any point starting let's say around here
 moving towards zero would end up here so all the points that end up in this region probably
 moved towards zero during training and therefore are cut away so there's there's not just like
 for there to be points there would have been points that started even more in the middle
 and then moved out to here right and there's just not as many so that's why you have that
 the V shape is very natural to expect right here so they they analyze this then in terms
 of the where the model cuts the weights out now they experiment on a BERT base thing that
 which is a transformer with 12 layers and if you don't know what BERT is you can go
 look at my video on BERT but you can see that the magnitude pruning will sort of cut all
 the weights on the layers equally so it will sort of go through the layers and take away
 let's say 90 percent of each here you can see 10 percent of weights remaining whereas
 the movement pruning especially the soft movement pruning will actually make a large difference
 it will remove much much more of the later layers weights and keep the lower layer weights
 which i think if you do transfer learning from these language models it tends to be
 that the lower layers maybe pick up if you if you think of a CNN the lower layers might
 pick up you know on these essential image features like corners and so on and the higher
 layers will pick up on the task specific things now if you do like a big pre-training tasks
 you might have a lot of information that you need there but then if you distill it and
 transfer it down to like a small set small task where only a single thing is important
 like in squad it's only important what's the answer to the question then you can probably
 remove a lot of that superfluous information that was there like high level features from
 the pre-training task i mean that's my my guess here but they also have explained that
 so yeah that was this paper if you're still here and you enjoyed it leave a like tell
 me in the comments what you think and i'll see you next time bye bye