Feature Visualization & The OpenAI microscope
deep learning machine learning imagenet visualization features intermediate hidden layers activations patterns openai google interactive explanation
A closer look at the OpenAI microscope, a database of visualizations of the inner workings of ImageNet classifiers, along with an explanation of how to obtain these visualizations.

https://distill.pub/2017/feature-visualization/
https://microscope.openai.com/models
https://github.com/tensorflow/lucid

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
 Hi there! Today we're going to take a look at the OpenAI microscope and this article on Distill called Feature Visualization. So the Feature Visualization article is by Chris Ola, Alexander Mortwintsev and Ludwig Schubert of the Google Brain team, while the OpenAI microscope is by OpenAI. So keep that in mind. These tools are tools for visualizing what neural networks learn and specifically here we're dealing with image classifiers and even more specifically ImageNet classifiers. So you know ImageNet is this big data set with a thousand classes of images and the images are somewhat like 200 by 200 pixels and you're just supposed to put them into one of these 1000 classes and networks have become really good at these kinds of things. So our question is what do these networks learn? And there are a number of very cool works that have started to investigate what these networks learn. So this started with work like Deep Dream or even before that but this is a very good summary and also an overview of this. So in this article they first showcase these patterns here where you can see that as you go through the network in the layer, this is layer conf2d0, which is a very low layer in the network, you have things what looks like pattern, just pattern detectors. So these are the things that the network is excited by and we'll just get, we'll get in a second at how you create these things. Just be sure these are the things that these particular layers in the networks are most excited by. So this layer is super excited by these textures. But as you go higher the network gets excited by these kind of textures, then it gets excited by more complex things. So the pattern is the higher you go in the layers and these people have been seeing and claiming this and measuring this since RBF networks and whatnot, that the higher you go in these networks the more complex features that they build. So they always build hypothesis is they build very complex features from the lower layers and the lower layers they have less less complex features and until you're the very bottom layer they simply extract edges and patterns of texture like this. Whereas in the top layers all of these are hierarchically assembled to give you very very intricate features. So these usually look pretty funky that's why I like to investigate. So this article focuses on how is this done and the answer is by optimization. Now what do we mean? I actually have the article here somewhat printed out but the graphics they don't really print very well to the notebook here. So imagine this here. So what you want to do is you want to see how much activation is in the network for a particular input and you can do this in many many different ways. The easiest form, let's actually start over here, if you have a neural network here and this is the softmax classifier and these are the classes in this case let's say this is dog, this is cat and this is car, house. Let's go with house. You can think of okay I want to know when the network sees a cat. What does the network think of cats? So what I would do is I would take an image that is just noise, just random noise like this one on the left here and I would start to optimize using backpropagation. I would start to optimize this image. Now you usually know backpropagation as the thing that will optimize these weights if we have given an image and a label. But right now you have to, so and we optimize the weights here, sorry I should say that, but right now you have to rethink what we have given is the label and the weights. We keep them constant but we optimize the input image to maximize this label as much as possible. So we ask of xx please please update yourself such that the output is as much cat as possible. And then we just optimize for that. So we hope that this picture would turn out to be as much cat as we like. And usually here, I don't know exactly which class they have here, but you won't get a cat. Sorry over here. You won't get a cat or here if you optimize the logits it's the same, just you do the same thing before the softmax. What you will get is some weird trippy thing. So in our classifier you might get something like there's a cat here but there's also one here right because two cats are more cat than one cat and there's like a giant cat head here and inside the cat head there is another cat head and there's like a cat tail somewhere here and again there's a cat eye right here. So you get like something super trippy that is as much cat as possible. Alright now so this is somewhat interesting because you can find out what does the network think is the most cat-like thing there is and the most dog-like thing there is. But what you can also do is you can see what do the intermediate layers of the networks, what do they get excited by. So what you can do is for example you can take an individual neuron in one of the layers right so this here might be a convolutional layer with its convolutional filters right and these are the different channels of the convolutional layer and then this thing here is just a single neuron there and what you can do is you can say okay X we have now again we input the image X and we optimize the image X such that with a given weight this particular neuron let's call it N and N is maybe this neuron right here right this particular neuron is as much activated as possible right so we no longer optimize for a label but we optimize for a particular neuron to be activated as much as possible and then we can sort of see what is an image that activates this neuron as much as possible in this case that thing you can actually do the same thing with an entire channel if you simply say when is the channel as a whole activated as much as possible and something like deep dream did the same thing but with an entire layer in the neural network like what is this layer activated by they can imagine there's not only one image that activates it the most but probably depending on your start here it's it's it's a you'll get different different results for depending on where you start we'll go into that later as well so these are the kinds of things you can do to investigate these neural networks and see what they what they pay attention to in each of the layers right so let's go on here and so they say they say okay if we optimize by optimization you get something like this here on the bottom row looks pretty funky right but what you could also do is you can visualize by what there's called data set examples and date with data set examples you don't do this thing you don't do the optimization procedure but you go into your into your database into your data set and you find the images that activate so the XI from your data set that activate a particular neuron the most so you simply sort all the images in your database and you just pick the ten or so that activate that particular neuron the most and that would also give you an understanding it's actually a valid thing and the AI microscope combines both of them so on the bottom you see what particular neurons are most excited by and at the top you see that that the data set examples they're most excited by so you can see that there is a there's a healthy diversity in the data set examples but they also all kind of map to the to the what it's most excited by it's pretty cool the last point they make in this article is that of diversity with the data set examples you do get naturally sort of a diverse set of images of course where you can guess you can also say okay whether the maximum activations and only slightly positive you can even give negative examples but with the positive you can only either maximize fully or you can you know take the negative and you can say okay what is this neuron not excited by at all but you you won't get kind of a spectrum of what the neuron is excited by or the unit or the layer and what they're doing is simply they add a diversity term they say that works best so what does that mean it means that here if we optimize X and Y right let's go up here if we optimize X in that right and try to maximize the activation of n we don't want to do X by itself but we want to do is we want to do an entire set of X I right so we feed an entire mini batch in there and we want to maximize n but also maximize a diversity term let's call that D between X 1 to X be right so you want to maximize the activation of the neuron but at the same time in your loss function that you optimize you also have this diversity term where you say the images that you produce should be far apart from each other or kind of apart from each other and thus you do get diverse samples okay the printing again doesn't work so here you see that if you just simply optimize you get the thing on the left but if you have a batch of things and you optimize them to be diverse from each other but also activate the layer or the unit you get a variety of high activations and you can see here that this is some sort of curve this could be a beak of a bird but on the right here this could also be kind of a snout of a monkey or something okay so it's just curvy curvy things that is activated by here they give another example you can clearly see that there is like some sort of eye in the picture but then if you optimize with diversity you can see that some of them do not have this eye thing and in fact also some of the data set examples do not have this eye thing so it might be interesting to to to optimize here with diversity and even in they say in higher layers it gets even more more diverse with what you achieve with this ball detector they also say they research interactions between neurons where they can interpolate between them right so you can here have two different new units let's say this top left one here is the thing that we've just seen with the curvature activator and then on the right we can select this thing here that is appears to be activated by these bird like bird like things and if you optimize an image that activates both of them you get the thing on the bottom left and this is very good for understanding how neural networks work because what a neural network will do is exactly it will take the thing on the top left and the top right from lower layers and it will combine them to form features of higher layers so while the top right thing looks like generic birds the bottom left thing looks much more like birds with let's say long necks and then kind of curved necks so more stork ish birds right because we've added in this curvature thing so this is very very cool to play around you can also here interpolate between into neurons like you would interpolate in a in a GAN right and yeah so they do make a point of regularization I don't want to go into that particularly but you have to be careful you can't just apply the optimization procedure as I said right now you have to actually have to do some regularization and to get to get rid of what are essentially adversarial examples in this process because if you just straight-up optimize you will get pretty high frequency crappy results I actually want to jump over now to this OpenAI microscope so this is a tool that lets you explore these visualizations so at the beginning you can pick one of these models and I'll pick inception v1 just because some of the other ones they don't have everything done quite yet the all the all the all the visualization so on the right here you can actually see the architecture of the network if you know what an inception network is this what this looks like and you would be able from here to select one of these units straight away but I'm gonna sorry we're gonna go to the left here so you have deep dream activated which means the entire layer is optimized for so per layer on the right side you have an image here and you can already see that if we go from the bottom what we saw before we get patterns that become more and more complex as you go up the layers and then more and more until you finally have what the network appears to be most activated by is mostly dogs which is okay because image net is dominated by dogs so you can click on any of these right here like this one and now you'll be able to inspect the individual nodes in this layer so before we had the whole layer right the whole layer was activated by something but these layers they have different channels and also different neurons within the channel so you can select this here you can go neuron activation or channel activation and these are the images that these channels are excited by the most you see you get pretty funky pattern if we select one interesting one maybe this this one right here you can see on the left this is the channel optimizing optimization on the right this is the neuron optimization and here you get the data set examples that are most activated that mostly activate this particular channel or neuron so sorry this particular yeah channel so you can see this is pretty similar to the thing I drew where except for it being a cat it's some sort of a fox dog thing right and and you can explore the neural network in this fashion so you can go through the layer here and look at that good units this seems to be whiskers classifier and lo and behold things with whiskers will activate the the neuron and as you go up the layer and this is the the cool thing right so we're right now we're here in this layer for as you go up you will see more and more intricate patterns of activations I I could play around this for very very long time but I won't I won't waste your time too much they there is a dist sorry there is a slack workspace where people discuss interesting patterns what is this okay yes this is a some sort of temple temple constructor very cool there is a slack workspace where people discuss interesting things for example they discuss how the car detector that you see right here is one of the units there is literally endless units to look at that detects cars can be clearly seen to be built from lower level features such as this wheel detector you see this wheel detector here is the unit three three seven in the mixed four layer and this car hood detector right here is unit two three seven also in the in one of the layer fours right so these are both from layer four and then the car detector I haven't looked this up ah isn't layer four as well but let's check it out this isn't layer 4b this isn't layer 4b and this isn't layer 4c ah so you see I this was a risk the car detector is built from lower level features of car hood and car wheel right the car wheel right here detects wheels and the car hood detector detects hoods and then the car detector detects cars so there are very like I really invite you to go look at it check out what people find and explore these models all of this is based on this lucid library right here also invite you to check that out where you can perform such optimizations yourself I'll link to that and with that bye bye
 Hi there! Today we're going to take a look at the OpenAI microscope and this
 article on Distill called Feature Visualization. So the Feature
 Visualization article is by Chris Ola, Alexander Mortwintsev and Ludwig
 Schubert of the Google Brain team, while the OpenAI microscope is by OpenAI.
 So keep that in mind. These tools are tools for visualizing what neural
 networks learn and specifically here we're dealing with image classifiers and
 even more specifically ImageNet classifiers. So you know ImageNet is this big
 data set with a thousand classes of images and the images are somewhat like
 200 by 200 pixels and you're just supposed to put them into one of these
 1000 classes and networks have become really good at these kinds of things.
 So our question is what do these networks learn? And there are a number of very cool
 works that have started to investigate what these networks learn. So this
 started with work like Deep Dream or even before that but this is a very good
 summary and also an overview of this. So in this article they first
 showcase these patterns here where you can see that as you go through the
 network in the layer, this is layer conf2d0, which is a very low layer in
 the network, you have things what looks like pattern, just pattern detectors. So
 these are the things that the network is excited by and we'll just get, we'll get
 in a second at how you create these things. Just be sure these are the things
 that these particular layers in the networks are most excited by. So this
 layer is super excited by these textures. But as you go higher the
 network gets excited by these kind of textures, then it gets excited by more
 complex things. So the pattern is the higher you go in the layers and
 these people have been seeing and claiming this and measuring this since
 RBF networks and whatnot, that the higher you go in these networks the
 more complex features that they build. So they always build hypothesis is they
 build very complex features from the lower layers and the lower layers they
 have less less complex features and until you're the very bottom layer they
 simply extract edges and patterns of texture like this. Whereas in the top
 layers all of these are hierarchically assembled to give you very very intricate
 features. So these usually look pretty funky that's why I like to investigate.
 So this article focuses on how is this done and the answer is by optimization.
 Now what do we mean? I actually have the article here somewhat printed out but
 the graphics they don't really print very well to the notebook here. So
 imagine this here. So what you want to do is you want to see how much
 activation is in the network for a particular input and you can do this in
 many many different ways. The easiest form, let's actually start over here, if
 you have a neural network here and this is the softmax classifier and these are
 the classes in this case let's say this is dog, this is cat and this is car, house.
 Let's go with house. You can think of okay I want to know when
 the network sees a cat. What does the network think of cats? So what I would do
 is I would take an image that is just noise, just random noise like this
 one on the left here and I would start to optimize using backpropagation. I would
 start to optimize this image. Now you usually know backpropagation as the
 thing that will optimize these weights if we have given an image and a label.
 But right now you have to, so and we optimize the weights here, sorry I
 should say that, but right now you have to rethink what we have given is the
 label and the weights. We keep them constant but we optimize the input image
 to maximize this label as much as possible. So we ask of xx please please
 update yourself such that the output is as much cat as possible. And then
 we just optimize for that. So we hope that this picture would turn out to
 be as much cat as we like. And usually here, I don't know exactly which class
 they have here, but you won't get a cat. Sorry over here. You won't get a cat or
 here if you optimize the logits it's the same, just you do the same thing before
 the softmax. What you will get is some weird trippy thing. So in our classifier
 you might get something like there's a cat here but there's also one here right
 because two cats are more cat than one cat and there's like a giant cat head
 here and inside the cat head there is another cat head and there's like a cat
 tail somewhere here and again there's a cat eye right here. So you get like
 something super trippy that is as much cat as possible.
 Alright now so this is somewhat interesting because you can find
 out what does the network think is the most cat-like thing there is and
 the most dog-like thing there is. But what you can also do is you can see what
 do the intermediate layers of the networks, what do they get excited by. So
 what you can do is for example you can take an individual neuron in one of the
 layers right so this here might be a convolutional layer with its
 convolutional filters right and these are the different channels of the
 convolutional layer and then this thing here is just a single neuron there and
 what you can do is you can say okay X we have now again we input the
 image X and we optimize the image X such that with a given weight this
 particular neuron let's call it N and N is maybe this neuron right
 here right this particular neuron is as much activated as possible right so we
 no longer optimize for a label but we optimize for a particular neuron to be
 activated as much as possible and then we can sort of see what is an image that
 activates this neuron as much as possible in this case that thing you can
 actually do the same thing with an entire channel if you simply say when is
 the channel as a whole activated as much as possible and something like deep
 dream did the same thing but with an entire layer in the neural network like
 what is this layer activated by they can imagine there's not only one image that
 activates it the most but probably depending on your start here it's it's
 it's a you'll get different different results for depending on where you start
 we'll go into that later as well so these are the kinds of things you can do
 to investigate these neural networks and see what they what they pay attention to
 in each of the layers right so let's go on here and so they say they say okay if
 we optimize by optimization you get something like this here on the bottom
 row looks pretty funky right but what you could also do is you can visualize by
 what there's called data set examples and date with data set examples you
 don't do this thing you don't do the optimization procedure but you go into
 your into your database into your data set and you find the images that
 activate so the XI from your data set that activate a particular neuron the
 most so you simply sort all the images in your database and you just pick the
 ten or so that activate that particular neuron the most and that would also
 give you an understanding it's actually a valid thing and the AI microscope
 combines both of them so on the bottom you see what particular neurons are most
 excited by and at the top you see that that the data set examples they're most
 excited by so you can see that there is a there's a healthy diversity in the
 data set examples but they also all kind of map to the to the what it's most
 excited by it's pretty cool the last point they make in this article is that
 of diversity with the data set examples you do get naturally sort of a diverse
 set of images of course where you can guess you can also say okay whether the
 maximum activations and only slightly positive you can even give negative
 examples but with the positive you can only either maximize fully or you can
 you know take the negative and you can say okay what is this neuron not excited
 by at all but you you won't get kind of a spectrum of what the neuron is
 excited by or the unit or the layer and what they're doing is simply they add a
 diversity term they say that works best so what does that mean it means that
 here if we optimize X and Y right let's go up here if we optimize X in that
 right and try to maximize the activation of n we don't want to do X by itself but
 we want to do is we want to do an entire set of X I right so we feed an entire
 mini batch in there and we want to maximize n but also maximize a diversity
 term let's call that D between X 1 to X be right so you want to maximize the
 activation of the neuron but at the same time in your loss function that you
 optimize you also have this diversity term where you say the images that you
 produce should be far apart from each other or kind of apart from each other
 and thus you do get diverse samples okay the printing again doesn't work so here
 you see that if you just simply optimize you get the thing on the left but if you
 have a batch of things and you optimize them to be diverse from each other but
 also activate the layer or the unit you get a variety of high activations and
 you can see here that this is some sort of curve this could be a beak of a bird
 but on the right here this could also be kind of a snout of a monkey or something
 okay so it's just curvy curvy things that is activated by here they give
 another example you can clearly see that there is like some sort of eye in the
 picture but then if you optimize with diversity you can see that some of them
 do not have this eye thing and in fact also some of the data set examples do
 not have this eye thing so it might be interesting to to to optimize here with
 diversity and even in they say in higher layers it gets even more more diverse
 with what you achieve with this ball detector they also say they research
 interactions between neurons where they can interpolate between them right so
 you can here have two different new units let's say this top left one here
 is the thing that we've just seen with the curvature activator and then on the
 right we can select this thing here that is appears to be activated by these bird
 like bird like things and if you optimize an image that activates both of
 them you get the thing on the bottom left and this is very good for
 understanding how neural networks work because what a neural network will do is
 exactly it will take the thing on the top left and the top right from lower
 layers and it will combine them to form features of higher layers so while the
 top right thing looks like generic birds the bottom left thing looks much more
 like birds with let's say long necks and then kind of curved necks so more
 stork ish birds right because we've added in this curvature thing so this is
 very very cool to play around you can also here interpolate between into
 neurons like you would interpolate in a in a GAN right and yeah so they do make
 a point of regularization I don't want to go into that particularly but you
 have to be careful you can't just apply the optimization procedure as I said
 right now you have to actually have to do some regularization and to get to
 get rid of what are essentially adversarial examples in this process
 because if you just straight-up optimize you will get pretty high frequency
 crappy results I actually want to jump over now to this OpenAI microscope so
 this is a tool that lets you explore these visualizations so at the beginning
 you can pick one of these models and I'll pick inception v1 just because some
 of the other ones they don't have everything done quite yet the all the
 all the all the visualization so on the right here you can actually see the
 architecture of the network if you know what an inception network is this what
 this looks like and you would be able from here to select one of these units
 straight away but I'm gonna sorry we're gonna go to the left here so you have
 deep dream activated which means the entire layer is optimized for so per
 layer on the right side you have an image here and you can already see that
 if we go from the bottom what we saw before we get patterns that become more
 and more complex as you go up the layers and then more and more until you finally
 have what the network appears to be most activated by is mostly dogs which is
 okay because image net is dominated by dogs so you can click on any of these
 right here like this one and now you'll be able to inspect the individual nodes
 in this layer so before we had the whole layer right the whole layer was
 activated by something but these layers they have different channels and also
 different neurons within the channel so you can select this here you can go
 neuron activation or channel activation and these are the images that these
 channels are excited by the most you see you get pretty funky pattern if we
 select one interesting one maybe this this one right here you can see on the
 left this is the channel optimizing optimization on the right this is the
 neuron optimization and here you get the data set examples that are most
 activated that mostly activate this particular channel or neuron so sorry
 this particular yeah channel so you can see this is pretty similar to the thing
 I drew where except for it being a cat it's some sort of a fox dog thing right
 and and you can explore the neural network in this fashion so you can go
 through the layer here and look at that good units this seems to be whiskers
 classifier and lo and behold things with whiskers will activate the the neuron
 and as you go up the layer and this is the the cool thing right so we're right
 now we're here in this layer for as you go up you will see more and more
 intricate patterns of activations I I could play around this for very very
 long time but I won't I won't waste your time too much they there is a dist sorry
 there is a slack workspace where people discuss interesting patterns what is
 this okay yes this is a some sort of temple temple constructor very cool
 there is a slack workspace where people discuss interesting things for example
 they discuss how the car detector that you see right here is one of the units
 there is literally endless units to look at that detects cars can be clearly seen
 to be built from lower level features such as this wheel detector you see this
 wheel detector here is the unit three three seven in the mixed four layer and
 this car hood detector right here is unit two three seven also in the in one
 of the layer fours right so these are both from layer four and then the car
 detector I haven't looked this up ah isn't layer four as well but let's check
 it out this isn't layer 4b this isn't layer 4b and this isn't layer 4c ah so
 you see I this was a risk the car detector is built from lower level
 features of car hood and car wheel right the car wheel right here detects wheels
 and the car hood detector detects hoods and then the car detector detects cars so
 there are very like I really invite you to go look at it check out what people
 find and explore these models all of this is based on this lucid library
 right here also invite you to check that out where you can perform such
 optimizations yourself I'll link to that and with that bye bye