VOS: Learning What You Don't Know by Virtual Outlier Synthesis (Paper Explained)
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper paper explained virtual outliers how to detect outliers deep learning outliers deep learning outlier detection vos deep learning energy latent space outliers density estimation classification boundaries generative models
#vos #outliers #deeplearning
Sponsor: Assembly AI
Check them out here: https://www.assemblyai.com/?utm_source=youtube&utm_medium=social&utm_campaign=yannic1

Outliers are data points that are highly unlikely to be seen in the training distribution, and therefore deep neural networks have troubles when dealing with them. Many approaches to detecting outliers at inference time have been proposed, but most of them show limited success. This paper presents Virtual Outlier Synthesis, which is a method that pairs synthetic outliers, forged in the latent space, with an energy-based regularization of the network at training time. The result is a deep network that can reliably detect outlier datapoints during inference with minimal overhead.

OUTLINE:
0:00 - Intro
2:00 - Sponsor: Assembly AI (Link below)
4:05 - Paper Overview
6:45 - Where do traditional classifiers fail?
11:00 - How object detectors work
17:00 - What are virtual outliers and how are they created?
24:00 - Is this really an appropriate model for outliers?
26:30 - How virtual outliers are used during training
34:00 - Plugging it all together to detect outliers

Paper: https://arxiv.org/abs/2202.01197
Code: https://github.com/deeplearning-wisc/vos

Abstract:
Out-of-distribution (OOD) detection has received much attention lately due to its importance in the safe deployment of neural networks. One of the key challenges is that models lack supervision signals from unknown data, and as a result, can produce overconfident predictions on OOD data. Previous approaches rely on real outlier datasets for model regularization, which can be costly and sometimes infeasible to obtain in practice. In this paper, we present VOS, a novel framework for OOD detection by adaptively synthesizing virtual outliers that can meaningfully regularize the model's decision boundary during training. Specifically, VOS samples virtual outliers from the low-likelihood region of the class-conditional distribution estimated in the feature space. Alongside, we introduce a novel unknown-aware training objective, which contrastively shapes the uncertainty space between the ID data and synthesized outlier data. VOS achieves state-of-the-art performance on both object detection and image classification models, reducing the FPR95 by up to 7.87% compared to the previous best method. Code is available at this https URL.

Authors: Xuefeng Du, Zhaoning Wang, Mu Cai, Yixuan Li

Links:
Merch: http://store.ykilcher.com
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Outliers, we all know them, we all hate them. How can these data points just be out of distribution, not in the training data, things that we haven't seen before, things that we don't even expect? Well, they suck. So today we're going to look at what you can do about it. Specifically, we're going to look at the paper learning what you don't know by virtual outlier synthesis. This paper presents a technique to generate what it calls virtual outliers, which are synthetic data points that are out of distribution. The core idea is that rather than trying to come up with data space out of distribution samples, this paper comes up with latent space out of distribution samples, which is much easier and much more useful. They're then designing a loss that pushes down the energy of the model wherever the outliers are and pushes up the energy wherever the data is. This paper is really interesting because it presented very successful results on a multitude of benchmarks. So definitely this technique looks like it works. However, when I read the paper, I was quite critical. I had a lot of criticisms, I had a lot of open questions, and that's why I've invited the authors for an interview to the channel. So this video right here is a comprehensive paper review. I'll explain in detail what is in the paper, what the method does, what its contributions are, what its experimental results look like, what is good about it, and what I think is bad about it. Then in the next video released tomorrow, I'll interview the authors of the paper, the authors will have seen my review, and therefore are able to respond to any criticism and any questions that I had. So be sure to check out the interview part as well, because it was really, really cool to get all my questions answered. As always, let me know how I can improve these videos by leaving a comment, leave a like if you do like and I'll see you around. Bye bye. And this works in the traditional way where you upload audio and you get back the transcription, but they can also do this real time. So you get a web socket to their neural network powered backend and in real time, it gives you back text for your speech. That's insane. But this is not all they have a ton of features on top of that. For example, they can do summarization, they can do topic detection, they can do bad word detection, content moderation in your audio. And I have to say, this is really good. In fact, I have uploaded this video right here to their API's and the text you see on screen is the raw output of that model. So judge yourself how good it is. We'll actually try some Swiss German words on it. It is an English model, but we'll just give it a shot. Oh, well, isn't that great. So give them a try. They even have a basic free tier at their documentation is super extensive. They give you walkthroughs and examples of all the parameters that you can send. They have a great blog where they describe different feature sets and different ways of applying their technology. And yeah, it's a really cool thing. Now I've only scratched the surface right here. They do much more. They have features upon features on this, but it's best you check them out yourself. So thank you very much to assembly AI for sponsoring this video is really great. Please check them out. A link is in the description and I wish you a lot of fun. Hello there today we'll look at VOS learning what you don't know by virtual outlier synthesis by Shefeng Du, Zhao Ning Wang, Mu Cai and Yixuan Li. This paper presents a model that can do out of distribution detection in object detection networks, but not only in object detection, they show it on object detection, but it is a general framework for detecting out of distribution data at inference time. If this really works, this could mean a lot for especially for safety critical applications, networks that are deployed as a classifier or a detector somewhere. And they would be able to recognize accurately when they are presented with something they didn't learn at training time, like some out of distribution class. And this particular case on the left here, you see an image, which is an object detection network at inference time, it has correctly recognized the car on the right hand side. However, it thinks that the moose here is a pedestrian, it doesn't even classify all of the moose, but it recognizes there is an object. And the class is pedestrian, probably because it hasn't hasn't seen mooses, meese. What's the plural of moose? In any case, it hasn't seen a moose or multiple meese at training time. And therefore, it cannot classify it. And very often these networks make very, very high confidence predictions for classes that they haven't seen. This paper tackles this and proposes this technique called virtual outlier synthesis, to which we'll get to in a second. As I said, it's a general framework. They demonstrated on object detection, which is a particularly hard task, but this could also be applied to image classification. They do make the point that if you have an image like this, and you haven't seen the moose class during training, most of the image will still be in distribution. Like this will not be a particularly out of distribution image, except for that small part with the moose. However, if you do object detection, then the object itself here is out of distribution. And maybe that makes actually their tasks as researchers a bit more easy, because they are less often in these ambiguous cases where like half the data point is out of distribution. In any case, they mentioned here, they that the networks that we currently have, they often struggle to handle the unknowns. And they assign high posterior probability for out of distribution test inputs. Now, why might that be? If you train a typical classifier, the classifier will just attempt to separate classes from each other. You see this here in the middle. This is a projection of the last layer of a neural network right before the classifier layer. So right before the softmax. So the classification layer, all it can do is it can lay linear decision boundaries, essentially, through the distribution of data points. So what the model does is it sees three classes right here. So this is class one, this is class two, this is class three. And what it needs to do is linearly separate them. So it says, well, okay, I'm gonna this is not an ideal color for this. I'm going to just put my decision boundaries like this. And now I've essentially separated the classes, because all that is important to a classification loss is that, you know, points in class three are away from points in class one and away from points in class two. So that also means that the more away from classes one and two I go, the better, like the more likely it is to be class three, because all I've ever seen at training is samples from class three. And my entire objective was just to make it, to push it away or distinguish it, to discriminate it from class one and class two. So obviously, if I go more into the direction of class three, the network will become will output a more and more confident number about this being class three, even though, as you can see, the data is all in this region right here. And out there, there is no data, yet the network is still very, very confident. Red here means quite confident. An ideal situation would be if the network was very confident, where the training data is right here. However, again, we have the decision boundaries like this. However, if you go further out, it will say something like, wait a minute, even though this is not class one, for sure, and not class two, for sure, it's most likely class three, but still, I haven't seen any training data around that area. So I'm also going to be to just output a low probability or a low confidence score. I'm going to say it's class three, but I'm going to assign it a low confidence, because I haven't seen actual training data in that vicinity. Now, this all seems intuitive and makes sense and so on. Mostly, that is because low dimensionality and high dimensionality data is very different and can deceive if you look at it in this in a kind of a very simple projection like this, you as a human, you see this data and you go like, of course, that makes total sense. However, this becomes very different if you look at high dimensional data. Note that there is a reason why our classifiers do the thing on the left, because the thing on the right essentially amounts to like a probabilistic model of the data distribution, right? The thing on the right, it has an idea where all the data is, right? The thing on the left, it just needs to separate data from each other. Three lines are enough for that. The thing on the right actually needs to model the data in the latent space, which can become pretty complicated in high dimensions, and it needs some very, very distinct assumptions to make it tractable. So the right thing is essentially a generative model of the data, like a distributional model of the data, which needs a lot more resources and power and could pull away resources from the classification task to be solved. So what does this model do? First of all, they have some notation right here, which I found to be... Well, let's just first look at the diagram right here. So this is the whole model architecture. They have an input over here. So there's input X, right? I'm going to use the green highlighter, I guess, for this stuff. There's input X. You can see this is the input image. In general, first you have this proposal generator, and that proposal generator will generate bounding boxes. So some of these detection networks, they have two stages. First, proposal generation, and then a post-processing stage where they assign labels to the proposals. So the proposal generator would simply ask, where are objects? Any sort of object. The objectness property, it generalizes between objects. So it makes sense to train the object detector to just predict where are bounding boxes. In this case, it will predict, well, there is one here, there is an object, and there is an object here. And then it will pass on those to the classifier to determine what's in the bounding boxes. And you can already see the object detector has done a good job. It detected that this thing right here is an object. However, the classifier, what can it do? It has to assign a label. There is no option for it to say, no, actually, this isn't an object. And previous methods have tried this. They've just added like an extra class for outlier. It usually doesn't work too well, because the reason is pretty simple. In order to do that here on the left, you'd have to introduce like another line and say, okay, so I'm going to introduce another line, I'm running out of colors here, introduce another line, you know, like right here. So this would now be outlier, sorry, outlier space. Well, that doesn't cover, that doesn't cover this region or this region, or the region back here, right. So having a single class for outliers is sort of useless, because there are just so many places where outliers could be, and not just like a single, a single slice of the space. So you'd have to have many, you'd actually have to have like a lot. And ultimately, that amounts to exactly the situation on the right where, you know, ultimately, you're going to train a classifier that is a threshold between low and high density areas. And that's exactly a generative model of the data. All right, first stage is the bounding box proposal, this thing right here. Then you pass on the bounding box to multiple things. First of all, there is a loss that's simply concerned with did you detect the objects correctly. So during training, the proposal generator would simply be trained with that loss right here. Now everything here is back propagated, obviously, but that would be the main loss to localize the bounding boxes. The second, the second stage here would be the assignment of a label, this would be the so called classification head. So that would take the latent representation that is generated, including the bounding box, right. So we're going to feed this through a neural network. And that will give us a latent representation, this H thing mean that they call that the latent representation right before the classification layer, and the classification layer would assign a label to it. And that would be the normal way of doing things. And now we augment that by a bit. Just to say they formulate this here, as saying we have a data set, the data set here contains x is data, b is bounding box and y is labels. So b and y would be the labels, right, those would be the things to predict. And then they say they split it up into two things. So first of all, the p of the bounding box, and then the one of the label. And I don't think that's correct. I think that's a typo right here. I think this should be the probability of the bounding box given x, not the label. And this should probably be the probability of the label given x as well as the predicted bounding box. Let's call this b hat right here, the predicted bounding box. So b hat would be sampled from this. But this is minor, because the rest of the paper essentially treats it as I think I write it down. In any case, what they do in addition to that is they also have this classifier right here. The classifier that takes into a sample and the bounding box and it tries to predict this number g. And g is one if the object is in distribution and g should be zero if it's out of distribution. So this is a binary classifier that classifies any sample into in or out of distribution, independent of what the classifier head says what class it is. So that would amount to the situation on the right, where if you're anywhere in this region right here, the classifier would still say, well, that's clearly class three, because that's the region of class three. But your other classifier would say yes, but the the outlier probability is very high, the in in layer probability is very low for that region. So you can do outlier detection at inference time. How do we do this? We do this by generating these virtual outliers during training. Virtual outliers are essentially outlier data points that you synthesize. Now, you what you could do, and they mentioned that what you could do is you could train like again, you can simply train a generative model of the data, and then use that to sample out of distribution data. However, they mentioned that synthesizing images in the high dimensional pixel space can be difficult to optimize. Instead, our key idea is to synthesize virtual outliers in the feature space. So the feature space is if you have your have your image, right, let's just talk about classifier, you feed it through a bunch of neural networks. And then here is the last layer. And all you do at the end is you have a classification head that classifies it into multiple classes. And this right here is just described by a matrix W. This is just a linear layer that goes from the amount of features, I guess D or something like this to the amount of classes C. That's the dimensionality. So in this space at the end, you would do in this space right here, that's the space we've seen in in these diagrams up there. Here is where we would sample the virtual outliers. So what we would do is we would look at our training data, where does our training data fall? And we say, aha, okay, there is class one, two and three, as we had it. Then we'd build a Gaussian mixture model of the training data. Essentially, we'd assume that each class is described well by a high dimensional, by a multivariate Gaussian. They all share the covariance matrix, by the way. And then we would say, well, okay, given that that is the case, which ends up at the situation in the right, we would sample data points from outside of those Gaussians. So that have a sufficiently low probability. So these would be these virtual outliers. We would just sample them anywhere where we where our Gaussian mixture model says that there is no data. But still, we sample according to the Gaussians. So we're not going to be like way out here in undefined space. Just because this is in our support set, we're still going to sample from these Gaussians. But we're going to sample until we get a sample that has a very low likelihood. So we're deliberately going to sample outliers from these Gaussians. And those are going to serve as samples for our outlier classifier. So then the outlier classifier, what it needs to do is it needs to find a decision boundary between these virtual outliers and the data. You can see, draw this right here. So there's going to be a decision boundary. Now, you can see this decision boundary gets quite a bit more complicated than the decision boundary of between the classes, especially, you know, given that we do it in the last layer. So we'll go on in the paper a little bit. What we just said is going to come up in a second here. So they say we assume the feature representation of object instances forms a class conditional multivariate Gaussian distribution. And they state this right here. So every class has a mean, all the classes share a covariance matrix. And they do calculate, they don't learn these things, they do just calculate them from the training data in an online fashion. So this is in the penultimate layer of the neural network, as I just said. Yeah, they compute empirical class mean and covariance of training samples. And they do this in an online, sorry about that, in an online estimation fashion, which means that as they train the network, they collect the training data. And then in an online fashion, they compute these metrics to always be up to date. They do say here, we assume the feature representation is this Gaussian, and they say see figure three, and figure three is a UMAP projection of UMAP visualization of feature embeddings of the Pascal VOC data set. And I'm not sure what they mean by look at figure three. This is a UMAP. This is like a projection, a nonlinear projection into low dimensional space. If I'm not exactly remembering what UMAP does, but for sure, this is a projection. This doesn't convince me that the data is Gaussian. It convinces me that the data is kind of in one place-ish, right? Or it convinces me that all the blue points are closer, or most of the blue points are closer to each other than they are close to, for example, the green points here. Like that is what is convincing to me from this graphic. It is not at all convincing that in the original high dimensional space where they come from, they are somehow a cluster or a Gaussian, even, or even that all of these classes would have the same covariance matrix, even if they were Gaussians. So that is a wild assumption. But it seems to work. So the results of the paper are that they are very, very good at this outlier detection. They reduce false positive rates by a lot. So it seems to work. I'm just saying this does not convince me. Or maybe I don't understand UMAP. Maybe there is something. So here is where they say they sample the virtual outliers from in this feature representation space using the multivariate distributions. So they would simply sample the virtual outliers from the Gaussians, but then evaluate them and only take them if their likelihood is smaller than some epsilon. They say it's sufficiently small so that the sample outliers are near the class boundary. These outliers would then be converted to the output. So this would be the output, the classifier head by the classifier matrix. Now, this is a very interesting example. That is how they sample the outliers. And you know, all good so far. I have a few concerns right here. For example, what you're going to teach the model is, you know, successfully, if in the last layer before the classifier, there is a data point, and that data point is not where the training data is, then if this model works, it will, in fact, recognize it as an outlier. What will not happen, and this seems okay, what will not be the case if that moose right here, for some reason, an earlier layer already confuses it with something. An earlier layer thinks, oh, this, you know, it's four legs, it's probably like it looks like a dog, right, then the moose will come to lie really inside of the dog class, because it would have the features of a dog, which the lower layers would have confused it. So you'd have to have done this technique in one of the lower layers. And there, you could see that this isn't an outlier. But the lower the layers, you go, you know, the less your data, even less your data looks like a Gaussian, I mean, ultimately, you'd have to do it in the input layer, right. And there, it becomes clear that this is just like a distribution of the data that you're trying to approximate. And in the input layer, certainly, this is not Gaussian at all. So I think this only works for specific outliers. If there is an outlier that, as I say, has like the same features as some in distribution data, resulting that in the last layer, they are in like inside of this cluster, then this method will not be able to detect it. Yeah, that is that is kind of my one concern. The other concern I've already said is that this is separating these outliers is naturally a harder task because as well, it essentially amounts to a generative or a distributional model of the data rather than just a discriminative classifier. So how are they incorporating this into training? During training, we still don't know, right, we have, so up here, right, we have our loss right here for the localization, we have a classification loss, which is fine, is good. So our classification loss tells us if we have the class correctly, but we still need a third thing, which is this uncertainty loss. We are going to estimate the uncertainty, which is going to be our measure of how much the model thinks that this is an out of distribution data point or not. And how are they doing it? They are using the log partition function for that. So the log partition function is this thing right here. It's essentially what is at the bottom of the softmax if you use a softmax for classification. So if the f here is the logit of class k, so if this is the output of your classifier, and then you do a softmax in the last layer across your logits, the softmax would look like this, right. So you'd have the class y at the top, and then you'd have that log some x of all the classes at the bottom. So the bottom right here is kind of like a measure of how peaky your distribution is, right. If your logits are, you know, one is just standing out heavily, then that is kind of a measure for low uncertainty, like you're quite sure about what you're doing. And if all the logits are kind of the same, then they are all more even. So this measure is a little bit of an indicator of certainty, right. So this was already shown to be an effective uncertainty measurement for out of distribution detection. So what we're going to do is we're going to use this as an uncertainty loss right here. So what we're going to do is we're going to train, or not to train, we're going to have a logit-based loss. So we're going to say we are going to use a sigmoid. And what we want is we want this measure right here. We want this right here, which is one is the logit and one is one minus the logit. I can't remember which one is which. In any case, we want this to be a in any case, we want this measure to be high for in distribution data and low for out of distribution data or the other way around. We want the uncertainty to be high for out of distribution data and low for in distribution data. So if we get a data point, we'll plug it in to this free energy. Well, the, by the way, this the negative of the log partition function is called the free energy. Sorry, I forgot to mention that that would make some connections to other, even other fields of science. So we're going to take our data point. And we're going to not plug it into the classifier, but just this bottom part of the classifier, right, to measure is the distribution data that we're getting very certain or very uncertain. And then what we want is that if we have a true data point, then we want the we want the uncertainty to be very low. If we have a fake data point, we want the uncertainty to be very high. So by adding this loss right here, by adding this loss, what this does is this trains our classifier to be more certain if the data point is real, and less certain if the data point is fake, which ultimately, right, will result in decision in decision boundaries like this or or certainty estimates like this on the right here. So the certainty estimate on the left would just be if we just train the classifier objective, the thing will get more and more certain as we go away from the classification boundaries. If we look at this certainty measure, and now we explicitly train the model to only be certain around the data, and to be again very uncertain around all the virtual outliers. So that's why you see blue anywhere away from the data. We explicitly train the model to do that. So our uncertainty classifier that we talked about, where was it? This thing right here. Our uncertainty classifier is not in fact an additionally trained model. It is simply us plugging a data point into this uncertainty measure. And during training, we make sure that this measure is low for fake data and high for clean data. Now, this loss, if I see this correctly, this uncertainty loss, initially, it will directly affect this parameter set right here. Since we only generate the fake data in the last layer, the only parameters that are really affected by this loss in that case is the classification weights right here. However, implicitly, obviously, by saying that the true data here must have a high certainty or a low uncertainty, and by contrasting this with the fake data in the last layer, it may also be that through back propagation, the entire network is shaped such that the latent space will be more optimal for doing this classification. However, I cannot conceive super well how all the effects and counter effects and so on are going to work out. But it would be interesting to think a bit more clearly through that. So what we're going to end up with is a probabilistic score for out of distribution detection. Our loss is going to be a mixture of these classification and localization losses and the uncertainty loss added with a given hyperparameter. So this is going to be our detector for in distribution. We simply take predicted or we take an inference sample, we take the predicted bounding box, we'll plug it into this uncertainty estimate right here. So this here is this free energy, we plug it into the sigmoid formula here. And that will give us one, if the classifier is very certain and zero, if it's very uncertain, that this is in distribution data, we can define a threshold, and that's going to be our out of distribution classifier. So that's it for the method. They go through a bunch of results. Now I'll shorten the results by saying they're just very good at everything like at the data sets they try against the baseline, baselines. They do ablations, and particularly noteworthy, for example, here is the false positive rate where lower is better. You can see if they were just to add an outlier class, this would hurt the performance quite a bit, like more than other modifications right here, which I found interesting to see. Yeah, they detect they compare against other outlier detection methods. And they they do have, I believe, some samples right here. Needless to say, I have my concerns, but it does work pretty well. So and I'm just a person that looks at this paper for the first time and hasn't worked in this field at all and hasn't tried anything. So I'm going to give the the the right away to the authors right here. But let me know what you think, and I'll see you next time.
 Outliers, we all know them, we all hate them. How can these data points just be out of distribution,
 not in the training data, things that we haven't seen before, things that we don't even expect?
 Well, they suck. So today we're going to look at what you can do about it. Specifically,
 we're going to look at the paper learning what you don't know by virtual outlier synthesis. This
 paper presents a technique to generate what it calls virtual outliers, which are synthetic data
 points that are out of distribution. The core idea is that rather than trying to come up with data
 space out of distribution samples, this paper comes up with latent space out of distribution samples,
 which is much easier and much more useful. They're then designing a loss that pushes down the energy
 of the model wherever the outliers are and pushes up the energy wherever the data is. This paper is
 really interesting because it presented very successful results on a multitude of benchmarks.
 So definitely this technique looks like it works. However, when I read the paper, I was quite
 critical. I had a lot of criticisms, I had a lot of open questions, and that's why I've invited the
 authors for an interview to the channel. So this video right here is a comprehensive paper review.
 I'll explain in detail what is in the paper, what the method does, what its contributions are,
 what its experimental results look like, what is good about it, and what I think is bad about it.
 Then in the next video released tomorrow, I'll interview the authors of the paper, the authors
 will have seen my review, and therefore are able to respond to any criticism and any questions that
 I had. So be sure to check out the interview part as well, because it was really, really cool to get
 all my questions answered. As always, let me know how I can improve these videos by leaving a comment,
 leave a like if you do like and I'll see you around. Bye bye.
 And this works in the traditional way where you upload audio and you get back the transcription,
 but they can also do this real time. So you get a web socket to their neural network powered backend
 and in real time, it gives you back text for your speech. That's insane. But this is not all they
 have a ton of features on top of that. For example, they can do summarization, they can do topic
 detection, they can do bad word detection, content moderation in your audio. And I have to say,
 this is really good. In fact, I have uploaded this video right here to their API's and the text you
 see on screen is the raw output of that model. So judge yourself how good it is. We'll actually try
 some Swiss German words on it. It is an English model, but we'll just give it a shot. Oh, well,
 isn't that great. So give them a try. They even have a basic free tier at their documentation
 is super extensive. They give you walkthroughs and examples of all the parameters that you can send.
 They have a great blog where they describe different feature sets and different ways of
 applying their technology. And yeah, it's a really cool thing. Now I've only scratched the surface
 right here. They do much more. They have features upon features on this, but it's best you check
 them out yourself. So thank you very much to assembly AI for sponsoring this video is really
 great. Please check them out. A link is in the description and I wish you a lot of fun.
 Hello there today we'll look at VOS learning what you don't know by virtual outlier synthesis by
 Shefeng Du, Zhao Ning Wang, Mu Cai and Yixuan Li. This paper presents a model that can do
 out of distribution detection in object detection networks, but not only in object detection,
 they show it on object detection, but it is a general framework for detecting out of distribution
 data at inference time. If this really works, this could mean a lot for especially for safety
 critical applications, networks that are deployed as a classifier or a detector somewhere. And they
 would be able to recognize accurately when they are presented with something they didn't learn
 at training time, like some out of distribution class. And this particular case on the left here,
 you see an image, which is an object detection network at inference time, it has correctly
 recognized the car on the right hand side. However, it thinks that the moose here is a
 pedestrian, it doesn't even classify all of the moose, but it recognizes there is an object.
 And the class is pedestrian, probably because it hasn't hasn't seen mooses,
 meese. What's the plural of moose? In any case, it hasn't seen a moose or multiple meese
 at training time. And therefore, it cannot classify it. And very often these networks make very,
 very high confidence predictions for classes that they haven't seen. This paper tackles this
 and proposes this technique called virtual outlier synthesis, to which we'll get to in a second. As
 I said, it's a general framework. They demonstrated on object detection, which is a particularly hard
 task, but this could also be applied to image classification. They do make the point that if
 you have an image like this, and you haven't seen the moose class during training, most of the image
 will still be in distribution. Like this will not be a particularly out of distribution image,
 except for that small part with the moose. However, if you do object detection, then the object itself
 here is out of distribution. And maybe that makes actually their tasks as researchers a bit more
 easy, because they are less often in these ambiguous cases where like half the data point
 is out of distribution. In any case, they mentioned here, they that the networks that we currently
 have, they often struggle to handle the unknowns. And they assign high posterior probability for
 out of distribution test inputs. Now, why might that be? If you train a typical classifier,
 the classifier will just attempt to separate classes from each other. You see this here
 in the middle. This is a projection of the last layer of a neural network right before the
 classifier layer. So right before the softmax. So the classification layer, all it can do
 is it can lay linear decision boundaries, essentially, through the distribution of data
 points. So what the model does is it sees three classes right here. So this is class one, this is
 class two, this is class three. And what it needs to do is linearly separate them. So it says, well,
 okay, I'm gonna this is not an ideal color for this. I'm going to just put my decision boundaries
 like this. And now I've essentially separated the classes, because all that is important to a
 classification loss is that, you know, points in class three are away from points in class one and
 away from points in class two. So that also means that the more away from classes one and two I go,
 the better, like the more likely it is to be class three, because all I've ever seen at training is
 samples from class three. And my entire objective was just to make it, to push it away or distinguish
 it, to discriminate it from class one and class two. So obviously, if I go more into the direction
 of class three, the network will become will output a more and more confident number about
 this being class three, even though, as you can see, the data is all in this region right here.
 And out there, there is no data, yet the network is still very, very confident. Red here means
 quite confident. An ideal situation would be if the network was very confident, where the training
 data is right here. However, again, we have the decision boundaries like this. However, if you go
 further out, it will say something like, wait a minute, even though this is not class one, for sure,
 and not class two, for sure, it's most likely class three, but still, I haven't seen any training data
 around that area. So I'm also going to be to just output a low probability or a low confidence score.
 I'm going to say it's class three, but I'm going to assign it a low confidence, because I haven't
 seen actual training data in that vicinity. Now, this all seems intuitive and makes sense and so on.
 Mostly, that is because low dimensionality and high dimensionality data is very different and
 can deceive if you look at it in this in a kind of a very simple projection like this, you as a human,
 you see this data and you go like, of course, that makes total sense. However, this becomes very
 different if you look at high dimensional data. Note that there is a reason why our classifiers
 do the thing on the left, because the thing on the right essentially amounts to like a probabilistic
 model of the data distribution, right? The thing on the right, it has an idea where all the data is,
 right? The thing on the left, it just needs to separate data from each other. Three lines are
 enough for that. The thing on the right actually needs to model the data in the latent space,
 which can become pretty complicated in high dimensions, and it needs some very, very
 distinct assumptions to make it tractable. So the right thing is essentially a generative model of
 the data, like a distributional model of the data, which needs a lot more resources and power and
 could pull away resources from the classification task to be solved. So what does this model do?
 First of all, they have some notation right here, which I found to be...
 Well, let's just first look at the diagram right here. So this is the whole model architecture.
 They have an input over here. So there's input X, right? I'm going to use the green highlighter,
 I guess, for this stuff. There's input X. You can see this is the input image. In general,
 first you have this proposal generator, and that proposal generator will generate bounding boxes.
 So some of these detection networks, they have two stages. First, proposal generation, and then
 a post-processing stage where they assign labels to the proposals. So the proposal generator
 would simply ask, where are objects? Any sort of object. The objectness property, it generalizes
 between objects. So it makes sense to train the object detector to just predict where are bounding
 boxes. In this case, it will predict, well, there is one here, there is an object, and there is an
 object here. And then it will pass on those to the classifier to determine what's in the bounding
 boxes. And you can already see the object detector has done a good job. It detected that this thing
 right here is an object. However, the classifier, what can it do? It has to assign a label. There
 is no option for it to say, no, actually, this isn't an object. And previous methods have tried
 this. They've just added like an extra class for outlier. It usually doesn't work too well,
 because the reason is pretty simple. In order to do that here on the left, you'd have to introduce
 like another line and say, okay, so I'm going to introduce another line, I'm running out of colors
 here, introduce another line, you know, like right here. So this would now be outlier, sorry,
 outlier space. Well, that doesn't cover, that doesn't cover this region or this region, or the
 region back here, right. So having a single class for outliers is sort of useless, because there are
 just so many places where outliers could be, and not just like a single, a single slice of the space.
 So you'd have to have many, you'd actually have to have like a lot. And ultimately, that amounts to
 exactly the situation on the right where, you know, ultimately, you're going to train a classifier
 that is a threshold between low and high density areas. And that's exactly a generative model of
 the data. All right, first stage is the bounding box proposal, this thing right here. Then you pass
 on the bounding box to multiple things. First of all, there is a loss that's simply concerned with
 did you detect the objects correctly. So during training, the proposal generator would simply be
 trained with that loss right here. Now everything here is back propagated, obviously, but that would
 be the main loss to localize the bounding boxes. The second, the second stage here would be the
 assignment of a label, this would be the so called classification head. So that would take the latent
 representation that is generated, including the bounding box, right. So we're going to feed this
 through a neural network. And that will give us a latent representation, this H thing mean that they
 call that the latent representation right before the classification layer, and the classification
 layer would assign a label to it. And that would be the normal way of doing things. And now we
 augment that by a bit. Just to say they formulate this here, as saying we have a data set, the data
 set here contains x is data, b is bounding box and y is labels. So b and y would be the labels,
 right, those would be the things to predict. And then they say they split it up into two things.
 So first of all, the p of the bounding box, and then the one of the label. And I don't think that's
 correct. I think that's a typo right here. I think this should be the probability of the bounding box
 given x, not the label. And this should probably be the probability of the label given x as well
 as the predicted bounding box. Let's call this b hat right here, the predicted bounding box. So b
 hat would be sampled from this. But this is minor, because the rest of the paper essentially treats
 it as I think I write it down. In any case, what they do in addition to that is they also have this
 classifier right here. The classifier that takes into a sample and the bounding box and it tries
 to predict this number g. And g is one if the object is in distribution and g should be zero
 if it's out of distribution. So this is a binary classifier that classifies any sample into in or
 out of distribution, independent of what the classifier head says what class it is. So that
 would amount to the situation on the right, where if you're anywhere in this region right here,
 the classifier would still say, well, that's clearly class three, because that's the region
 of class three. But your other classifier would say yes, but the the outlier probability is very
 high, the in in layer probability is very low for that region. So you can do outlier detection
 at inference time. How do we do this? We do this by generating these virtual outliers during training.
 Virtual outliers are essentially outlier data points that you synthesize. Now, you what you
 could do, and they mentioned that what you could do is you could train like again, you can simply
 train a generative model of the data, and then use that to sample out of distribution data. However,
 they mentioned that synthesizing images in the high dimensional pixel space can be difficult
 to optimize. Instead, our key idea is to synthesize virtual outliers in the feature space.
 So the feature space is if you have your have your image, right, let's just talk about classifier,
 you feed it through a bunch of neural networks. And then here is the last layer. And all you do
 at the end is you have a classification head that classifies it into multiple classes. And this right
 here is just described by a matrix W. This is just a linear layer that goes from the amount of
 features, I guess D or something like this to the amount of classes C. That's the dimensionality.
 So in this space at the end, you would do in this space right here, that's the space we've seen in
 in these diagrams up there. Here is where we would sample the virtual outliers. So what we would do
 is we would look at our training data, where does our training data fall? And we say, aha,
 okay, there is class one, two and three, as we had it. Then we'd build a Gaussian mixture model
 of the training data. Essentially, we'd assume that each class is described well by a high
 dimensional, by a multivariate Gaussian. They all share the covariance matrix, by the way.
 And then we would say, well, okay, given that that is the case, which ends up at the situation
 in the right, we would sample data points from outside of those Gaussians. So that have a
 sufficiently low probability. So these would be these virtual outliers. We would just sample them
 anywhere where we where our Gaussian mixture model says that there is no data. But still,
 we sample according to the Gaussians. So we're not going to be like way out here in undefined space.
 Just because this is in our support set, we're still going to sample from these Gaussians.
 But we're going to sample until we get a sample that has a very low likelihood. So
 we're deliberately going to sample outliers from these Gaussians. And those are going to serve
 as samples for our outlier classifier. So then the outlier classifier, what it needs to do is
 it needs to find a decision boundary between these virtual outliers and the data. You can see,
 draw this right here. So there's going to be a decision boundary. Now, you can see this decision
 boundary gets quite a bit more complicated than the decision boundary of between the classes,
 especially, you know, given that we do it in the last layer. So we'll go on in the paper a little
 bit. What we just said is going to come up in a second here. So they say we assume the feature
 representation of object instances forms a class conditional multivariate Gaussian distribution.
 And they state this right here. So every class has a mean, all the classes share a covariance
 matrix. And they do calculate, they don't learn these things, they do just calculate them from
 the training data in an online fashion. So this is in the penultimate layer of the neural network,
 as I just said. Yeah, they compute empirical class mean and covariance of training samples.
 And they do this in an online, sorry about that, in an online estimation fashion, which means that
 as they train the network, they collect the training data. And then in an online fashion,
 they compute these metrics to always be up to date. They do say here, we assume the feature
 representation is this Gaussian, and they say see figure three, and figure three is a UMAP projection
 of UMAP visualization of feature embeddings of the Pascal VOC data set. And I'm not sure what they
 mean by look at figure three. This is a UMAP. This is like a projection, a nonlinear projection
 into low dimensional space. If I'm not exactly remembering what UMAP does, but for sure, this is
 a projection. This doesn't convince me that the data is Gaussian. It convinces me that the data
 is kind of in one place-ish, right? Or it convinces me that all the blue points are closer, or most of
 the blue points are closer to each other than they are close to, for example, the green points here.
 Like that is what is convincing to me from this graphic. It is not at all convincing that in the
 original high dimensional space where they come from, they are somehow a cluster or a Gaussian,
 even, or even that all of these classes would have the same covariance matrix, even if they were
 Gaussians. So that is a wild assumption. But it seems to work. So the results of the paper are that
 they are very, very good at this outlier detection. They reduce false positive rates by a lot. So
 it seems to work. I'm just saying this does not convince me. Or maybe I don't understand UMAP.
 Maybe there is something. So here is where they say they sample the virtual outliers from in this
 feature representation space using the multivariate distributions. So they would simply sample the
 virtual outliers from the Gaussians, but then evaluate them and only take them if their
 likelihood is smaller than some epsilon. They say it's sufficiently small so that the sample
 outliers are near the class boundary. These outliers would then be converted to the output. So this
 would be the output, the classifier head by the classifier matrix. Now, this is a very interesting
 example. That is how they sample the outliers. And you know, all good so far. I have a few concerns
 right here. For example, what you're going to teach the model is, you know, successfully,
 if in the last layer before the classifier, there is a data point, and that data point
 is not where the training data is, then if this model works, it will, in fact, recognize it as an
 outlier. What will not happen, and this seems okay, what will not be the case if that moose right here,
 for some reason, an earlier layer already confuses it with something. An earlier layer thinks,
 oh, this, you know, it's four legs, it's probably like it looks like a dog, right, then the moose
 will come to lie really inside of the dog class, because it would have the features of a dog,
 which the lower layers would have confused it. So you'd have to have done this technique in one of
 the lower layers. And there, you could see that this isn't an outlier. But the lower the layers,
 you go, you know, the less your data, even less your data looks like a Gaussian, I mean, ultimately,
 you'd have to do it in the input layer, right. And there, it becomes clear that this is just like a
 distribution of the data that you're trying to approximate. And in the input layer, certainly,
 this is not Gaussian at all. So I think this only works for specific outliers. If there is an outlier
 that, as I say, has like the same features as some in distribution data, resulting that in the last
 layer, they are in like inside of this cluster, then this method will not be able to detect it.
 Yeah, that is that is kind of my one concern. The other concern I've already said is that this
 is separating these outliers is naturally a harder task because as well, it essentially amounts to a
 generative or a distributional model of the data rather than just a discriminative classifier.
 So how are they incorporating this into training? During training, we still don't know, right, we
 have, so up here, right, we have our loss right here for the localization, we have a classification
 loss, which is fine, is good. So our classification loss tells us if we have the class correctly,
 but we still need a third thing, which is this uncertainty loss. We are going to estimate
 the uncertainty, which is going to be our measure of how much the model thinks that this is an out
 of distribution data point or not. And how are they doing it? They are using the log partition
 function for that. So the log partition function is this thing right here. It's essentially what
 is at the bottom of the softmax if you use a softmax for classification. So if the f here is the
 logit of class k, so if this is the output of your classifier, and then you do a softmax in the last
 layer across your logits, the softmax would look like this, right. So you'd have the class y at the
 top, and then you'd have that log some x of all the classes at the bottom. So the bottom right here
 is kind of like a measure of how peaky your distribution is, right. If your logits are,
 you know, one is just standing out heavily, then that is kind of a measure for low uncertainty,
 like you're quite sure about what you're doing. And if all the logits are kind of the same, then
 they are all more even. So this measure is a little bit of an indicator of certainty, right.
 So this was already shown to be an effective uncertainty measurement for out of distribution
 detection. So what we're going to do is we're going to use this as an uncertainty loss right here.
 So what we're going to do is we're going to train, or not to train, we're going to have a
 logit-based loss. So we're going to say we are going to use a sigmoid. And what we want is we want
 this measure right here. We want this right here, which is one is the logit and one is one minus
 the logit. I can't remember which one is which. In any case, we want this to be a
 in any case, we want this measure to be high for in distribution data and low for out of distribution
 data or the other way around. We want the uncertainty to be high for out of distribution data
 and low for in distribution data. So if we get a data point, we'll plug it in to this free energy.
 Well, the, by the way, this the negative of the log partition function is called the free energy.
 Sorry, I forgot to mention that that would make some connections to other, even other fields of
 science. So we're going to take our data point. And we're going to not plug it into the classifier,
 but just this bottom part of the classifier, right, to measure is the distribution data
 that we're getting very certain or very uncertain. And then what we want is that if we have a true
 data point, then we want the we want the uncertainty to be very low. If we have a fake data point,
 we want the uncertainty to be very high. So by adding this loss right here, by adding this loss,
 what this does is this trains our classifier to be more certain if the data point is real,
 and less certain if the data point is fake, which ultimately, right, will result in decision
 in decision boundaries like this or or certainty estimates like this on the right here. So the
 certainty estimate on the left would just be if we just train the classifier objective,
 the thing will get more and more certain as we go away from the classification boundaries.
 If we look at this certainty measure, and now we explicitly train the model to only be certain
 around the data, and to be again very uncertain around all the virtual outliers. So that's why
 you see blue anywhere away from the data. We explicitly train the model to do that.
 So our uncertainty classifier that we talked about, where was it? This thing right here.
 Our uncertainty classifier is not in fact an additionally trained model. It is simply us
 plugging a data point into this uncertainty measure. And during training, we make sure
 that this measure is low for fake data and high for clean data. Now, this loss, if I see this
 correctly, this uncertainty loss, initially, it will directly affect this parameter set right here.
 Since we only generate the fake data in the last layer, the only parameters that are really affected
 by this loss in that case is the classification weights right here. However, implicitly,
 obviously, by saying that the true data here must have a high certainty or a low uncertainty,
 and by contrasting this with the fake data in the last layer, it may also be that through back
 propagation, the entire network is shaped such that the latent space will be more optimal for
 doing this classification. However, I cannot conceive super well how all the effects and
 counter effects and so on are going to work out. But it would be interesting to think a bit more
 clearly through that. So what we're going to end up with is a probabilistic score for out of
 distribution detection. Our loss is going to be a mixture of these classification and localization
 losses and the uncertainty loss added with a given hyperparameter. So this is going to be our
 detector for in distribution. We simply take predicted or we take an inference sample, we
 take the predicted bounding box, we'll plug it into this uncertainty estimate right here. So this
 here is this free energy, we plug it into the sigmoid formula here. And that will give us one,
 if the classifier is very certain and zero, if it's very uncertain, that this is in distribution
 data, we can define a threshold, and that's going to be our out of distribution classifier.
 So that's it for the method. They go through a bunch of results. Now I'll shorten the results by
 saying they're just very good at everything like at the data sets they try against the baseline,
 baselines. They do ablations, and particularly noteworthy, for example, here is the false
 positive rate where lower is better. You can see if they were just to add an outlier class,
 this would hurt the performance quite a bit, like more than other modifications right here,
 which I found interesting to see. Yeah, they detect they compare against other outlier detection
 methods. And they they do have, I believe, some samples right here. Needless to say,
 I have my concerns, but it does work pretty well. So and I'm just a person that looks at this paper
 for the first time and hasn't worked in this field at all and hasn't tried anything. So I'm going to
 give the the the right away to the authors right here. But let me know what you think,
 and I'll see you next time.