Evaluating NLP Models via Contrast Sets
deep learning machine learning nlp natural language processing arxiv attention evaluation cheat easy hard adversarial counterfactual hand-crafted test set supervised
Current NLP models are often "cheating" on supervised learning tasks by exploiting correlations that arise from the particularities of the dataset. Therefore they often fail to learn the original intent of the dataset creators. This paper argues that NLP models should be evaluated on Contrast Sets, which are hand-crafted perturbations by the dataset authors that capture their intent in a meaningful way.

https://arxiv.org/abs/2004.02709

Abstract:
Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities. We propose a new annotation paradigm for NLP that helps to close systematic gaps in the test data. In particular, after a dataset is constructed, we recommend that the dataset authors manually perturb the test instances in small but meaningful ways that (typically) change the gold label, creating contrast sets. Contrast sets provide a local view of a model's decision boundary, which can be used to more accurately evaluate a model's true linguistic capabilities. We demonstrate the efficacy of contrast sets by creating them for 10 diverse NLP datasets (e.g., DROP reading comprehension, UD parsing, IMDb sentiment analysis). Although our contrast sets are not explicitly adversarial, model performance is significantly lower on them than on the original test sets---up to 25\% in some cases. We release our contrast sets as new evaluation benchmarks and encourage future dataset construction efforts to follow similar annotation processes.

Authors: Matt Gardner, Yoav Artzi, Victoria Basmova, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hanna Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
 Hi there! Today we're looking at evaluating NLP models via contrast sets. These are too many authors from too many places for me to read out. We'll just jump right into the problem. What is the problem? Let's jump into the solution. Here you see a visual question answering task. Visual question answering in this case. You have two pictures right here. Picture one, picture two and a sentence. Two similarly colored and similarly posed chow dogs are face-to-face in one image. I guess the task here is to have the system answer. Is this correct or incorrect? As you see here I believe that's a correct statement. Or you're maybe tasked to ask which is the image that this applies to. Is it image one or image two? Of course here it's image one. The problem with such systems is that there are a lot of easy things that the models can do that will usually get them the answer. What we like to imagine is that the model will look at this and recognize that this is a dog here. This is a dog. Here is its face and this is a dog and here is its face. It will see there's a count. There's two of them. There's two of them. There's a notion of face and there's notion of pose and so on. Usually there are tricks that the models can do to get this easier. For example I know that in a particular visual question answering system whenever there is a question of what is the ground covered in or something like this. The answer is always snow. You don't even have to look at the image. Similarly there are a lot of these kind of tricks that the models learn and the authors recognize correctly that this is mostly a data set problem. Usually what you do in these data sets is you have an image that you scrape from the web or something and it has some mountains and then there's snow on the mountains, on the ground. You give this to a bunch of mechanical turks or someone like a raider and you instruct them. You produce a question to this image. You give them a couple of examples and they're usually kind of lazy and they will just look at it and be like what questions could I ask? You need to ask something. Usually the instructions are it must be visual and it must maybe be answerable with a one word answer or something like this. Or it must be a multiple choice question. There are these number of instructions and they will usually be like what's kind of special about this picture? There's snow so I'm gonna ask about that. Snow right? The problem is mainly the process of data set generation. That will lead to biases and easy solutions for the models where the models will simply learn statistical correlations between things and the intention. We have a big divergence between the intention of what the data set creators want. The intention is in this case is visual understanding, visual of the world. There's a big difference between this and between how the data set is really constructed. The authors are trying to address this with what they call contrast sets. They say you get out of this process a data set. You get a training data set and a test data set. Maybe here a smaller test data set. What they say is what we should do is we should additionally have these things called contrast sets. This is train and this is test. Usually these two come from the same distribution. You simply make them and then you split them somehow and you take the test from the train. But these here are not from the same distribution. This is the contrast. What they argue is that the authors of the data set should create the contrast set. You see that there's a split here where the data set comes from. They argue that the authors of the data set with knowing what intention they have, they should create the contrast data set manually by hand in order to make really hard examples that show what they want out of a system. They capture this here in their example. If we go back to the example, here are things. They suggest to do this via perturbations. What they would do is they would start at this example up here. They would start and they would perturb it textually or via image. They would perturb it to make it change the gold label. This is different from adversarial examples. In adversarial examples you would want to perturb a sample such that it is still the same but to the classifier it's different. Here you have the opposite gold. You want to make something that is means kind of the opposite but you want to test whether your classifier can pick up on that. In this case the one example would be two similarly colored and similarly posed cats instead of dogs are face to face in one image. That would change the label. Whereas before the answer was yes that's a correct sentence. Now it's no that's not a correct sentence. There are no cats in these images. Also here three similarly colored dogs. The intention of the authors, you have to view it through this lens, the intention here is that the system can recognize the species of the entities in the images. The system can count and the system can compare right compare in this case colors. You want to kind of make perturbations on these attributes from a test image. You can also think about image perturbations where you keep the sentence but you modify the image such that there are still two dogs and they're still facing each other. But they're not similarly colored anymore. So the similarly colored here would be the attribute that where before it was true now it's false with the new image. You get the gist that the people that created the data set that know their intention will create manually these samples. The authors they propose a new metric to track this but essentially the authors propose how well the models do on these contrast sets will be a reflection. It should be kind of an additional thing that people do with their NLP models. Alright so you get the picture. That is I believe the entire gist of this paper and I have some problems. First of all here they say alright let's give a toy example in two dimensions. Say you have this data set right and the red one is the correct decision boundary right and you want to capture that but because you only have limited training data and because you in in this generation processes you have systematic biases. So if we had non-systematic biases we would think that okay we maybe pick this and this one and this one here and this one here and this one here right. We don't get all of them but we kind of get an IID sample right. That wouldn't be so much of a problem. You could still kind of recover the decision boundary but because we have systematic biases the authors argue we actually introduce biases. So the systematic bias here is that we would of the blue ones we would only capture things on this sorry on the on this layer up here and of the red ones orange ones we'd only capture things of the level down here and thereby we introduce the kind of data set, the bias. It doesn't require this complex decision boundary anymore. Right and if we now the problem is if we collect the data set like this and we simply say well these ones are the test set and these ones are the train set right it will generalize well to the test set but it will not generalize well to what we actually want and therefore the authors say well if we introduce these contrast sets here then you see that the decision boundary that we found will not perform well on these contrast sets right. So they would say we take one example of the test set right. This is you can see this is this example right here and we would perturb it to make it change its label or in this case one of them where the label remains. We would kind of perturb it meaningfully and yeah so as I said I have multiple problems with this. First 2D toy examples very very bad for NLP models. First of all low-dimensional intuition does not generalize to high-dimensional intuition like very very little. Second of all even worse usually these NLP models have so many parameters much more parameters than you have data set which means that your decision boundary is incidentally going to be simple even if you had all the data you could possibly want. It's just a very different kind of problem and then the next problem is if even with by doing this contrast set and you already see it here right you already see it you can only kind of bicker about the data okay but with the contrast that you only really capture this one aspect so if that was actually well adhered to you could measure very locally whether or not this this would work or not and the ability to come up with meaningful contrast sets to ever capture what the model is doing is almost impossible because you have to create them manually and then you suggest that the authors themselves make these contrast sets. Remember the authors are the ones that gave these instructions right these instructions right here the authors provided them to the to the data set annotators so the authors will probably be even more biased if they have to do their own right if they have to now create their own contrast examples they will probably even though they know their intention they will probably be like more biased than if you at least this here at least this here is a distributed process across people right so you get things that you wouldn't have thought of but if just the three authors of the date of the paper make the contrast examples I would argue that that's an even more biased measure often so all of this it just strikes me as as the paper is basically saying let's try on a few things and I think the fundamental problem is much much deeper and it goes with this intention part like I get it the the visual question answering data set doesn't capture the doesn't capture what you want it doesn't make the model suddenly understand that there are dogs and there are species of animal and so on it simply makes it correlate things but that's what deep learning and especially NLP does so right it's like it's like saying you you build a build an image net classifier and it can't fly and identify if I try it on my tests that it requires my computer to fly and my image net model can't do this then it doesn't serve my intention right and I mean it's it's a crass example but ultimately you the correct approach should be to better encapsulate your intention into the data set generating process and then correctly interpreting the results that mean okay on this data set as far as we can tell the way we created it this is the performance of the model it doesn't the model will never learn to fulfill your intention and I get it that's what you're saying but still even with this contrast set I think it's a really bad measure to formally propose it's I think you should much more propose how is the data set generating process different from what you want and what are the limitations there right and so that's that that I think that will lead to much more meaningful meaningful results than simply the authors providing a few manually put examples that they feel capture their intention it will not will not the reason we do deep learning instead of straightforward if else programming is because we cannot capture even our intentions and therefore data set generation is the only is the only method we have so to say all right so ultimately I believe these these whole NLP especially the visual question answering and so on the natural language understanding part needs to have a grounding so ultimately I think grounding grounded NLP it means basically that you're not only doing NLP which is simply you take text and you take images and you correlate them somehow right you just make a statistical connection grounded NLP models is the hope that you could build something that actually understands the world understands that there's entities that is interacted there's something like a pose that there is something like what the color means right what a dog is and so on and as entities I think we're not there yet and I think that will be the ultimate solution to these kind of tasks not not any sort of local very local very low dimensional perturbation I mean yeah let's say you create a contrast set you will be able to capture one tiny little bit of your intention one tiny little bit even though you know your intention you will capture a tiny little bit all of the thousand other degrees of freedom of your own intention you won't be able in there to capture in the contrast set I guarantee you all right that was my quarrels with that I invite you to read the whole paper they actually do this for NLP datasets it's a lot of work and they show that the models perform much worse on their contrast sets and interestingly the humans don't the humans are able to solve the contrast set of course of course because you tell the humans what the task is right that's like humans succeed on contrasts at like how surprising what you should do is you should just provide the humans with the data set not tell them what the task is even worse just provide them with the encoded data set like not the text itself but actually the token IDs right and then and then make them do the thing and the humans will just as well make a statistical correlation between the tokens and the images or whatnot and the humans will fail just as well on the test on these contrast sets because the humans maybe they'll figure out what the task is but probably not so humans succeed on contrasts at how surprising you tell them the intention while you don't tell it to the model yes I see critical but yeah please read the paper it's an interesting paper and with that goodbye
 Hi there! Today we're looking at evaluating NLP models via contrast sets.
 These are too many authors from too many places for me to read out.
 We'll just jump right into the problem. What is the problem? Let's jump into
 the solution. Here you see a visual question answering task. Visual question
 answering in this case. You have two pictures right here. Picture one, picture
 two and a sentence. Two similarly colored and similarly posed chow dogs are
 face-to-face in one image. I guess the task here is to have the
 system answer. Is this correct or incorrect? As you see here I believe
 that's a correct statement. Or you're maybe tasked to ask which is the
 image that this applies to. Is it image one or image two? Of course
 here it's image one. The problem with such systems is that there are a
 lot of easy things that the models can do that will usually get them the
 answer. What we like to imagine is that the model will look at this and recognize
 that this is a dog here. This is a dog. Here is its face and this is a dog and
 here is its face. It will see there's a count. There's two of them.
 There's two of them. There's a notion of face and there's notion of pose and so
 on. Usually there are tricks that the models can do to get this easier.
 For example I know that in a particular visual question answering system
 whenever there is a question of what is the ground covered in or something like
 this. The answer is always snow. You don't even have to look at the image.
 Similarly there are a lot of these kind of tricks that the models learn and the
 authors recognize correctly that this is mostly a data set problem.
 Usually what you do in these data sets is you have an image
 that you scrape from the web or something
 and it has some mountains and then there's snow on the mountains, on the ground.
 You give this to a bunch of mechanical turks or someone like a raider and you
 instruct them. You produce a question to this image. You give them a couple of
 examples and they're usually kind of lazy and they will just look at it and
 be like what questions could I ask? You need to ask something.
 Usually the instructions are it must be visual and it must maybe be answerable
 with a one word answer or something like this. Or it must be a
 multiple choice question. There are these number of instructions and they will
 usually be like what's kind of special about this picture? There's snow
 so I'm gonna ask about that. Snow right? The problem is mainly the process
 of data set generation. That will lead to biases and easy
 solutions for the models where the models will simply learn
 statistical correlations between things and the intention. We have a big
 divergence between the intention of what the data set creators
 want. The intention is in this case is visual understanding, visual of the
 world. There's a big difference between this and between how the data
 set is really constructed. The authors are trying to address this with
 what they call contrast sets. They say you get out of this process
 a data set. You get a training data set and a test data set.
 Maybe here a smaller test data set. What they say is what we should do is we
 should additionally have these things called contrast sets. This is train
 and this is test. Usually these two come from the same distribution. You
 simply make them and then you split them somehow and you take the test from the
 train. But these here are not from the same distribution. This is the contrast.
 What they argue is that the authors of the data set should create the contrast
 set. You see that there's a split here where the data set comes from.
 They argue that the authors of the data set with knowing what intention they
 have, they should create the contrast data set manually by hand in order to
 make really hard examples that show what they want out of a system.
 They capture this here in their example. If we go back to the example, here
 are things. They suggest to do this via perturbations. What they would do
 is they would start at this example up here. They would start and they would
 perturb it textually or via image. They would perturb it to make it change
 the gold label. This is different from adversarial examples. In
 adversarial examples you would want to perturb a sample such that it is still
 the same but to the classifier it's different. Here you have the opposite gold.
 You want to make something that is means kind of the opposite but you want to
 test whether your classifier can pick up on that. In this case the one example
 would be two similarly colored and similarly posed cats instead of dogs
 are face to face in one image. That would change the label. Whereas
 before the answer was yes that's a correct sentence. Now it's no that's not
 a correct sentence. There are no cats in these images. Also here three similarly
 colored dogs. The intention of the authors, you have to view it through
 this lens, the intention here is that the system can recognize the species of
 the entities in the images. The system can count and the system can compare
 right compare in this case colors. You want to kind of make perturbations on
 these attributes from a test image. You can also think about image
 perturbations where you keep the sentence but you modify the image such
 that there are still two dogs and they're still facing each other.
 But they're not similarly colored anymore. So the similarly
 colored here would be the attribute that where before it was true now it's false
 with the new image. You get the gist that the people that created the
 data set that know their intention will create manually these samples. The
 authors they propose a new metric to track this but essentially the authors
 propose how well the models do on these contrast sets will be a reflection.
 It should be kind of an additional thing that people do with their NLP
 models. Alright so you get the picture. That is I believe the entire gist of
 this paper and I have some problems. First of all here they say alright let's
 give a toy example in two dimensions. Say you have this data set right and the red
 one is the correct decision boundary right and you want to capture that but
 because you only have limited training data and because you in in this
 generation processes you have systematic biases. So if we had non-systematic
 biases we would think that okay we maybe pick this and this one and this one here
 and this one here and this one here right. We don't get all of them but we
 kind of get an IID sample right. That wouldn't be so much of a problem. You
 could still kind of recover the decision boundary but because we have
 systematic biases the authors argue we actually introduce biases. So the
 systematic bias here is that we would of the blue ones we would only capture
 things on this sorry on the on this layer up here and of the red ones orange
 ones we'd only capture things of the level down here and thereby we introduce
 the kind of data set, the bias. It doesn't require this complex decision
 boundary anymore. Right and if we now the problem is if we collect the data set
 like this and we simply say well these ones are the test set and these ones
 are the train set right it will generalize well to the test set but it
 will not generalize well to what we actually want and therefore the authors
 say well if we introduce these contrast sets here then you see that the decision
 boundary that we found will not perform well on these contrast sets right. So
 they would say we take one example of the test set right. This is you can see
 this is this example right here and we would perturb it to make it change its
 label or in this case one of them where the label remains. We would kind of
 perturb it meaningfully and yeah so as I said I have multiple problems with this.
 First 2D toy examples very very bad for NLP models. First of all low-dimensional
 intuition does not generalize to high-dimensional intuition like very very
 little. Second of all even worse usually these NLP models have so many parameters
 much more parameters than you have data set which means that your decision
 boundary is incidentally going to be simple even if you had all the data you
 could possibly want. It's just a very different kind of problem and then the
 next problem is if even with by doing this contrast set and you already see it
 here right you already see it you can only kind of bicker about the data okay
 but with the contrast that you only really capture this one aspect so if
 that was actually well adhered to you could measure very locally whether or
 not this this would work or not and the ability to come up with meaningful
 contrast sets to ever capture what the model is doing is almost impossible
 because you have to create them manually and then you suggest that the authors
 themselves make these contrast sets. Remember the authors are the ones that
 gave these instructions right these instructions right here the authors
 provided them to the to the data set annotators so the authors will probably
 be even more biased if they have to do their own right if they have to now
 create their own contrast examples they will probably even though they know
 their intention they will probably be like more biased than if you at least
 this here at least this here is a distributed process across people right
 so you get things that you wouldn't have thought of but if just the three authors
 of the date of the paper make the contrast examples I would argue that
 that's an even more biased measure often so all of this it just strikes me as as
 the paper is basically saying let's try on a few things and I think the
 fundamental problem is much much deeper and it goes with this intention part
 like I get it the the visual question answering data set doesn't capture the
 doesn't capture what you want it doesn't make the model suddenly understand that
 there are dogs and there are species of animal and so on it simply makes it
 correlate things but that's what deep learning and especially NLP does so
 right it's like it's like saying you you build a build an image net classifier
 and it can't fly and identify if I try it on my tests that it requires my
 computer to fly and my image net model can't do this then it doesn't serve my
 intention right and I mean it's it's a crass example but ultimately you the
 correct approach should be to better encapsulate your intention into the
 data set generating process and then correctly interpreting the results that
 mean okay on this data set as far as we can tell the way we created it this is
 the performance of the model it doesn't the model will never learn to fulfill
 your intention and I get it that's what you're saying but still even with this
 contrast set I think it's a really bad measure to formally propose it's I think
 you should much more propose how is the data set generating process different
 from what you want and what are the limitations there right and so that's
 that that I think that will lead to much more meaningful meaningful results than
 simply the authors providing a few manually put examples that they feel
 capture their intention it will not will not the reason we do deep learning
 instead of straightforward if else programming is because we cannot
 capture even our intentions and therefore data set generation is the
 only is the only method we have so to say all right so ultimately I believe
 these these whole NLP especially the visual question answering and so on the
 natural language understanding part needs to have a grounding so ultimately I
 think grounding grounded NLP it means basically that you're not only doing NLP
 which is simply you take text and you take images and you correlate them
 somehow right you just make a statistical connection grounded NLP
 models is the hope that you could build something that actually understands the
 world understands that there's entities that is interacted there's something
 like a pose that there is something like what the color means right what a dog is
 and so on and as entities I think we're not there yet and I think that will be
 the ultimate solution to these kind of tasks not not any sort of local very
 local very low dimensional perturbation I mean yeah let's say you create a
 contrast set you will be able to capture one tiny little bit of your intention
 one tiny little bit even though you know your intention you will capture a tiny
 little bit all of the thousand other degrees of freedom of your own intention
 you won't be able in there to capture in the contrast set I guarantee you all
 right that was my quarrels with that I invite you to read the whole paper they
 actually do this for NLP datasets it's a lot of work and they show that the
 models perform much worse on their contrast sets and interestingly the
 humans don't the humans are able to solve the contrast set of course of
 course because you tell the humans what the task is right that's like humans
 succeed on contrasts at like how surprising what you should do is you
 should just provide the humans with the data set not tell them what the task is
 even worse just provide them with the encoded data set like not the text
 itself but actually the token IDs right and then and then make them do the thing
 and the humans will just as well make a statistical correlation between the
 tokens and the images or whatnot and the humans will fail just as well on the
 test on these contrast sets because the humans maybe they'll figure out what the
 task is but probably not so humans succeed on contrasts at how surprising
 you tell them the intention while you don't tell it to the model yes I see
 critical but yeah please read the paper it's an interesting paper and with that
 goodbye