
# Dataset Card for Bias-amplified Splits for Adversarial NLI

## Table of Contents
- [Table of Contents](#table-of-contents)
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Annotations](#annotations)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Citation Information](#citation-information)

## Dataset Description

- **Repository:** [Fighting Bias with Bias repo](https://github.com/schwartz-lab-nlp/fight-bias-with-bias)
- **Paper:** [arXiv](https://arxiv.org/abs/2305.18917)
- **Point of Contact:** [Yuval Reif](mailto:yuval.reif@mail.huji.ac.il)
- **Original Dataset's Paper:** [ANLI](https://arxiv.org/abs/1910.14599)

### Dataset Summary

Bias-amplified splits is a novel evaluation framework to assess model robustness, by amplifying dataset biases in the training data and challenging models to generalize beyond them. This framework is defined by a bias-amplified training set and a hard, anti-biased test set, which we automatically extract from existing datasets using model-based methods.

Our experiments show that the identified anti-biased examples are naturally challenging for models, and moreover, models trained on bias-amplified data exhibit dramatic performance drops on anti-biased examples, which are not mitigated by common approaches to improve generalization.

Here we apply our framework to Adversarial Natural Language Inference (ANLI), a large-scale NLI benchmark dataset. The dataset was collected via an iterative, adversarial human-and-model-in-the-loop procedure. ANLI is much more difficult than its predecessors including SNLI and MNLI.

Our evaluation framework can be applied to any existing dataset, even those considered obsolete, to test model robustness. We hope our work will guide the development of robust models that do not rely on superficial biases and correlations.

#### Evaluation Results (DeBERTa-large)

##### For splits based on minority examples:

| Training Data \ Test Data | Original test | Anti-biased test |
|---------------------------|---------------|------------------|
| Original training split   | 67.5          | 58.3             |
| Biased training split     | 60.6          | 21.4             |

##### For splits based on partial-input model:

| Training Data \ Test Data | Original test | Anti-biased test |
|---------------------------|---------------|------------------|
| Original training split   | 67.5          | 50.0             |
| Biased training split     | 62.5          | 28.3             |

#### Loading the Data

ANLI contains three rounds of data collection, and each round has train/dev/test splits. We concatenated the splits from all rounds to create one train/dev/test splits.

```
from datasets import load_dataset

# choose which bias detection method to use for the bias-amplified splits: either "minority_examples" or "partial_input"
dataset = load_dataset("bias-amplified-splits/anli", "minority_examples")

# use the biased training split and anti-biased test split
train_dataset = dataset['train.biased']
eval_dataset = dataset['validation.anti_biased']
```

## Dataset Structure

### Data Instances

Data instances are taken directly from ANLI, and re-split into biased and anti-biased subsets. Here is an example of an instance from the dataset:
```
{
  "round": "r1",
  "idx": "20a331ee-cf54-4e8a-9ff9-6152cd679780",
  "premise": "Milton Teagle "Richard" Simmons (born July 12, 1948) is an American fitness guru, actor, and comedian. He promotes weight-loss programs, prominently through his "Sweatin' to the Oldies" line of aerobics videos and is known for his eccentric, flamboyant, and energetic personality.",
  "hypothesis": "Milton Teagle "Richard" Simmons created his "Sweatin' to the Oldies" line of aerobics videos without help or input from anyone else.",
  "label": 1,
  "reason": "The context gives no information as to how the "Sweatin' to the Oldies" videos are produced, Simmons may well produce them alone, or may produce them with a team. The system may have had difficulty with this because it is unlikely that Simmons produced the videos alone."
}
```
 
### Data Fields

- `round`: which round of data collection the example comes from (one of `r1`, `r2` and `r3`) 
- `uid`: unique identifier for the example.
- `premise`: a piece of text 
- `hypothesis`: a piece of text that may be true, false, or whose truth conditions may not be knowable when compared to the premise
- `label`: one of `0`, `1` and `2` (`entailment`, `neutral`, and `contradiction`)
- `reason`: explanation why the label is true (only for some examples).

### Data Splits

Bias-amplified splits require a method to detect *biased* and *anti-biased* examples in datasets. We release bias-amplified splits based created with each of these two methods:
- **Minority examples**: A novel method we introduce that leverages representation learning and clustering for identifying anti-biased *minority examples* (Tu et al., 2020)—examples that defy common statistical patterns found in the rest of the dataset.
- **Partial-input baselines**: A common method for identifying biased examples containing annotation artifacts in a dataset, which examines the performance of models that are restricted to using only part of the input. Such models, if successful, are bound to rely on unintended or spurious patterns in the dataset.

Using each of the two methods, we split each of the original train and test splits into biased and anti-biased subsets. See the [paper](https://arxiv.org/abs/2305.18917) for more details.

#### Minority Examples

| Dataset Split            | Number of Instances in Split |
|--------------------------|------------------------------|
| Train - biased           | 134068                       |
| Train - anti-biased      | 28797                        |
| Validation - biased      | 2317                         |
| Validation - anti-biased | 883                          |
| Test - biased            | 2262                         |
| Test - anti-biased       | 938                          |

#### Partial-input Baselines

| Dataset Split            | Number of Instances in Split |
|--------------------------|------------------------------|
| Train - biased           | 134068                       |
| Train - anti-biased      | 28797                        |
| Validation - biased      | 2634                         |
| Validation - anti-biased | 566                          |
| Test - biased            | 2634                         |
| Test - anti-biased       | 566                          |

## Dataset Creation

### Curation Rationale

NLP models often rely on superficial cues known as *dataset biases* to achieve impressive performance, and can fail on examples where these biases do not hold. To develop more robust, unbiased models, recent work aims to filter bisased examples from training sets. We argue that in order to encourage the development of robust models, we should in fact **amplify** biases in the training sets, while adopting the challenge set approach and making test sets anti-biased. To implement our approach, we introduce a simple framework that can be applied automatically to any existing dataset to use it for testing model robustness.

### Annotations

#### Annotation process

No new annotations are required to create bias-amplified splits. Existing data instances are split into *biased* and *anti-biased* splits based on automatic model-based methods to detect such examples.

## Considerations for Using the Data

### Social Impact of Dataset

Bias-amplified splits were created to promote the development of robust NLP models that do not rely on superficial biases and correlations, and provide more challenging evaluation of existing systems.

### Discussion of Biases

We propose to use bias-amplified splits to complement benchmarks with challenging evaluation settings that test model robustness, in addition to the dataset’s main training and test sets. As such, while existing dataset biases are *amplified* during training with bias-amplified splits, these splits are intended primarily for model evaluation, to expose the bias-exploiting behaviors of models and to identify more robsut models and effective robustness interventions.

## Additional Information

### Dataset Curators

Bias-amplified splits were introduced by Yuval Reif and Roy Schwartz from the [Hebrew University of Jerusalem](https://schwartz-lab-huji.github.io).

ANLI was developed by Adina Williams, Tristan Thrush and Douwe Kiela.

### Citation Information

```
@misc{reif2023fighting,
    title = "Fighting Bias with Bias: Promoting Model Robustness by Amplifying Dataset Biases",
    author = "Yuval Reif and Roy Schwartz",
    month = may,
    year = "2023",
    url = "https://arxiv.org/pdf/2305.18917",
}
```

Source dataset:
```
@article{williams-etal-2020-anlizing,
  title = "ANLIzing the Adversarial Natural Language Inference Dataset",
  author = "Adina Williams and
    Tristan Thrush and
    Douwe Kiela",
  booktitle = "Proceedings of the 5th Annual Meeting of the Society for Computation in Linguistics",
  year = "2022",
  publisher = "Association for Computational Linguistics",
}
```
task_categories:text-classification size_categories:100K<n<1M language:en license:cc-by-nc-4.0 arxiv:2305.18917 arxiv:1910.14599 region:us
{"language": ["en"], "license": "cc-by-nc-4.0", "size_categories": ["100K<n<1M"], "task_categories": ["text-classification"], "pretty_name": "Adversarial NLI", "dataset_info": [{"config_name": "minority_examples", "features": [{"name": "round", "dtype": "string"}, {"name": "uid", "dtype": "string"}, {"name": "premise", "dtype": "string"}, {"name": "hypothesis", "dtype": "string"}, {"name": "label", "dtype": {"class_label": {"names": {"0": "entailment", "1": "neutral", "2": "contradiction"}}}}, {"name": "reason", "dtype": "string"}], "splits": [{"name": "train.biased", "num_bytes": 61260115, "num_examples": 134068}, {"name": "train.anti_biased", "num_bytes": 13246263, "num_examples": 28797}, {"name": "validation.biased", "num_bytes": 1311433, "num_examples": 2317}, {"name": "validation.anti_biased", "num_bytes": 500409, "num_examples": 883}, {"name": "test.biased", "num_bytes": 1284544, "num_examples": 2262}, {"name": "test.anti_biased", "num_bytes": 539798, "num_examples": 938}], "download_size": 86373189, "dataset_size": 78142562}, {"config_name": "partial_input", "features": [{"name": "round", "dtype": "string"}, {"name": "uid", "dtype": "string"}, {"name": "premise", "dtype": "string"}, {"name": "hypothesis", "dtype": "string"}, {"name": "label", "dtype": {"class_label": {"names": {"0": "entailment", "1": "neutral", "2": "contradiction"}}}}, {"name": "reason", "dtype": "string"}], "splits": [{"name": "train.biased", "num_bytes": 60769911, "num_examples": 134068}, {"name": "train.anti_biased", "num_bytes": 13736467, "num_examples": 28797}, {"name": "validation.biased", "num_bytes": 1491254, "num_examples": 2634}, {"name": "validation.anti_biased", "num_bytes": 320588, "num_examples": 566}, {"name": "test.biased", "num_bytes": 1501586, "num_examples": 2634}, {"name": "test.anti_biased", "num_bytes": 322756, "num_examples": 566}], "download_size": 86373189, "dataset_size": 78142562}]}
TAGS
#task_categories-text-classification #size_categories-100K<n<1M #language-English #license-cc-by-nc-4.0 #arxiv-2305.18917 #arxiv-1910.14599 #region-us 

Dataset Card for Bias-amplified Splits for Adversarial NLI
==========================================================


Table of Contents
-----------------


* Table of Contents
* Dataset Description
	+ Dataset Summary
* Dataset Structure
	+ Data Instances
	+ Data Fields
	+ Data Splits
* Dataset Creation
	+ Curation Rationale
	+ Annotations
* Considerations for Using the Data
	+ Social Impact of Dataset
	+ Discussion of Biases
* Additional Information
	+ Dataset Curators
	+ Citation Information


Dataset Description
-------------------


* Repository: Fighting Bias with Bias repo
* Paper: arXiv
* Point of Contact: Yuval Reif
* Original Dataset's Paper: ANLI


### Dataset Summary


Bias-amplified splits is a novel evaluation framework to assess model robustness, by amplifying dataset biases in the training data and challenging models to generalize beyond them. This framework is defined by a bias-amplified training set and a hard, anti-biased test set, which we automatically extract from existing datasets using model-based methods.


Our experiments show that the identified anti-biased examples are naturally challenging for models, and moreover, models trained on bias-amplified data exhibit dramatic performance drops on anti-biased examples, which are not mitigated by common approaches to improve generalization.


Here we apply our framework to Adversarial Natural Language Inference (ANLI), a large-scale NLI benchmark dataset. The dataset was collected via an iterative, adversarial human-and-model-in-the-loop procedure. ANLI is much more difficult than its predecessors including SNLI and MNLI.


Our evaluation framework can be applied to any existing dataset, even those considered obsolete, to test model robustness. We hope our work will guide the development of robust models that do not rely on superficial biases and correlations.


#### Evaluation Results (DeBERTa-large)


##### For splits based on minority examples:


Training Data \ Test Data: Original training split, Original test: 67.5, Anti-biased test: 58.3
Training Data \ Test Data: Biased training split, Original test: 60.6, Anti-biased test: 21.4


##### For splits based on partial-input model:


Training Data \ Test Data: Original training split, Original test: 67.5, Anti-biased test: 50.0
Training Data \ Test Data: Biased training split, Original test: 62.5, Anti-biased test: 28.3


#### Loading the Data


ANLI contains three rounds of data collection, and each round has train/dev/test splits. We concatenated the splits from all rounds to create one train/dev/test splits.


Dataset Structure
-----------------


### Data Instances


Data instances are taken directly from ANLI, and re-split into biased and anti-biased subsets. Here is an example of an instance from the dataset:


### Data Fields


* 'round': which round of data collection the example comes from (one of 'r1', 'r2' and 'r3')
* 'uid': unique identifier for the example.
* 'premise': a piece of text
* 'hypothesis': a piece of text that may be true, false, or whose truth conditions may not be knowable when compared to the premise
* 'label': one of '0', '1' and '2' ('entailment', 'neutral', and 'contradiction')
* 'reason': explanation why the label is true (only for some examples).


### Data Splits


Bias-amplified splits require a method to detect *biased* and *anti-biased* examples in datasets. We release bias-amplified splits based created with each of these two methods:


* Minority examples: A novel method we introduce that leverages representation learning and clustering for identifying anti-biased *minority examples* (Tu et al., 2020)—examples that defy common statistical patterns found in the rest of the dataset.
* Partial-input baselines: A common method for identifying biased examples containing annotation artifacts in a dataset, which examines the performance of models that are restricted to using only part of the input. Such models, if successful, are bound to rely on unintended or spurious patterns in the dataset.


Using each of the two methods, we split each of the original train and test splits into biased and anti-biased subsets. See the paper for more details.


#### Minority Examples



#### Partial-input Baselines



Dataset Creation
----------------


### Curation Rationale


NLP models often rely on superficial cues known as *dataset biases* to achieve impressive performance, and can fail on examples where these biases do not hold. To develop more robust, unbiased models, recent work aims to filter bisased examples from training sets. We argue that in order to encourage the development of robust models, we should in fact amplify biases in the training sets, while adopting the challenge set approach and making test sets anti-biased. To implement our approach, we introduce a simple framework that can be applied automatically to any existing dataset to use it for testing model robustness.


### Annotations


#### Annotation process


No new annotations are required to create bias-amplified splits. Existing data instances are split into *biased* and *anti-biased* splits based on automatic model-based methods to detect such examples.


Considerations for Using the Data
---------------------------------


### Social Impact of Dataset


Bias-amplified splits were created to promote the development of robust NLP models that do not rely on superficial biases and correlations, and provide more challenging evaluation of existing systems.


### Discussion of Biases


We propose to use bias-amplified splits to complement benchmarks with challenging evaluation settings that test model robustness, in addition to the dataset’s main training and test sets. As such, while existing dataset biases are *amplified* during training with bias-amplified splits, these splits are intended primarily for model evaluation, to expose the bias-exploiting behaviors of models and to identify more robsut models and effective robustness interventions.


Additional Information
----------------------


### Dataset Curators


Bias-amplified splits were introduced by Yuval Reif and Roy Schwartz from the Hebrew University of Jerusalem.


ANLI was developed by Adina Williams, Tristan Thrush and Douwe Kiela.


Source dataset:



### Dataset Summary


Bias-amplified splits is a novel evaluation framework to assess model robustness, by amplifying dataset biases in the training data and challenging models to generalize beyond them. This framework is defined by a bias-amplified training set and a hard, anti-biased test set, which we automatically extract from existing datasets using model-based methods.


Our experiments show that the identified anti-biased examples are naturally challenging for models, and moreover, models trained on bias-amplified data exhibit dramatic performance drops on anti-biased examples, which are not mitigated by common approaches to improve generalization.


Here we apply our framework to Adversarial Natural Language Inference (ANLI), a large-scale NLI benchmark dataset. The dataset was collected via an iterative, adversarial human-and-model-in-the-loop procedure. ANLI is much more difficult than its predecessors including SNLI and MNLI.


Our evaluation framework can be applied to any existing dataset, even those considered obsolete, to test model robustness. We hope our work will guide the development of robust models that do not rely on superficial biases and correlations. #### Evaluation Results (DeBERTa-large) ##### For splits based on minority examples:


Training Data \ Test Data: Original training split, Original test: 67.5, Anti-biased test: 58.3
Training Data \ Test Data: Biased training split, Original test: 60.6, Anti-biased test: 21.4 ##### For splits based on partial-input model:


Training Data \ Test Data: Original training split, Original test: 67.5, Anti-biased test: 50.0
Training Data \ Test Data: Biased training split, Original test: 62.5, Anti-biased test: 28.3 #### Loading the Data


ANLI contains three rounds of data collection, and each round has train/dev/test splits. We concatenated the splits from all rounds to create one train/dev/test splits.


Dataset Structure
----------------- ### Data Instances


Data instances are taken directly from ANLI, and re-split into biased and anti-biased subsets. Here is an example of an instance from the dataset: ### Data Fields


* 'round': which round of data collection the example comes from (one of 'r1', 'r2' and 'r3')
* 'uid': unique identifier for the example.
* 'premise': a piece of text
* 'hypothesis': a piece of text that may be true, false, or whose truth conditions may not be knowable when compared to the premise
* 'label': one of '0', '1' and '2' ('entailment', 'neutral', and 'contradiction')
* 'reason': explanation why the label is true (only for some examples). ### Data Splits


Bias-amplified splits require a method to detect *biased* and *anti-biased* examples in datasets. We release bias-amplified splits based created with each of these two methods:


* Minority examples: A novel method we introduce that leverages representation learning and clustering for identifying anti-biased *minority examples* (Tu et al., 2020)—examples that defy common statistical patterns found in the rest of the dataset.
* Partial-input baselines: A common method for identifying biased examples containing annotation artifacts in a dataset, which examines the performance of models that are restricted to using only part of the input. Such models, if successful, are bound to rely on unintended or spurious patterns in the dataset.


Using each of the two methods, we split each of the original train and test splits into biased and anti-biased subsets. See the paper for more details. #### Minority Examples #### Partial-input Baselines



Dataset Creation
---------------- ### Curation Rationale


NLP models often rely on superficial cues known as *dataset biases* to achieve impressive performance, and can fail on examples where these biases do not hold. To develop more robust, unbiased models, recent work aims to filter bisased examples from training sets. We argue that in order to encourage the development of robust models, we should in fact amplify biases in the training sets, while adopting the challenge set approach and making test sets anti-biased. To implement our approach, we introduce a simple framework that can be applied automatically to any existing dataset to use it for testing model robustness. ### Annotations #### Annotation process


No new annotations are required to create bias-amplified splits. Existing data instances are split into *biased* and *anti-biased* splits based on automatic model-based methods to detect such examples.


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset


Bias-amplified splits were created to promote the development of robust NLP models that do not rely on superficial biases and correlations, and provide more challenging evaluation of existing systems. ### Discussion of Biases


We propose to use bias-amplified splits to complement benchmarks with challenging evaluation settings that test model robustness, in addition to the dataset’s main training and test sets. As such, while existing dataset biases are *amplified* during training with bias-amplified splits, these splits are intended primarily for model evaluation, to expose the bias-exploiting behaviors of models and to identify more robsut models and effective robustness interventions.


Additional Information
---------------------- ### Dataset Curators


Bias-amplified splits were introduced by Yuval Reif and Roy Schwartz from the Hebrew University of Jerusalem.


ANLI was developed by Adina Williams, Tristan Thrush and Douwe Kiela.


Source dataset:
TAGS
#task_categories-text-classification #size_categories-100K<n<1M #language-English #license-cc-by-nc-4.0 #arxiv-2305.18917 #arxiv-1910.14599 #region-us 
 ### Dataset Summary


Bias-amplified splits is a novel evaluation framework to assess model robustness, by amplifying dataset biases in the training data and challenging models to generalize beyond them. This framework is defined by a bias-amplified training set and a hard, anti-biased test set, which we automatically extract from existing datasets using model-based methods.


Our experiments show that the identified anti-biased examples are naturally challenging for models, and moreover, models trained on bias-amplified data exhibit dramatic performance drops on anti-biased examples, which are not mitigated by common approaches to improve generalization.


Here we apply our framework to Adversarial Natural Language Inference (ANLI), a large-scale NLI benchmark dataset. The dataset was collected via an iterative, adversarial human-and-model-in-the-loop procedure. ANLI is much more difficult than its predecessors including SNLI and MNLI.


Our evaluation framework can be applied to any existing dataset, even those considered obsolete, to test model robustness. We hope our work will guide the development of robust models that do not rely on superficial biases and correlations. #### Evaluation Results (DeBERTa-large) ##### For splits based on minority examples:


Training Data \ Test Data: Original training split, Original test: 67.5, Anti-biased test: 58.3
Training Data \ Test Data: Biased training split, Original test: 60.6, Anti-biased test: 21.4 ##### For splits based on partial-input model:


Training Data \ Test Data: Original training split, Original test: 67.5, Anti-biased test: 50.0
Training Data \ Test Data: Biased training split, Original test: 62.5, Anti-biased test: 28.3 #### Loading the Data


ANLI contains three rounds of data collection, and each round has train/dev/test splits. We concatenated the splits from all rounds to create one train/dev/test splits.


Dataset Structure
----------------- ### Data Instances


Data instances are taken directly from ANLI, and re-split into biased and anti-biased subsets. Here is an example of an instance from the dataset: ### Data Fields


* 'round': which round of data collection the example comes from (one of 'r1', 'r2' and 'r3')
* 'uid': unique identifier for the example.
* 'premise': a piece of text
* 'hypothesis': a piece of text that may be true, false, or whose truth conditions may not be knowable when compared to the premise
* 'label': one of '0', '1' and '2' ('entailment', 'neutral', and 'contradiction')
* 'reason': explanation why the label is true (only for some examples). ### Data Splits


Bias-amplified splits require a method to detect *biased* and *anti-biased* examples in datasets. We release bias-amplified splits based created with each of these two methods:


* Minority examples: A novel method we introduce that leverages representation learning and clustering for identifying anti-biased *minority examples* (Tu et al., 2020)—examples that defy common statistical patterns found in the rest of the dataset.
* Partial-input baselines: A common method for identifying biased examples containing annotation artifacts in a dataset, which examines the performance of models that are restricted to using only part of the input. Such models, if successful, are bound to rely on unintended or spurious patterns in the dataset.


Using each of the two methods, we split each of the original train and test splits into biased and anti-biased subsets. See the paper for more details. #### Minority Examples #### Partial-input Baselines



Dataset Creation
---------------- ### Curation Rationale


NLP models often rely on superficial cues known as *dataset biases* to achieve impressive performance, and can fail on examples where these biases do not hold. To develop more robust, unbiased models, recent work aims to filter bisased examples from training sets. We argue that in order to encourage the development of robust models, we should in fact amplify biases in the training sets, while adopting the challenge set approach and making test sets anti-biased. To implement our approach, we introduce a simple framework that can be applied automatically to any existing dataset to use it for testing model robustness. ### Annotations #### Annotation process


No new annotations are required to create bias-amplified splits. Existing data instances are split into *biased* and *anti-biased* splits based on automatic model-based methods to detect such examples.


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset


Bias-amplified splits were created to promote the development of robust NLP models that do not rely on superficial biases and correlations, and provide more challenging evaluation of existing systems. ### Discussion of Biases


We propose to use bias-amplified splits to complement benchmarks with challenging evaluation settings that test model robustness, in addition to the dataset’s main training and test sets. As such, while existing dataset biases are *amplified* during training with bias-amplified splits, these splits are intended primarily for model evaluation, to expose the bias-exploiting behaviors of models and to identify more robsut models and effective robustness interventions.


Additional Information
---------------------- ### Dataset Curators


Bias-amplified splits were introduced by Yuval Reif and Roy Schwartz from the Hebrew University of Jerusalem.


ANLI was developed by Adina Williams, Tristan Thrush and Douwe Kiela.


Source dataset:
passage: TAGS
#task_categories-text-classification #size_categories-100K<n<1M #language-English #license-cc-by-nc-4.0 #arxiv-2305.18917 #arxiv-1910.14599 #region-us 
### Dataset Summary


Bias-amplified splits is a novel evaluation framework to assess model robustness, by amplifying dataset biases in the training data and challenging models to generalize beyond them. This framework is defined by a bias-amplified training set and a hard, anti-biased test set, which we automatically extract from existing datasets using model-based methods.


Our experiments show that the identified anti-biased examples are naturally challenging for models, and moreover, models trained on bias-amplified data exhibit dramatic performance drops on anti-biased examples, which are not mitigated by common approaches to improve generalization.


Here we apply our framework to Adversarial Natural Language Inference (ANLI), a large-scale NLI benchmark dataset. The dataset was collected via an iterative, adversarial human-and-model-in-the-loop procedure. ANLI is much more difficult than its predecessors including SNLI and MNLI.


Our evaluation framework can be applied to any existing dataset, even those considered obsolete, to test model robustness. We hope our work will guide the development of robust models that do not rely on superficial biases and correlations.#### Evaluation Results (DeBERTa-large)##### For splits based on minority examples:


Training Data \ Test Data: Original training split, Original test: 67.5, Anti-biased test: 58.3
Training Data \ Test Data: Biased training split, Original test: 60.6, Anti-biased test: 21.4##### For splits based on partial-input model:


Training Data \ Test Data: Original training split, Original test: 67.5, Anti-biased test: 50.0
Training Data \ Test Data: Biased training split, Original test: 62.5, Anti-biased test: 28.3 passage: #### Loading the Data


ANLI contains three rounds of data collection, and each round has train/dev/test splits. We concatenated the splits from all rounds to create one train/dev/test splits.


Dataset Structure
-----------------### Data Instances


Data instances are taken directly from ANLI, and re-split into biased and anti-biased subsets. Here is an example of an instance from the dataset:### Data Fields


* 'round': which round of data collection the example comes from (one of 'r1', 'r2' and 'r3')
* 'uid': unique identifier for the example.
* 'premise': a piece of text
* 'hypothesis': a piece of text that may be true, false, or whose truth conditions may not be knowable when compared to the premise
* 'label': one of '0', '1' and '2' ('entailment', 'neutral', and 'contradiction')
* 'reason': explanation why the label is true (only for some examples).### Data Splits


Bias-amplified splits require a method to detect *biased* and *anti-biased* examples in datasets. We release bias-amplified splits based created with each of these two methods:


* Minority examples: A novel method we introduce that leverages representation learning and clustering for identifying anti-biased *minority examples* (Tu et al., 2020)—examples that defy common statistical patterns found in the rest of the dataset.
* Partial-input baselines: A common method for identifying biased examples containing annotation artifacts in a dataset, which examines the performance of models that are restricted to using only part of the input. Such models, if successful, are bound to rely on unintended or spurious patterns in the dataset.


Using each of the two methods, we split each of the original train and test splits into biased and anti-biased subsets. See the paper for more details.#### Minority Examples#### Partial-input Baselines



Dataset Creation
----------------