
# Dataset Card for CrowS-Pairs

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** [Add homepage URL here if available (unless it's a GitHub repository)]()
- **Repository:** https://github.com/nyu-mll/crows-pairs
- **Paper:** https://aclanthology.org/2020.emnlp-main.154
- **Leaderboard:** [If the dataset supports an active leaderboard, add link here]()
- **Point of Contact:** [If known, name and email of at least one person the reader can contact for questions about the dataset.]()

### Dataset Summary

[More Information Needed]

### Supported Tasks and Leaderboards

[More Information Needed]

### Languages

[More Information Needed]

## Dataset Structure

### Data Instances

[More Information Needed]

### Data Fields

[More Information Needed]

### Data Splits

[More Information Needed]
## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

[More Information Needed]

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

[More Information Needed]

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

CrowS-Pairs is licensed under a [Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-sa/4.0/).

It is created using prompts taken from the [ROCStories corpora](https://cs.rochester.edu/nlp/rocstories/) and the fiction part of [MNLI](https://cims.nyu.edu/~sbowman/multinli/). Please refer to their papers for more details.

### Citation Information

```
@inproceedings{nangia-etal-2020-crows,
    title = "{C}row{S}-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models",
    author = "Nangia, Nikita  and
      Vania, Clara  and
      Bhalerao, Rasika  and
      Bowman, Samuel R.",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2020.emnlp-main.154",
    doi = "10.18653/v1/2020.emnlp-main.154",
    pages = "1953--1967",
}
```

### Contributions

Thanks to [@patil-suraj](https://github.com/patil-suraj) for adding this dataset.
task_categories:text-classification task_ids:text-scoring annotations_creators:crowdsourced language_creators:crowdsourced multilinguality:monolingual size_categories:1K<n<10K source_datasets:original language:en license:cc-by-sa-4.0 bias-evaluation region:us
{"annotations_creators": ["crowdsourced"], "language_creators": ["crowdsourced"], "language": ["en"], "license": ["cc-by-sa-4.0"], "multilinguality": ["monolingual"], "size_categories": ["1K<n<10K"], "source_datasets": ["original"], "task_categories": ["text-classification"], "task_ids": ["text-scoring"], "paperswithcode_id": "crows-pairs", "pretty_name": "CrowS-Pairs", "tags": ["bias-evaluation"], "dataset_info": {"features": [{"name": "id", "dtype": "int32"}, {"name": "sent_more", "dtype": "string"}, {"name": "sent_less", "dtype": "string"}, {"name": "stereo_antistereo", "dtype": {"class_label": {"names": {"0": "stereo", "1": "antistereo"}}}}, {"name": "bias_type", "dtype": {"class_label": {"names": {"0": "race-color", "1": "socioeconomic", "2": "gender", "3": "disability", "4": "nationality", "5": "sexual-orientation", "6": "physical-appearance", "7": "religion", "8": "age"}}}}, {"name": "annotations", "sequence": {"sequence": {"class_label": {"names": {"0": "race-color", "1": "socioeconomic", "2": "gender", "3": "disability", "4": "nationality", "5": "sexual-orientation", "6": "physical-appearance", "7": "religion", "8": "age"}}}}}, {"name": "anon_writer", "dtype": "string"}, {"name": "anon_annotators", "sequence": "string"}], "config_name": "crows_pairs", "splits": [{"name": "test", "num_bytes": 419976, "num_examples": 1508}], "download_size": 437764, "dataset_size": 419976}}
TAGS
#task_categories-text-classification #task_ids-text-scoring #annotations_creators-crowdsourced #language_creators-crowdsourced #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-English #license-cc-by-sa-4.0 #bias-evaluation #region-us 


# Dataset Card for CrowS-Pairs

## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions

## Dataset Description

- Homepage: [Add homepage URL here if available (unless it's a GitHub repository)]()
- Repository: URL
- Paper: URL
- Leaderboard: [If the dataset supports an active leaderboard, add link here]()
- Point of Contact: [If known, name and email of at least one person the reader can contact for questions about the dataset.]()

### Dataset Summary



### Supported Tasks and Leaderboards



### Languages



## Dataset Structure

### Data Instances



### Data Fields



### Data Splits


## Dataset Creation

### Curation Rationale



### Source Data



#### Initial Data Collection and Normalization



#### Who are the source language producers?



### Annotations



#### Annotation process



#### Who are the annotators?



### Personal and Sensitive Information



## Considerations for Using the Data

### Social Impact of Dataset



### Discussion of Biases



### Other Known Limitations



## Additional Information

### Dataset Curators



### Licensing Information

CrowS-Pairs is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

It is created using prompts taken from the ROCStories corpora and the fiction part of MNLI. Please refer to their papers for more details.





### Contributions

Thanks to @patil-suraj for adding this dataset.
# Dataset Card for CrowS-Pairs ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions ## Dataset Description

- Homepage: [Add homepage URL here if available (unless it's a GitHub repository)]()
- Repository: URL
- Paper: URL
- Leaderboard: [If the dataset supports an active leaderboard, add link here]()
- Point of Contact: [If known, name and email of at least one person the reader can contact for questions about the dataset.]() ### Dataset Summary ### Supported Tasks and Leaderboards ### Languages ## Dataset Structure ### Data Instances ### Data Fields ### Data Splits ## Dataset Creation ### Curation Rationale ### Source Data #### Initial Data Collection and Normalization #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information ## Considerations for Using the Data ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations ## Additional Information ### Dataset Curators ### Licensing Information

CrowS-Pairs is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

It is created using prompts taken from the ROCStories corpora and the fiction part of MNLI. Please refer to their papers for more details. ### Contributions

Thanks to @patil-suraj for adding this dataset.
TAGS
#task_categories-text-classification #task_ids-text-scoring #annotations_creators-crowdsourced #language_creators-crowdsourced #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-English #license-cc-by-sa-4.0 #bias-evaluation #region-us 
 # Dataset Card for CrowS-Pairs ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions ## Dataset Description

- Homepage: [Add homepage URL here if available (unless it's a GitHub repository)]()
- Repository: URL
- Paper: URL
- Leaderboard: [If the dataset supports an active leaderboard, add link here]()
- Point of Contact: [If known, name and email of at least one person the reader can contact for questions about the dataset.]() ### Dataset Summary ### Supported Tasks and Leaderboards ### Languages ## Dataset Structure ### Data Instances ### Data Fields ### Data Splits ## Dataset Creation ### Curation Rationale ### Source Data #### Initial Data Collection and Normalization #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information ## Considerations for Using the Data ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations ## Additional Information ### Dataset Curators ### Licensing Information

CrowS-Pairs is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

It is created using prompts taken from the ROCStories corpora and the fiction part of MNLI. Please refer to their papers for more details. ### Contributions

Thanks to @patil-suraj for adding this dataset.