
# Dataset Card for Dataset Name

## Dataset Description

- **Homepage:** 
- **Repository:** https://github.com/mskandalis/gqnli-french
- **Paper:** 
- **Leaderboard:** 
- **Point of Contact:** 

### Dataset Summary

This repository contains a manually translated French version of the [GQNLI](https://github.com/ruixiangcui/GQNLI) challenge dataset, originally written in English. GQNLI is an evaluation corpus that is aimed for testing language model's generalized quantifier reasoning ability.

### Supported Tasks and Leaderboards

This dataset can be used for the task of Natural Language Inference (NLI), also known as Recognizing Textual Entailment (RTE), which is a sentence-pair classification task.

## Dataset Structure

### Data Fields

- `uid`: Index number.
- `premise`: The translated premise in the target language.
- `hypothesis`: The translated premise in the target language.
- `label`: The classification label, with possible values 0 (`entailment`), 1 (`neutral`), 2 (`contradiction`).
- `label_text`: The classification label, with possible values `entailment` (0), `neutral` (1), `contradiction` (2).
- `premise_original`: The original premise from the English source dataset.
- `hypothesis_original`: The original hypothesis from the English source dataset.

### Data Splits

|     name    |entailment|neutral|contradiction|
|-------------|---------:|------:|------------:|
|     test    |    97    |  100  |     103     |

## Additional Information

### Citation Information

**BibTeX:**

````BibTeX
@inproceedings{cui-etal-2022-generalized-quantifiers,
    title = "Generalized Quantifiers as a Source of Error in Multilingual {NLU} Benchmarks",
    author = "Cui, Ruixiang  and
      Hershcovich, Daniel  and
      S{\o}gaard, Anders",
    booktitle = "Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
    month = jul,
    year = "2022",
    address = "Seattle, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.naacl-main.359",
    doi = "10.18653/v1/2022.naacl-main.359",
    pages = "4875--4893",
    abstract = "Logical approaches to representing language have developed and evaluated computational models of quantifier words since the 19th century, but today{'}s NLU models still struggle to capture their semantics. We rely on Generalized Quantifier Theory for language-independent representations of the semantics of quantifier words, to quantify their contribution to the errors of NLU models. We find that quantifiers are pervasive in NLU benchmarks, and their occurrence at test time is associated with performance drops. Multilingual models also exhibit unsatisfying quantifier reasoning abilities, but not necessarily worse for non-English languages. To facilitate directly-targeted probing, we present an adversarial generalized quantifier NLI task (GQNLI) and show that pre-trained language models have a clear lack of robustness in generalized quantifier reasoning.",
}
````

**ACL:**

Ruixiang Cui, Daniel Hershcovich, and Anders Søgaard. 2022. [Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks](https://aclanthology.org/2022.naacl-main.359). In *Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 4875–4893, Seattle, United States. Association for Computational Linguistics.

### Acknowledgements

This work was supported by the Defence Innovation Agency (AID) of the Directorate General of Armament (DGA) of the French Ministry of Armed Forces, and by the ICO, _Institut Cybersécurité Occitanie_, funded by Région Occitanie, France.
task_categories:text-classification task_ids:natural-language-inference task_ids:multi-input-text-classification size_categories:n<1K language:fr license:cc-by-4.0 region:us
{"language": ["fr"], "license": "cc-by-4.0", "size_categories": ["n<1K"], "task_categories": ["text-classification"], "task_ids": ["natural-language-inference", "multi-input-text-classification"]}
TAGS
#task_categories-text-classification #task_ids-natural-language-inference #task_ids-multi-input-text-classification #size_categories-n<1K #language-French #license-cc-by-4.0 #region-us 

Dataset Card for Dataset Name
=============================


Dataset Description
-------------------


* Homepage:
* Repository: URL
* Paper:
* Leaderboard:
* Point of Contact:


### Dataset Summary


This repository contains a manually translated French version of the GQNLI challenge dataset, originally written in English. GQNLI is an evaluation corpus that is aimed for testing language model's generalized quantifier reasoning ability.


### Supported Tasks and Leaderboards


This dataset can be used for the task of Natural Language Inference (NLI), also known as Recognizing Textual Entailment (RTE), which is a sentence-pair classification task.


Dataset Structure
-----------------


### Data Fields


* 'uid': Index number.
* 'premise': The translated premise in the target language.
* 'hypothesis': The translated premise in the target language.
* 'label': The classification label, with possible values 0 ('entailment'), 1 ('neutral'), 2 ('contradiction').
* 'label\_text': The classification label, with possible values 'entailment' (0), 'neutral' (1), 'contradiction' (2).
* 'premise\_original': The original premise from the English source dataset.
* 'hypothesis\_original': The original hypothesis from the English source dataset.


### Data Splits



Additional Information
----------------------


BibTeX:


'


ACL:


Ruixiang Cui, Daniel Hershcovich, and Anders Søgaard. 2022. Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks. In *Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 4875–4893, Seattle, United States. Association for Computational Linguistics.


### Acknowledgements


This work was supported by the Defence Innovation Agency (AID) of the Directorate General of Armament (DGA) of the French Ministry of Armed Forces, and by the ICO, *Institut Cybersécurité Occitanie*, funded by Région Occitanie, France.



### Dataset Summary


This repository contains a manually translated French version of the GQNLI challenge dataset, originally written in English. GQNLI is an evaluation corpus that is aimed for testing language model's generalized quantifier reasoning ability. ### Supported Tasks and Leaderboards


This dataset can be used for the task of Natural Language Inference (NLI), also known as Recognizing Textual Entailment (RTE), which is a sentence-pair classification task.


Dataset Structure
----------------- ### Data Fields


* 'uid': Index number.
* 'premise': The translated premise in the target language.
* 'hypothesis': The translated premise in the target language.
* 'label': The classification label, with possible values 0 ('entailment'), 1 ('neutral'), 2 ('contradiction').
* 'label\_text': The classification label, with possible values 'entailment' (0), 'neutral' (1), 'contradiction' (2).
* 'premise\_original': The original premise from the English source dataset.
* 'hypothesis\_original': The original hypothesis from the English source dataset. ### Data Splits



Additional Information
----------------------


BibTeX:


'


ACL:


Ruixiang Cui, Daniel Hershcovich, and Anders Søgaard. 2022. Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks. In *Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 4875–4893, Seattle, United States. Association for Computational Linguistics. ### Acknowledgements


This work was supported by the Defence Innovation Agency (AID) of the Directorate General of Armament (DGA) of the French Ministry of Armed Forces, and by the ICO, *Institut Cybersécurité Occitanie*, funded by Région Occitanie, France.
TAGS
#task_categories-text-classification #task_ids-natural-language-inference #task_ids-multi-input-text-classification #size_categories-n<1K #language-French #license-cc-by-4.0 #region-us 
 ### Dataset Summary


This repository contains a manually translated French version of the GQNLI challenge dataset, originally written in English. GQNLI is an evaluation corpus that is aimed for testing language model's generalized quantifier reasoning ability. ### Supported Tasks and Leaderboards


This dataset can be used for the task of Natural Language Inference (NLI), also known as Recognizing Textual Entailment (RTE), which is a sentence-pair classification task.


Dataset Structure
----------------- ### Data Fields


* 'uid': Index number.
* 'premise': The translated premise in the target language.
* 'hypothesis': The translated premise in the target language.
* 'label': The classification label, with possible values 0 ('entailment'), 1 ('neutral'), 2 ('contradiction').
* 'label\_text': The classification label, with possible values 'entailment' (0), 'neutral' (1), 'contradiction' (2).
* 'premise\_original': The original premise from the English source dataset.
* 'hypothesis\_original': The original hypothesis from the English source dataset. ### Data Splits



Additional Information
----------------------


BibTeX:


'


ACL:


Ruixiang Cui, Daniel Hershcovich, and Anders Søgaard. 2022. Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks. In *Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 4875–4893, Seattle, United States. Association for Computational Linguistics. ### Acknowledgements


This work was supported by the Defence Innovation Agency (AID) of the Directorate General of Armament (DGA) of the French Ministry of Armed Forces, and by the ICO, *Institut Cybersécurité Occitanie*, funded by Région Occitanie, France.