task_categories:text-classification annotations_creators:expert-generated language_creators:found multilinguality:multilingual size_categories:100K<n<1M source_datasets:extended|xnli language:en language:de language:fr mode classification aligned region:us
This dataset was taken from XNLI for a binary text classification task. It has been parallelized in English, German, and French.
# @InProceedings{huggingface:dataset,
# title = {A great new dataset},
# author={huggingface, Inc.
# },
# year={2020}
# }
---
annotations_creators:
- expert-generated
language:
- en
- de
- fr
language_creators:
- found
license: []
multilinguality:
- multilingual
pretty_name: XNLI Parallel Corpus
size_categories:
- 100K<n<1M
source_datasets:
- extended|xnli
tags:
- mode classification
- aligned
task_categories:
- text-classification
task_ids: []
dataset_info:

  - config_name: en
    features:
      - name: text
        dtype: string
      - name: label
        dtype:
          class_label:
            names:
              '0': spoken
              '1': written
    splits:
      - name: train
        num_bytes:  92288
        num_examples: 830
      - name: test
        num_bytes:  186853
        num_examples: 1669

  - config_name: de
    features:
        - name: text
          dtype: string
        - name: label
          dtype:
            class_label:
              names:
                '0': spoken
                '1': written
    splits:
      - name: train
        num_bytes: 105681
        num_examples: 830
      - name: test
        num_bytes: 214008
        num_examples: 1669

  - config_name: fr
    features:
      - name: text
        dtype: string
      - name: label
        dtype:
          class_label:
            names:
              '0': spoken
              '1': written

    splits:
      - name: train
        num_bytes: 830
        num_examples: 109164
      - name: test
        num_bytes: 221286
        num_examples: 1669

download_size: 1864
dataset_size: 1840
---

# Dataset Card for XNLI Parallel Corpus

## Dataset Description

- **Homepage:** 
- **Repository:** 
- **Paper:** 
- **Leaderboard:** 
- **Point of Contact:** 

### Dataset Summary


### Supported Tasks and Leaderboards

Binary mode classification (spoken vs written)

### Languages
- English
- German
- French

## Dataset Structure

### Data Instances

{
  'text': "And he said , Mama , I 'm home .", 
  'label': 0
}

### Data Fields
- text: sentence
- label: binary label of text (0: spoken 1: written)

### Data Splits
  - train: 830
  - test: 1669

### Other Statistics

#### Vocabulary Size
- English
  - train: 4363
  - test: 7128
 
- German
  - train: 5070
  - test: 8601
    
- French
  - train: 4881
  - test: 7935

#### Average Sentence Length
- English
  - train: 20.689156626506023
  - test: 20.75254643499101
 
- German
  - train: 20.367469879518072
  - test: 20.639904134212102
    
- French
  - train: 23.455421686746988
  - test: 23.731575793888556

#### Label Split
  - train:
    - 0: 166
    - 1: 664
  - test:
    - 0: 334
    - 1: 1335
   
#### Out-of-vocabulary words in model
- English
  - BERT (bert-base-uncased)
    - train: 800
    - test: 1638
   
  - mBERT (bert-base-multilingual-uncased)
    - train: 1347
    - test: 2693
   
  - German BERT (bert-base-german-dbmdz-uncased)
    - train: 3228
    - test: 5581
   
  - flauBERT (flaubert-base-uncased)
    - train: 4363
    - test: 7128
   
- German
  - BERT (bert-base-uncased)
    - train: 4285
    - test: 7387
      
  - mBERT (bert-base-multilingual-uncased)
    - train: 3126
    - test: 5863

  - German BERT (bert-base-german-dbmdz-uncased)
    - train: 2033
    - test: 3938
      
  - flauBERT (flaubert-base-uncased)
    - train: 5069
    - test: 8600
   
- French
  - BERT (bert-base-uncased)
    - train: 3784
    - test: 6289
      
  - mBERT (bert-base-multilingual-uncased)
    - train: 2847
    - test: 5084
      
  - German BERT (bert-base-german-dbmdz-uncased)
    - train: 4212
    - test: 6964
      
  - flauBERT (flaubert-base-uncased)
    - train: 4881
    - test: 7935

## Dataset Creation

### Curation Rationale

N/A

### Source Data

https://github.com/facebookresearch/XNLI

Here is the citation for the original XNLI paper.

```
@InProceedings{conneau2018xnli,
  author = "Conneau, Alexis
        and Rinott, Ruty
        and Lample, Guillaume
        and Williams, Adina
        and Bowman, Samuel R.
        and Schwenk, Holger
        and Stoyanov, Veselin",
  title = "XNLI: Evaluating Cross-lingual Sentence Representations",
  booktitle = "Proceedings of the 2018 Conference on Empirical Methods
               in Natural Language Processing",
  year = "2018",
  publisher = "Association for Computational Linguistics",
  location = "Brussels, Belgium",
}
```

#### Initial Data Collection and Normalization

N/A

#### Who are the source language producers?

N/A

### Annotations

#### Annotation process

N/A

#### Who are the annotators?

N/A

### Personal and Sensitive Information

N/A

## Considerations for Using the Data

### Social Impact of Dataset

N/A

### Discussion of Biases

N/A

### Other Known Limitations

N/A

## Additional Information

### Dataset Curators

N/A

### Licensing Information

N/A

### Citation Information


### Contributions

N/A