
# Dataset Card for Norwegian PAWS-X

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Languages](#languages)
  - [Dataset Structure](#dataset-structure)
    - [Data Instances](#data-instances)
    - [Data Fields](#data-fields)
    - [Data Splits](#data-splits)
  - [Dataset Creation](#dataset-creation)
    - [Curation Rationale](#curation-rationale)
    - [Source Data](#source-data)
    - [Annotations](#annotations)
    - [Personal and Sensitive Information](#personal-and-sensitive-information)
  - [Considerations for Using the Data](#considerations-for-using-the-data)
    - [Social Impact of Dataset](#social-impact-of-dataset)
    - [Discussion of Biases](#discussion-of-biases)
    - [Other Known Limitations](#other-known-limitations)
  - [Additional Information](#additional-information)
    - [Dataset Curators](#dataset-curators)
    - [Licensing Information](#licensing-information)
    - [Citation Information](#citation-information)
    - [Contributions](#contributions)

## Dataset Description

- **Homepage:** [NB AiLab](https://ai.nb.no/)
- **Repository:** [Norwegian PAWS-X Repository](#)
- **Point of Contact:** [ai-lab@nb.no](mailto:ai-lab@nb.no)

### Dataset Summary

Norwegian PAWS-X is an extension of the PAWS-X dataset. PAWS-X is a multilingual version of PAWS (Paraphrase Adversaries from Word Scrambling) for six languages. The Norwegian PAWS-X dataset has machine-translated versions of the original PAWS-X dataset into Norwegian Bokmål and Nynorsk.

### Languages

- Norwegian Bokmål (`nb`)
- Norwegian Nynorsk (`nn`)

## Dataset Structure

### Data Instances

Each instance includes a pair of sentences in Norwegian along with a binary label indicating whether the sentences are paraphrases of each other.

### Data Fields

- `id`: An identifier for each example (int32)
- `sentence1`: The first sentence in Norwegian (string)
- `sentence2`: The second sentence in Norwegian (string)
- `label`: Binary label, where '1' indicates the sentences are paraphrases and '0' indicates they are not (class_label: '0', '1')

### Data Splits

The dataset is divided into training, validation, and test sets. The exact numbers of instances in each split will be as per the original PAWS-X dataset.

## Dataset Creation

### Curation Rationale

Norwegian PAWS-X was created to extend the PAWS paraphrase identification task to the Norwegian language, including both Bokmål and Nynorsk standards. This promotes multilingual and cross-lingual research in paraphrase identification.

### Source Data

The source data consists of human-translated PAWS pairs in six languages. For the Norwegian PAWS-X dataset, these pairs were translated into Norwegian Bokmål and Nynorsk using FAIR’s No Language Left Behind 3.3B parameters model.

### Annotations

The dataset retains the original PAWS labels, which were created through a combination of expert and machine-generated annotations.

### Personal and Sensitive Information

There is no known personal or sensitive information in this dataset.

## Considerations for Using the Data

### Social Impact of Dataset

This dataset helps in promoting the development of NLP technologies in Norwegian.

### Other Known Limitations

There may be potential issues related to the translation quality, as the translations were generated using a machine translation model.

## Additional Information

### Dataset Curators

The dataset was curated by researcher Javier de la Rosa.

### Licensing Information

Original PAWS-X License:
- The dataset may be freely used for any purpose, with acknowledgment of Google LLC as the data source being appreciated. The dataset is provided "AS IS" without any warranty, express or implied. Google disclaims all liability for any damages, direct or indirect, resulting from the use of the dataset.

Norwegian PAWS-X License:
- CC BY 4.0


task_categories:text-classification task_ids:semantic-similarity-classification task_ids:semantic-similarity-scoring task_ids:text-scoring task_ids:multi-input-text-classification annotations_creators:expert-generated annotations_creators:machine-generated language_creators:machine-generated multilinguality:multilingual size_categories:10K<n<100K source_datasets:extended|other-paws language:nb language:nn license:cc-by-4.0 region:us
{"annotations_creators": ["expert-generated", "machine-generated"], "language_creators": ["machine-generated"], "language": ["nb", "nn"], "license": ["cc-by-4.0"], "multilinguality": ["multilingual"], "size_categories": ["10K<n<100K"], "source_datasets": ["extended|other-paws"], "task_categories": ["text-classification"], "task_ids": ["semantic-similarity-classification", "semantic-similarity-scoring", "text-scoring", "multi-input-text-classification"], "pretty_name": "NbAiLab/norwegian-paws-x"}
TAGS
#task_categories-text-classification #task_ids-semantic-similarity-classification #task_ids-semantic-similarity-scoring #task_ids-text-scoring #task_ids-multi-input-text-classification #annotations_creators-expert-generated #annotations_creators-machine-generated #language_creators-machine-generated #multilinguality-multilingual #size_categories-10K<n<100K #source_datasets-extended|other-paws #language-Norwegian Bokmål #language-Norwegian Nynorsk #license-cc-by-4.0 #region-us 


# Dataset Card for Norwegian PAWS-X

## Table of Contents
- Dataset Description
  - Dataset Summary
  - Languages
  - Dataset Structure
    - Data Instances
    - Data Fields
    - Data Splits
  - Dataset Creation
    - Curation Rationale
    - Source Data
    - Annotations
    - Personal and Sensitive Information
  - Considerations for Using the Data
    - Social Impact of Dataset
    - Discussion of Biases
    - Other Known Limitations
  - Additional Information
    - Dataset Curators
    - Licensing Information
    - Citation Information
    - Contributions

## Dataset Description

- Homepage: NB AiLab
- Repository: Norwegian PAWS-X Repository
- Point of Contact: ai-lab@URL

### Dataset Summary

Norwegian PAWS-X is an extension of the PAWS-X dataset. PAWS-X is a multilingual version of PAWS (Paraphrase Adversaries from Word Scrambling) for six languages. The Norwegian PAWS-X dataset has machine-translated versions of the original PAWS-X dataset into Norwegian Bokmål and Nynorsk.

### Languages

- Norwegian Bokmål ('nb')
- Norwegian Nynorsk ('nn')

## Dataset Structure

### Data Instances

Each instance includes a pair of sentences in Norwegian along with a binary label indicating whether the sentences are paraphrases of each other.

### Data Fields

- 'id': An identifier for each example (int32)
- 'sentence1': The first sentence in Norwegian (string)
- 'sentence2': The second sentence in Norwegian (string)
- 'label': Binary label, where '1' indicates the sentences are paraphrases and '0' indicates they are not (class_label: '0', '1')

### Data Splits

The dataset is divided into training, validation, and test sets. The exact numbers of instances in each split will be as per the original PAWS-X dataset.

## Dataset Creation

### Curation Rationale

Norwegian PAWS-X was created to extend the PAWS paraphrase identification task to the Norwegian language, including both Bokmål and Nynorsk standards. This promotes multilingual and cross-lingual research in paraphrase identification.

### Source Data

The source data consists of human-translated PAWS pairs in six languages. For the Norwegian PAWS-X dataset, these pairs were translated into Norwegian Bokmål and Nynorsk using FAIR’s No Language Left Behind 3.3B parameters model.

### Annotations

The dataset retains the original PAWS labels, which were created through a combination of expert and machine-generated annotations.

### Personal and Sensitive Information

There is no known personal or sensitive information in this dataset.

## Considerations for Using the Data

### Social Impact of Dataset

This dataset helps in promoting the development of NLP technologies in Norwegian.

### Other Known Limitations

There may be potential issues related to the translation quality, as the translations were generated using a machine translation model.

## Additional Information

### Dataset Curators

The dataset was curated by researcher Javier de la Rosa.

### Licensing Information

Original PAWS-X License:
- The dataset may be freely used for any purpose, with acknowledgment of Google LLC as the data source being appreciated. The dataset is provided "AS IS" without any warranty, express or implied. Google disclaims all liability for any damages, direct or indirect, resulting from the use of the dataset.

Norwegian PAWS-X License:
- CC BY 4.0


# Dataset Card for Norwegian PAWS-X ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Languages
  - Dataset Structure
    - Data Instances
    - Data Fields
    - Data Splits
  - Dataset Creation
    - Curation Rationale
    - Source Data
    - Annotations
    - Personal and Sensitive Information
  - Considerations for Using the Data
    - Social Impact of Dataset
    - Discussion of Biases
    - Other Known Limitations
  - Additional Information
    - Dataset Curators
    - Licensing Information
    - Citation Information
    - Contributions ## Dataset Description

- Homepage: NB AiLab
- Repository: Norwegian PAWS-X Repository
- Point of Contact: ai-lab@URL ### Dataset Summary

Norwegian PAWS-X is an extension of the PAWS-X dataset. PAWS-X is a multilingual version of PAWS (Paraphrase Adversaries from Word Scrambling) for six languages. The Norwegian PAWS-X dataset has machine-translated versions of the original PAWS-X dataset into Norwegian Bokmål and Nynorsk. ### Languages

- Norwegian Bokmål ('nb')
- Norwegian Nynorsk ('nn') ## Dataset Structure ### Data Instances

Each instance includes a pair of sentences in Norwegian along with a binary label indicating whether the sentences are paraphrases of each other. ### Data Fields

- 'id': An identifier for each example (int32)
- 'sentence1': The first sentence in Norwegian (string)
- 'sentence2': The second sentence in Norwegian (string)
- 'label': Binary label, where '1' indicates the sentences are paraphrases and '0' indicates they are not (class_label: '0', '1') ### Data Splits

The dataset is divided into training, validation, and test sets. The exact numbers of instances in each split will be as per the original PAWS-X dataset. ## Dataset Creation ### Curation Rationale

Norwegian PAWS-X was created to extend the PAWS paraphrase identification task to the Norwegian language, including both Bokmål and Nynorsk standards. This promotes multilingual and cross-lingual research in paraphrase identification. ### Source Data

The source data consists of human-translated PAWS pairs in six languages. For the Norwegian PAWS-X dataset, these pairs were translated into Norwegian Bokmål and Nynorsk using FAIR’s No Language Left Behind 3.3B parameters model. ### Annotations

The dataset retains the original PAWS labels, which were created through a combination of expert and machine-generated annotations. ### Personal and Sensitive Information

There is no known personal or sensitive information in this dataset. ## Considerations for Using the Data ### Social Impact of Dataset

This dataset helps in promoting the development of NLP technologies in Norwegian. ### Other Known Limitations

There may be potential issues related to the translation quality, as the translations were generated using a machine translation model. ## Additional Information ### Dataset Curators

The dataset was curated by researcher Javier de la Rosa. ### Licensing Information

Original PAWS-X License:
- The dataset may be freely used for any purpose, with acknowledgment of Google LLC as the data source being appreciated. The dataset is provided "AS IS" without any warranty, express or implied. Google disclaims all liability for any damages, direct or indirect, resulting from the use of the dataset.

Norwegian PAWS-X License:
- CC BY 4.0
TAGS
#task_categories-text-classification #task_ids-semantic-similarity-classification #task_ids-semantic-similarity-scoring #task_ids-text-scoring #task_ids-multi-input-text-classification #annotations_creators-expert-generated #annotations_creators-machine-generated #language_creators-machine-generated #multilinguality-multilingual #size_categories-10K<n<100K #source_datasets-extended|other-paws #language-Norwegian Bokmål #language-Norwegian Nynorsk #license-cc-by-4.0 #region-us 
 # Dataset Card for Norwegian PAWS-X ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Languages
  - Dataset Structure
    - Data Instances
    - Data Fields
    - Data Splits
  - Dataset Creation
    - Curation Rationale
    - Source Data
    - Annotations
    - Personal and Sensitive Information
  - Considerations for Using the Data
    - Social Impact of Dataset
    - Discussion of Biases
    - Other Known Limitations
  - Additional Information
    - Dataset Curators
    - Licensing Information
    - Citation Information
    - Contributions ## Dataset Description

- Homepage: NB AiLab
- Repository: Norwegian PAWS-X Repository
- Point of Contact: ai-lab@URL ### Dataset Summary

Norwegian PAWS-X is an extension of the PAWS-X dataset. PAWS-X is a multilingual version of PAWS (Paraphrase Adversaries from Word Scrambling) for six languages. The Norwegian PAWS-X dataset has machine-translated versions of the original PAWS-X dataset into Norwegian Bokmål and Nynorsk. ### Languages

- Norwegian Bokmål ('nb')
- Norwegian Nynorsk ('nn') ## Dataset Structure ### Data Instances

Each instance includes a pair of sentences in Norwegian along with a binary label indicating whether the sentences are paraphrases of each other. ### Data Fields

- 'id': An identifier for each example (int32)
- 'sentence1': The first sentence in Norwegian (string)
- 'sentence2': The second sentence in Norwegian (string)
- 'label': Binary label, where '1' indicates the sentences are paraphrases and '0' indicates they are not (class_label: '0', '1') ### Data Splits

The dataset is divided into training, validation, and test sets. The exact numbers of instances in each split will be as per the original PAWS-X dataset. ## Dataset Creation ### Curation Rationale

Norwegian PAWS-X was created to extend the PAWS paraphrase identification task to the Norwegian language, including both Bokmål and Nynorsk standards. This promotes multilingual and cross-lingual research in paraphrase identification. ### Source Data

The source data consists of human-translated PAWS pairs in six languages. For the Norwegian PAWS-X dataset, these pairs were translated into Norwegian Bokmål and Nynorsk using FAIR’s No Language Left Behind 3.3B parameters model. ### Annotations

The dataset retains the original PAWS labels, which were created through a combination of expert and machine-generated annotations. ### Personal and Sensitive Information

There is no known personal or sensitive information in this dataset. ## Considerations for Using the Data ### Social Impact of Dataset

This dataset helps in promoting the development of NLP technologies in Norwegian. ### Other Known Limitations

There may be potential issues related to the translation quality, as the translations were generated using a machine translation model. ## Additional Information ### Dataset Curators

The dataset was curated by researcher Javier de la Rosa. ### Licensing Information

Original PAWS-X License:
- The dataset may be freely used for any purpose, with acknowledgment of Google LLC as the data source being appreciated. The dataset is provided "AS IS" without any warranty, express or implied. Google disclaims all liability for any damages, direct or indirect, resulting from the use of the dataset.

Norwegian PAWS-X License:
- CC BY 4.0
passage: TAGS
#task_categories-text-classification #task_ids-semantic-similarity-classification #task_ids-semantic-similarity-scoring #task_ids-text-scoring #task_ids-multi-input-text-classification #annotations_creators-expert-generated #annotations_creators-machine-generated #language_creators-machine-generated #multilinguality-multilingual #size_categories-10K<n<100K #source_datasets-extended|other-paws #language-Norwegian Bokmål #language-Norwegian Nynorsk #license-cc-by-4.0 #region-us 
# Dataset Card for Norwegian PAWS-X## Table of Contents
- Dataset Description
  - Dataset Summary
  - Languages
  - Dataset Structure
    - Data Instances
    - Data Fields
    - Data Splits
  - Dataset Creation
    - Curation Rationale
    - Source Data
    - Annotations
    - Personal and Sensitive Information
  - Considerations for Using the Data
    - Social Impact of Dataset
    - Discussion of Biases
    - Other Known Limitations
  - Additional Information
    - Dataset Curators
    - Licensing Information
    - Citation Information
    - Contributions## Dataset Description

- Homepage: NB AiLab
- Repository: Norwegian PAWS-X Repository
- Point of Contact: ai-lab@URL### Dataset Summary

Norwegian PAWS-X is an extension of the PAWS-X dataset. PAWS-X is a multilingual version of PAWS (Paraphrase Adversaries from Word Scrambling) for six languages. The Norwegian PAWS-X dataset has machine-translated versions of the original PAWS-X dataset into Norwegian Bokmål and Nynorsk.### Languages

- Norwegian Bokmål ('nb')
- Norwegian Nynorsk ('nn')## Dataset Structure### Data Instances

Each instance includes a pair of sentences in Norwegian along with a binary label indicating whether the sentences are paraphrases of each other.