
# Dataset Card for [Dataset Name]

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7339006/
- **Repository:**
- **Paper:** https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7339006/
- **Leaderboard:** [More Information Needed]
- **Point of Contact:** [More Information Needed]

### Dataset Summary

The Swahili dataset developed specifically for language modeling task.
The dataset contains 28,000 unique words with 6.84M, 970k, and 2M words for the train,
valid and test partitions respectively which represent the ratio 80:10:10.
The entire dataset is lowercased, has no punctuation marks and,
the start and end of sentence markers have been incorporated to facilitate easy tokenization during language modeling.

### Supported Tasks and Leaderboards

Language Modeling

### Languages

Swahili (sw)

## Dataset Structure

### Data Instances

[More Information Needed]

### Data Fields

- text : A line of text in Swahili

### Data Splits

train = 80%, valid = 10%, test = 10%

## Dataset Creation

### Curation Rationale

Enhancing African low-resource languages

### Source Data

#### Initial Data Collection and Normalization

The dataset contains 28,000 unique words with 6.84 M, 970k, and 2 M words for the train, valid and test partitions respectively which represent the ratio 80:10:10. 
The entire dataset is lowercased, has no punctuation marks and, the start and end of sentence markers have been incorporated to facilitate easy tokenization during language modelling.

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

Unannotated data

#### Who are the annotators?

NA

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

Enhancing African low-resource languages

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

Creative Commons Attribution 4.0 International

### Citation Information

"""\
@InProceedings{huggingface:dataset,
title = Language modeling data for Swahili (Version 1),
authors={Shivachi Casper Shikali, & Mokhosi Refuoe.
},
year={2019},
link = http://doi.org/10.5281/zenodo.3553423
}
"""

### Contributions

Thanks to [@akshayb7](https://github.com/akshayb7) for adding this dataset.
task_categories:text-generation task_categories:fill-mask task_ids:language-modeling task_ids:masked-language-modeling annotations_creators:no-annotation language_creators:expert-generated multilinguality:monolingual size_categories:10K<n<100K source_datasets:original language:sw license:cc-by-4.0 region:us
{"annotations_creators": ["no-annotation"], "language_creators": ["expert-generated"], "language": ["sw"], "license": ["cc-by-4.0"], "multilinguality": ["monolingual"], "size_categories": ["10K<n<100K"], "source_datasets": ["original"], "task_categories": ["text-generation", "fill-mask"], "task_ids": ["language-modeling", "masked-language-modeling"], "pretty_name": "swahili", "dataset_info": {"features": [{"name": "text", "dtype": "string"}], "config_name": "swahili", "splits": [{"name": "train", "num_bytes": 7700136, "num_examples": 42069}, {"name": "test", "num_bytes": 695092, "num_examples": 3371}, {"name": "validation", "num_bytes": 663520, "num_examples": 3372}], "download_size": 2783330, "dataset_size": 9058748}}
TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-language-modeling #task_ids-masked-language-modeling #annotations_creators-no-annotation #language_creators-expert-generated #multilinguality-monolingual #size_categories-10K<n<100K #source_datasets-original #language-Swahili (macrolanguage) #license-cc-by-4.0 #region-us 


# Dataset Card for [Dataset Name]

## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions

## Dataset Description

- Homepage: URL
- Repository:
- Paper: URL
- Leaderboard: 
- Point of Contact: 

### Dataset Summary

The Swahili dataset developed specifically for language modeling task.
The dataset contains 28,000 unique words with 6.84M, 970k, and 2M words for the train,
valid and test partitions respectively which represent the ratio 80:10:10.
The entire dataset is lowercased, has no punctuation marks and,
the start and end of sentence markers have been incorporated to facilitate easy tokenization during language modeling.

### Supported Tasks and Leaderboards

Language Modeling

### Languages

Swahili (sw)

## Dataset Structure

### Data Instances



### Data Fields

- text : A line of text in Swahili

### Data Splits

train = 80%, valid = 10%, test = 10%

## Dataset Creation

### Curation Rationale

Enhancing African low-resource languages

### Source Data

#### Initial Data Collection and Normalization

The dataset contains 28,000 unique words with 6.84 M, 970k, and 2 M words for the train, valid and test partitions respectively which represent the ratio 80:10:10. 
The entire dataset is lowercased, has no punctuation marks and, the start and end of sentence markers have been incorporated to facilitate easy tokenization during language modelling.

#### Who are the source language producers?



### Annotations

#### Annotation process

Unannotated data

#### Who are the annotators?

NA

### Personal and Sensitive Information



## Considerations for Using the Data

### Social Impact of Dataset

Enhancing African low-resource languages

### Discussion of Biases



### Other Known Limitations



## Additional Information

### Dataset Curators



### Licensing Information

Creative Commons Attribution 4.0 International



"""\
@InProceedings{huggingface:dataset,
title = Language modeling data for Swahili (Version 1),
authors={Shivachi Casper Shikali, & Mokhosi Refuoe.
},
year={2019},
link = URL
}
"""

### Contributions

Thanks to @akshayb7 for adding this dataset.
# Dataset Card for [Dataset Name] ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions ## Dataset Description

- Homepage: URL
- Repository:
- Paper: URL
- Leaderboard: 
- Point of Contact: ### Dataset Summary

The Swahili dataset developed specifically for language modeling task.
The dataset contains 28,000 unique words with 6.84M, 970k, and 2M words for the train,
valid and test partitions respectively which represent the ratio 80:10:10.
The entire dataset is lowercased, has no punctuation marks and,
the start and end of sentence markers have been incorporated to facilitate easy tokenization during language modeling. ### Supported Tasks and Leaderboards

Language Modeling ### Languages

Swahili (sw) ## Dataset Structure ### Data Instances ### Data Fields

- text : A line of text in Swahili ### Data Splits

train = 80%, valid = 10%, test = 10% ## Dataset Creation ### Curation Rationale

Enhancing African low-resource languages ### Source Data #### Initial Data Collection and Normalization

The dataset contains 28,000 unique words with 6.84 M, 970k, and 2 M words for the train, valid and test partitions respectively which represent the ratio 80:10:10. 
The entire dataset is lowercased, has no punctuation marks and, the start and end of sentence markers have been incorporated to facilitate easy tokenization during language modelling. #### Who are the source language producers? ### Annotations #### Annotation process

Unannotated data #### Who are the annotators?

NA ### Personal and Sensitive Information ## Considerations for Using the Data ### Social Impact of Dataset

Enhancing African low-resource languages ### Discussion of Biases ### Other Known Limitations ## Additional Information ### Dataset Curators ### Licensing Information

Creative Commons Attribution 4.0 International



"""\
@InProceedings{huggingface:dataset,
title = Language modeling data for Swahili (Version 1),
authors={Shivachi Casper Shikali, & Mokhosi Refuoe.
},
year={2019},
link = URL
}
""" ### Contributions

Thanks to @akshayb7 for adding this dataset.
TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-language-modeling #task_ids-masked-language-modeling #annotations_creators-no-annotation #language_creators-expert-generated #multilinguality-monolingual #size_categories-10K<n<100K #source_datasets-original #language-Swahili (macrolanguage) #license-cc-by-4.0 #region-us 
 # Dataset Card for [Dataset Name] ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions ## Dataset Description

- Homepage: URL
- Repository:
- Paper: URL
- Leaderboard: 
- Point of Contact: ### Dataset Summary

The Swahili dataset developed specifically for language modeling task.
The dataset contains 28,000 unique words with 6.84M, 970k, and 2M words for the train,
valid and test partitions respectively which represent the ratio 80:10:10.
The entire dataset is lowercased, has no punctuation marks and,
the start and end of sentence markers have been incorporated to facilitate easy tokenization during language modeling. ### Supported Tasks and Leaderboards

Language Modeling ### Languages

Swahili (sw) ## Dataset Structure ### Data Instances ### Data Fields

- text : A line of text in Swahili ### Data Splits

train = 80%, valid = 10%, test = 10% ## Dataset Creation ### Curation Rationale

Enhancing African low-resource languages ### Source Data #### Initial Data Collection and Normalization

The dataset contains 28,000 unique words with 6.84 M, 970k, and 2 M words for the train, valid and test partitions respectively which represent the ratio 80:10:10. 
The entire dataset is lowercased, has no punctuation marks and, the start and end of sentence markers have been incorporated to facilitate easy tokenization during language modelling. #### Who are the source language producers? ### Annotations #### Annotation process

Unannotated data #### Who are the annotators?

NA ### Personal and Sensitive Information ## Considerations for Using the Data ### Social Impact of Dataset

Enhancing African low-resource languages ### Discussion of Biases ### Other Known Limitations ## Additional Information ### Dataset Curators ### Licensing Information

Creative Commons Attribution 4.0 International



"""\
@InProceedings{huggingface:dataset,
title = Language modeling data for Swahili (Version 1),
authors={Shivachi Casper Shikali, & Mokhosi Refuoe.
},
year={2019},
link = URL
}
""" ### Contributions

Thanks to @akshayb7 for adding this dataset.
passage: TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-language-modeling #task_ids-masked-language-modeling #annotations_creators-no-annotation #language_creators-expert-generated #multilinguality-monolingual #size_categories-10K<n<100K #source_datasets-original #language-Swahili (macrolanguage) #license-cc-by-4.0 #region-us 
# Dataset Card for [Dataset Name]## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions## Dataset Description

- Homepage: URL
- Repository:
- Paper: URL
- Leaderboard: 
- Point of Contact:### Dataset Summary

The Swahili dataset developed specifically for language modeling task.
The dataset contains 28,000 unique words with 6.84M, 970k, and 2M words for the train,
valid and test partitions respectively which represent the ratio 80:10:10.
The entire dataset is lowercased, has no punctuation marks and,
the start and end of sentence markers have been incorporated to facilitate easy tokenization during language modeling.### Supported Tasks and Leaderboards

Language Modeling### Languages

Swahili (sw)## Dataset Structure### Data Instances### Data Fields

- text : A line of text in Swahili### Data Splits

train = 80%, valid = 10%, test = 10%## Dataset Creation### Curation Rationale

Enhancing African low-resource languages### Source Data