# Dataset Card for "AToMiC-Texts-Mapped"

## Dataset Description

- **Homepage:** [AToMiC homepage](https://trec-atomic.github.io/)
- **Source:** [WIT](https://github.com/google-research-datasets/wit)
- **Paper:** [WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning](https://arxiv.org/abs/2103.01913)

### Languages

This dataset only contains English in Wikipedia (parsed from the 20221101 XML dump).

### Data Instances

Each instance is a section of a Wikipedia page. We also provide its page-level information, and associated information such as categories and media.
The `source_id` can be mapped back to the instance in the original [WIT instance](https://github.com/google-research-datasets/wit/blob/main/DATA.md).
Notice that the WIT dataset is crawled from the earlier version of Wikipedia (2020-08-30).
The WIT dataset is mapped to the new dump by pure BM25 matching with [Anserini](https://github.com/castorini/anserini).

### Intended Usage

1. Text collection for Image-to-Text retrieval
2. Language model pretraining
3. Document classification

### Licensing Information

[CC BY-SA 4.0 international license](https://creativecommons.org/licenses/by-sa/4.0/)

### Citation Information

TBA

### Acknowledgement

Thanks to:
[mwparserfromhell](https://github.com/earwig/mwparserfromhell)
[Datasets](https://github.com/huggingface/datasets)
[Anserini](https://github.com/castorini/anserini)

[More Information needed](https://github.com/huggingface/datasets/blob/main/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
size_categories:100M<n<1B license:cc-by-sa-4.0 arxiv:2103.01913 region:us
{"license": "cc-by-sa-4.0", "size_categories": ["100M<n<1B"], "dataset_info": {"features": [{"name": "text_id", "dtype": "string"}, {"name": "page_url", "dtype": "string"}, {"name": "page_title", "dtype": "string"}, {"name": "section_title", "dtype": "string"}, {"name": "context_page_description", "dtype": "string"}, {"name": "context_section_description", "dtype": "string"}, {"name": "media", "sequence": "string"}, {"name": "hierachy", "sequence": "string"}, {"name": "category", "sequence": "string"}, {"name": "source_id", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 14378574060.336058, "num_examples": 10134744}], "download_size": 6408012391, "dataset_size": 14378574060.336058}}
TAGS
#size_categories-100M<n<1B #license-cc-by-sa-4.0 #arxiv-2103.01913 #region-us 

# Dataset Card for "AToMiC-Texts-Mapped"

## Dataset Description

- Homepage: AToMiC homepage
- Source: WIT
- Paper: WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning

### Languages

This dataset only contains English in Wikipedia (parsed from the 20221101 XML dump).

### Data Instances

Each instance is a section of a Wikipedia page. We also provide its page-level information, and associated information such as categories and media.
The 'source_id' can be mapped back to the instance in the original WIT instance.
Notice that the WIT dataset is crawled from the earlier version of Wikipedia (2020-08-30).
The WIT dataset is mapped to the new dump by pure BM25 matching with Anserini.

### Intended Usage

1. Text collection for Image-to-Text retrieval
2. Language model pretraining
3. Document classification

### Licensing Information

CC BY-SA 4.0 international license



TBA

### Acknowledgement

Thanks to:
mwparserfromhell
Datasets
Anserini

More Information needed
# Dataset Card for "AToMiC-Texts-Mapped" ## Dataset Description

- Homepage: AToMiC homepage
- Source: WIT
- Paper: WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning ### Languages

This dataset only contains English in Wikipedia (parsed from the 20221101 XML dump). ### Data Instances

Each instance is a section of a Wikipedia page. We also provide its page-level information, and associated information such as categories and media.
The 'source_id' can be mapped back to the instance in the original WIT instance.
Notice that the WIT dataset is crawled from the earlier version of Wikipedia (2020-08-30).
The WIT dataset is mapped to the new dump by pure BM25 matching with Anserini. ### Intended Usage

1. Text collection for Image-to-Text retrieval
2. Language model pretraining
3. Document classification ### Licensing Information

CC BY-SA 4.0 international license



TBA ### Acknowledgement

Thanks to:
mwparserfromhell
Datasets
Anserini

More Information needed
TAGS
#size_categories-100M<n<1B #license-cc-by-sa-4.0 #arxiv-2103.01913 #region-us 
 # Dataset Card for "AToMiC-Texts-Mapped" ## Dataset Description

- Homepage: AToMiC homepage
- Source: WIT
- Paper: WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning ### Languages

This dataset only contains English in Wikipedia (parsed from the 20221101 XML dump). ### Data Instances

Each instance is a section of a Wikipedia page. We also provide its page-level information, and associated information such as categories and media.
The 'source_id' can be mapped back to the instance in the original WIT instance.
Notice that the WIT dataset is crawled from the earlier version of Wikipedia (2020-08-30).
The WIT dataset is mapped to the new dump by pure BM25 matching with Anserini. ### Intended Usage

1. Text collection for Image-to-Text retrieval
2. Language model pretraining
3. Document classification ### Licensing Information

CC BY-SA 4.0 international license



TBA ### Acknowledgement

Thanks to:
mwparserfromhell
Datasets
Anserini

More Information needed