# Dataset Card for unarXive IMRaD classification

## Dataset Description

* **Homepage:** [https://github.com/IllDepence/unarXive](https://github.com/IllDepence/unarXive)
* **Paper:** [unarXive 2022: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network](https://arxiv.org/abs/2303.14957)

### Dataset Summary

The unarXive IMRaD classification dataset contains 530k paragraphs from computer science papers and the IMRaD section they originate from. The paragraphs are derived from [unarXive](https://github.com/IllDepence/unarXive).

The dataset can be used as follows.

```
from datasets import load_dataset

imrad_data = load_dataset('saier/unarXive_imrad_clf')
imrad_data = imrad_data.class_encode_column('label')  # assign target label column
imrad_data = imrad_data.remove_columns('_id')         # remove sample ID column
```

## Dataset Structure

### Data Instances

Each data instance contains the paragraph’s text as well as one of the labels ('i', 'm', 'r', 'd', 'w' — for Introduction, Methods, Results, Discussion and Related Work). An example is shown below.

```
{'_id': '789f68e7-a1cc-4072-b07d-ecffc3e7ca38',
 'label': 'm',
 'text': 'To link the mentions encoded by BERT to the KGE entities, we define '
         'an entity linking loss as cross-entropy between self-supervised '
         'entity labels and similarities obtained from the linker in KGE '
         'space:\n'
         '\\(\\mathcal {L}_{EL}=\\sum -\\log \\dfrac{\\exp (h_m^{proj}\\cdot '
         '\\textbf {e})}{\\sum _{\\textbf {e}_j\\in \\mathcal {E}} \\exp '
         '(h_m^{proj}\\cdot \\textbf {e}_j)}\\) \n'}
```

### Data Splits

The data is split into training, development, and testing data as follows.

* Training: 520,053 instances
* Development: 5000 instances
* Testing: 5001 instances

## Dataset Creation

### Source Data

The paragraph texts are extracted from the data set [unarXive](https://github.com/IllDepence/unarXive).

#### Who are the source language producers?

The paragraphs were written by the authors of the arXiv papers. In file `license_info.jsonl` author and text licensing information can be found for all samples, An example is shown below.

```

{'authors': 'Yusuke Sekikawa, Teppei Suzuki',
 'license': 'http://creativecommons.org/licenses/by/4.0/',
 'paper_arxiv_id': '2011.09852',
 'sample_ids': ['cc375518-347c-43d0-bfb2-f88564d66df8',
                '18dc073e-a48e-488e-b34c-e5fc3cb8a4ca',
                '0c2e89b3-d863-4bc2-9e11-8f6c48d867cb',
                'd85e46cf-b11d-49b6-801b-089aa2dd037d',
                '92915cea-17ab-4a98-aad2-417f6cdd53d2',
                'e88cb422-47b7-4f69-9b0b-fbddf8140d98',
                '4f5094a4-0e6e-46ae-a34d-e15ce0b9803c',
                '59003494-096f-4a7c-ad65-342b74eed561',
                '6a99b3f5-217e-4d3d-a770-693483ef8670']}
```

### Annotations

Class labels were automatically determined ([see implementation](https://github.com/IllDepence/unarXive/blob/master/src/utility_scripts/ml_tasks_prep_data.py)).

## Considerations for Using the Data

### Discussion and Biases

Because only paragraphs unambiguously assignable to one of the IMRaD classeswere used, a certain selection bias is to be expected in the data.

### Other Known Limitations

Depending on authors’ writing styles as well LaTeX processing quirks, paragraphs can vary in length a significantly.

## Additional Information

### Licensing information

The dataset is released under the Creative Commons Attribution-ShareAlike 4.0.

### Citation Information

```
@inproceedings{Saier2023unarXive,
  author        = {Saier, Tarek and Krause, Johan and F\"{a}rber, Michael},
  title         = {{unarXive 2022: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network}},
  booktitle     = {Proceedings of the 23rd ACM/IEEE Joint Conference on Digital Libraries},
  year          = {2023},
  series        = {JCDL '23}
}
```

task_categories:text-classification task_ids:multi-class-classification annotations_creators:machine-generated language_creators:found multilinguality:monolingual size_categories:100K<n<1M source_datasets:extended|10.5281/zenodo.7752615 language:en license:cc-by-sa-4.0 arXiv.org arXiv IMRaD publication paper preprint section physics mathematics computer science cs arxiv:2303.14957 region:us
{"annotations_creators": ["machine-generated"], "language_creators": ["found"], "language": ["en"], "license": ["cc-by-sa-4.0"], "multilinguality": ["monolingual"], "size_categories": ["100K<n<1M"], "source_datasets": ["extended|10.5281/zenodo.7752615"], "task_categories": ["text-classification"], "task_ids": ["multi-class-classification"], "pretty_name": "unarXive IMRaD classification", "tags": ["arXiv.org", "arXiv", "IMRaD", "publication", "paper", "preprint", "section", "physics", "mathematics", "computer science", "cs"], "dataset_info": {"features": [{"name": "_id", "dtype": "string"}, {"name": "text", "dtype": "string"}, {"name": "label", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 451908280, "num_examples": 520053}, {"name": "test", "num_bytes": 4650429, "num_examples": 5000}, {"name": "validation", "num_bytes": 4315597, "num_examples": 5001}], "download_size": 482376743, "dataset_size": 460874306}}
TAGS
#task_categories-text-classification #task_ids-multi-class-classification #annotations_creators-machine-generated #language_creators-found #multilinguality-monolingual #size_categories-100K<n<1M #source_datasets-extended|10.5281/zenodo.7752615 #language-English #license-cc-by-sa-4.0 #arXiv.org #arXiv #IMRaD #publication #paper #preprint #section #physics #mathematics #computer science #cs #arxiv-2303.14957 #region-us 

# Dataset Card for unarXive IMRaD classification

## Dataset Description

* Homepage: URL
* Paper: unarXive 2022: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network

### Dataset Summary

The unarXive IMRaD classification dataset contains 530k paragraphs from computer science papers and the IMRaD section they originate from. The paragraphs are derived from unarXive.

The dataset can be used as follows.



## Dataset Structure

### Data Instances

Each data instance contains the paragraph’s text as well as one of the labels ('i', 'm', 'r', 'd', 'w' — for Introduction, Methods, Results, Discussion and Related Work). An example is shown below.



### Data Splits

The data is split into training, development, and testing data as follows.

* Training: 520,053 instances
* Development: 5000 instances
* Testing: 5001 instances

## Dataset Creation

### Source Data

The paragraph texts are extracted from the data set unarXive.

#### Who are the source language producers?

The paragraphs were written by the authors of the arXiv papers. In file 'license_info.jsonl' author and text licensing information can be found for all samples, An example is shown below.



### Annotations

Class labels were automatically determined (see implementation).

## Considerations for Using the Data

### Discussion and Biases

Because only paragraphs unambiguously assignable to one of the IMRaD classeswere used, a certain selection bias is to be expected in the data.

### Other Known Limitations

Depending on authors’ writing styles as well LaTeX processing quirks, paragraphs can vary in length a significantly.

## Additional Information

### Licensing information

The dataset is released under the Creative Commons Attribution-ShareAlike 4.0.





# Dataset Card for unarXive IMRaD classification ## Dataset Description

* Homepage: URL
* Paper: unarXive 2022: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network ### Dataset Summary

The unarXive IMRaD classification dataset contains 530k paragraphs from computer science papers and the IMRaD section they originate from. The paragraphs are derived from unarXive.

The dataset can be used as follows. ## Dataset Structure ### Data Instances

Each data instance contains the paragraph’s text as well as one of the labels ('i', 'm', 'r', 'd', 'w' — for Introduction, Methods, Results, Discussion and Related Work). An example is shown below. ### Data Splits

The data is split into training, development, and testing data as follows.

* Training: 520,053 instances
* Development: 5000 instances
* Testing: 5001 instances ## Dataset Creation ### Source Data

The paragraph texts are extracted from the data set unarXive. #### Who are the source language producers?

The paragraphs were written by the authors of the arXiv papers. In file 'license_info.jsonl' author and text licensing information can be found for all samples, An example is shown below. ### Annotations

Class labels were automatically determined (see implementation). ## Considerations for Using the Data ### Discussion and Biases

Because only paragraphs unambiguously assignable to one of the IMRaD classeswere used, a certain selection bias is to be expected in the data. ### Other Known Limitations

Depending on authors’ writing styles as well LaTeX processing quirks, paragraphs can vary in length a significantly. ## Additional Information ### Licensing information

The dataset is released under the Creative Commons Attribution-ShareAlike 4.0.
TAGS
#task_categories-text-classification #task_ids-multi-class-classification #annotations_creators-machine-generated #language_creators-found #multilinguality-monolingual #size_categories-100K<n<1M #source_datasets-extended|10.5281/zenodo.7752615 #language-English #license-cc-by-sa-4.0 #arXiv.org #arXiv #IMRaD #publication #paper #preprint #section #physics #mathematics #computer science #cs #arxiv-2303.14957 #region-us 
 # Dataset Card for unarXive IMRaD classification ## Dataset Description

* Homepage: URL
* Paper: unarXive 2022: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network ### Dataset Summary

The unarXive IMRaD classification dataset contains 530k paragraphs from computer science papers and the IMRaD section they originate from. The paragraphs are derived from unarXive.

The dataset can be used as follows. ## Dataset Structure ### Data Instances

Each data instance contains the paragraph’s text as well as one of the labels ('i', 'm', 'r', 'd', 'w' — for Introduction, Methods, Results, Discussion and Related Work). An example is shown below. ### Data Splits

The data is split into training, development, and testing data as follows.

* Training: 520,053 instances
* Development: 5000 instances
* Testing: 5001 instances ## Dataset Creation ### Source Data

The paragraph texts are extracted from the data set unarXive. #### Who are the source language producers?

The paragraphs were written by the authors of the arXiv papers. In file 'license_info.jsonl' author and text licensing information can be found for all samples, An example is shown below. ### Annotations

Class labels were automatically determined (see implementation). ## Considerations for Using the Data ### Discussion and Biases

Because only paragraphs unambiguously assignable to one of the IMRaD classeswere used, a certain selection bias is to be expected in the data. ### Other Known Limitations

Depending on authors’ writing styles as well LaTeX processing quirks, paragraphs can vary in length a significantly. ## Additional Information ### Licensing information

The dataset is released under the Creative Commons Attribution-ShareAlike 4.0.