
# CompoundPiece

Dataset of compound words for the paper [CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models](https://arxiv.org/abs/2305.14214).

Load the balanced dataset of hyphens and non-hyphenated words scraped from the web (used as pretraining data):

```python
load_dataset("benjamin/compoundpiece", "web")
```

Load the dataset of compound and non-compound words (used for fine-tuning):

```python
load_dataset("benjamin/compoundpiece", "wiktionary")
```

# Citation

```
@article{minixhofer2023compoundpiece,
  title={CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models},
  author={Minixhofer, Benjamin and Pfeiffer, Jonas and Vuli{\'c}, Ivan},
  journal={arXiv preprint arXiv:2305.14214},
  year={2023}
}
```

# License

MIT
license:mit arxiv:2305.14214 region:us
{"license": "mit", "configs": [{"config_name": "wiktionary", "data_files": [{"split": "train", "path": "wiktionary/train.csv"}, {"split": "validation", "path": "wiktionary/valid.csv"}]}, {"config_name": "web", "data_files": [{"split": "train", "path": "web/train.csv"}, {"split": "validation", "path": "web/valid.csv"}]}]}
TAGS
#license-mit #arxiv-2305.14214 #region-us 


# CompoundPiece

Dataset of compound words for the paper CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models.

Load the balanced dataset of hyphens and non-hyphenated words scraped from the web (used as pretraining data):



Load the dataset of compound and non-compound words (used for fine-tuning):



# License

MIT
# CompoundPiece

Dataset of compound words for the paper CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models.

Load the balanced dataset of hyphens and non-hyphenated words scraped from the web (used as pretraining data):



Load the dataset of compound and non-compound words (used for fine-tuning): # License

MIT
TAGS
#license-mit #arxiv-2305.14214 #region-us 
 # CompoundPiece

Dataset of compound words for the paper CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models.

Load the balanced dataset of hyphens and non-hyphenated words scraped from the web (used as pretraining data):



Load the dataset of compound and non-compound words (used for fine-tuning): # License

MIT