region:us
---
dataset_info:
  features:
  - name: id
    dtype: int64
  - name: genre
    dtype: string
  - name: sentence1
    dtype: string
  - name: sentence2
    dtype: string
  - name: score
    dtype: int64
  splits:
  - name: train
    num_bytes: 1034815
    num_examples: 5691
  - name: test
    num_bytes: 247409
    num_examples: 1376
  - name: valid
    num_bytes: 297254
    num_examples: 1465
  download_size: 828725
  dataset_size: 1579478
---
---
license: cc-by-sa-4.0
dataset_info:
  features:
  - name: id
    dtype: int64
  - name: genre
    dtype: string
  - name: sentence1
    dtype: string
  - name: sentence2
    dtype: string
  - name: score
    dtype: float64
  splits:
  - name: train
    num_bytes: 1034815
    num_examples: 5691
  - name: valid
    num_bytes: 297254
    num_examples: 1465
  - name: test
    num_bytes: 247409
    num_examples: 1376
  download_size: 837346
  dataset_size: 1579478
---

# Korean Semantic Textual Similarity (KorSTS) Dataset (with converted scores)

For a better dataset description, please visit this GitHub repository prepared by the authors of the article: [LINK](https://github.com/kakaobrain/kor-nlu-datasets) <br>
<br>
**This dataset was prepared by converting [this dataset](https://huggingface.co/datasets/dkoterwa/kor-sts)** Specifically, I've changed the values of scores. If the score was equal or above 2.5, then it was assigned with label "1" (similar). In other cases, label "-1" was assigned (not similar).

I've made these changes in order to apply CosineEmbeddingLoss from PyTorch library [LINK](https://pytorch.org/docs/stable/generated/torch.nn.CosineEmbeddingLoss.html).

**How to download**

```
from datasets import load_dataset
data = load_dataset("dkoterwa/kor-sts-cosine-embedding-loss")
```


**If you use this dataset for research, please cite this paper:**

```
@article{ham2020kornli,
  title={KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding},
  author={Ham, Jiyeon and Choe, Yo Joong and Park, Kyubyong and Choi, Ilji and Soh, Hyungjoon},
  journal={arXiv preprint arXiv:2004.03289},
  year={2020}
}
```