# Korean Natural Language Inference (KorNLI) for SimCSE Dataset

For a better dataset description, please visit this GitHub repository prepared by the authors of the article: [LINK](https://github.com/kakaobrain/kor-nlu-datasets) <br>
<br>
**This dataset was prepared by converting KorNLI dataset**. I took every unique premise of the dataset and searched for its entailment (positive example) and contradiction (negative example).
These changes have been made in order to apply SimCSE method.

**I additionaly share the code, which I used to convert the KorNLI dataset to make everything more clear**
```
from datasets import load_dataset
from tqdm import tqdm
import pandas as pd
import numpy as np

def create_trios(df, save_path):
    list_of_examples = []
    unique_premises = df.drop_duplicates("premise")["premise"]
    for premise in tqdm(unique_premises):
        premise_dataset = df[(df["premise"] == premise)] 
        positive_examples = premise_dataset[premise_dataset["label"] == "entailment"]["hypothesis"]
        negative_examples = premise_dataset[premise_dataset["label"] == "contradiction"]["hypothesis"]
        
        if len(positive_examples) == 0 or len(negative_examples) == 0:
            continue
        
        for positive_example in positive_examples:
            for negative_example in negative_examples:
                list_of_examples.append((premise, positive_example, negative_example))
        
        
    examples_df = pd.DataFrame(list_of_examples, columns=["premise", "entailment", "contradiction"])
    examples_df.to_csv(save_path)


if __name__ == "__main__":
    dataset1 = load_dataset("kor_nli", "snli")["train"]
    dataset2 = load_dataset("kor_nli", "multi_nli")["train"]
    df_1 = pd.DataFrame(dataset1)
    df_2 = pd.DataFrame(dataset2)
    df_full = pd.concat([df_1, df_2])
    df_full.dropna(inplace=True)
    df_full["label"] = ["neutral" if label == 1 else "contradiction" if label == 2 else "entailment" for label in df_full["label"]]
    create_trios(df_full, <insert your path>)
```

**How to download**

```
from datasets import load_dataset
data = load_dataset("dkoterwa/kor_nli_simcse")
```


**If you use this dataset for research, please cite this paper:**


```
@article{ham2020kornli,
  title={KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding},
  author={Ham, Jiyeon and Choe, Yo Joong and Park, Kyubyong and Choi, Ilji and Soh, Hyungjoon},
  journal={arXiv preprint arXiv:2004.03289},
  year={2020}
}
```
[More Information needed](https://github.com/huggingface/datasets/blob/main/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
region:us
{"dataset_info": {"features": [{"name": "premise", "dtype": "string"}, {"name": "entailment", "dtype": "string"}, {"name": "contradiction", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 90132700, "num_examples": 413837}, {"name": "valid", "num_bytes": 10572025, "num_examples": 48686}, {"name": "test", "num_bytes": 5289636, "num_examples": 24345}], "download_size": 64195317, "dataset_size": 105994361}, "configs": [{"config_name": "default", "data_files": [{"split": "train", "path": "data/train-*"}, {"split": "valid", "path": "data/valid-*"}, {"split": "test", "path": "data/test-*"}]}]}
TAGS
#region-us 

# Korean Natural Language Inference (KorNLI) for SimCSE Dataset

For a better dataset description, please visit this GitHub repository prepared by the authors of the article: LINK <br>
<br>
This dataset was prepared by converting KorNLI dataset. I took every unique premise of the dataset and searched for its entailment (positive example) and contradiction (negative example).
These changes have been made in order to apply SimCSE method.

I additionaly share the code, which I used to convert the KorNLI dataset to make everything more clear


How to download




If you use this dataset for research, please cite this paper:



More Information needed
# Korean Natural Language Inference (KorNLI) for SimCSE Dataset

For a better dataset description, please visit this GitHub repository prepared by the authors of the article: LINK <br>
<br>
This dataset was prepared by converting KorNLI dataset. I took every unique premise of the dataset and searched for its entailment (positive example) and contradiction (negative example).
These changes have been made in order to apply SimCSE method.

I additionaly share the code, which I used to convert the KorNLI dataset to make everything more clear


How to download




If you use this dataset for research, please cite this paper:



More Information needed
TAGS
#region-us 
 # Korean Natural Language Inference (KorNLI) for SimCSE Dataset

For a better dataset description, please visit this GitHub repository prepared by the authors of the article: LINK <br>
<br>
This dataset was prepared by converting KorNLI dataset. I took every unique premise of the dataset and searched for its entailment (positive example) and contradiction (negative example).
These changes have been made in order to apply SimCSE method.

I additionaly share the code, which I used to convert the KorNLI dataset to make everything more clear


How to download




If you use this dataset for research, please cite this paper:



More Information needed
passage: TAGS
#region-us 
# Korean Natural Language Inference (KorNLI) for SimCSE Dataset

For a better dataset description, please visit this GitHub repository prepared by the authors of the article: LINK <br>
<br>
This dataset was prepared by converting KorNLI dataset. I took every unique premise of the dataset and searched for its entailment (positive example) and contradiction (negative example).
These changes have been made in order to apply SimCSE method.

I additionaly share the code, which I used to convert the KorNLI dataset to make everything more clear


How to download




If you use this dataset for research, please cite this paper:



More Information needed