
---

This dataset is based on [SiberianPersonaChat Dataset](https://huggingface.co/datasets/SiberiaSoft/SiberianPersonaChat).

It was additionally filtered using:
- politics filter ([cointegrated/rubert-base-cased-nli-threeway](https://huggingface.co/cointegrated/rubert-base-cased-nli-threeway))
- toxicity filter ([cointegrated/rubert-tiny-toxicity](https://huggingface.co/cointegrated/rubert-tiny-toxicity))
- low quality qa pairs filter ([Andrilko/ruBert-base-reward](https://huggingface.co/Andrilko/ruBert-base-reward))

**Dataset Statistics:**

- wiki_qa: 4.746
- dialog_personal_context: 68.296
- russianinstructions2: 4.812
- yandexQ_instruct: 6.316
- rugpt4: 5.269
- trupalpaca: 4.284
- text_qa: 2.57
- long_answers_qa: 3.363
- chitchat: 0.198
- reaction: 0.108
- baby: 0.037

---
### Citation

```
@MISC{morzecrew/RefinedPersonaChat,
    author  = {Yuri Zaretskiy, Nikolas Ivanov, Igor Kuzmin},
    title   = {Refined dataset for conversational agents},
    url     = {https://huggingface.co/datasets/morzecrew/RefinedPersonaChat},
    year    = 2023
}
```
task_categories:text-generation task_categories:conversational task_categories:text2text-generation size_categories:100K<n<1M language:ru license:mit region:us
{"language": ["ru"], "license": "mit", "size_categories": ["100K<n<1M"], "task_categories": ["text-generation", "conversational", "text2text-generation"], "pretty_name": "ref-chat"}
TAGS
#task_categories-text-generation #task_categories-conversational #task_categories-text2text-generation #size_categories-100K<n<1M #language-Russian #license-mit #region-us 


---

This dataset is based on SiberianPersonaChat Dataset.

It was additionally filtered using:
- politics filter (cointegrated/rubert-base-cased-nli-threeway)
- toxicity filter (cointegrated/rubert-tiny-toxicity)
- low quality qa pairs filter (Andrilko/ruBert-base-reward)

Dataset Statistics:

- wiki_qa: 4.746
- dialog_personal_context: 68.296
- russianinstructions2: 4.812
- yandexQ_instruct: 6.316
- rugpt4: 5.269
- trupalpaca: 4.284
- text_qa: 2.57
- long_answers_qa: 3.363
- chitchat: 0.198
- reaction: 0.108
- baby: 0.037

---

TAGS
#task_categories-text-generation #task_categories-conversational #task_categories-text2text-generation #size_categories-100K<n<1M #language-Russian #license-mit #region-us 
