
# Stihi.ru dataset

## Table of Contents
- [Table of Contents](#table-of-contents)
- [Description](#description)
- [Usage](#usage)
- [Personal and Sensitive Information](#personal-and-sensitive-information)

## Description

**Summary:** A subset if [Taiga](https://tatianashavrina.github.io/taiga_site/), uploaded here for convenience. Additional cleaning was performed.

**Script:** [create_stihi.py](https://github.com/IlyaGusev/rulm/blob/master/data_processing/create_stihi.py)

**Point of Contact:** [Ilya Gusev](ilya.gusev@phystech.edu)

**Languages:** Russian.


## Usage

Prerequisites:
```bash
pip install datasets zstandard jsonlines pysimdjson
```

Dataset iteration:
```python
from datasets import load_dataset
dataset = load_dataset('IlyaGusev/stihi_ru', split="train", streaming=True)
for example in dataset:
    print(example["text"])
```

## Personal and Sensitive Information

The dataset is not anonymized, so individuals' names can be found in the dataset. Information about the original authors is included in the dataset where possible.
task_categories:text-generation size_categories:1M<n<10M language:ru region:us
{"language": ["ru"], "size_categories": ["1M<n<10M"], "task_categories": ["text-generation"], "dataset_info": {"features": [{"name": "id", "dtype": "string"}, {"name": "text", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "genre", "dtype": "string"}, {"name": "topic", "dtype": "string"}, {"name": "author", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6029108612, "num_examples": 5151050}], "download_size": 1892727043, "dataset_size": 6029108612}}
TAGS
#task_categories-text-generation #size_categories-1M<n<10M #language-Russian #region-us 


# URL dataset

## Table of Contents
- Table of Contents
- Description
- Usage
- Personal and Sensitive Information

## Description

Summary: A subset if Taiga, uploaded here for convenience. Additional cleaning was performed.

Script: create_stihi.py

Point of Contact: Ilya Gusev

Languages: Russian.


## Usage

Prerequisites:


Dataset iteration:


## Personal and Sensitive Information

The dataset is not anonymized, so individuals' names can be found in the dataset. Information about the original authors is included in the dataset where possible.
# URL dataset ## Table of Contents
- Table of Contents
- Description
- Usage
- Personal and Sensitive Information ## Description

Summary: A subset if Taiga, uploaded here for convenience. Additional cleaning was performed.

Script: create_stihi.py

Point of Contact: Ilya Gusev

Languages: Russian. ## Usage

Prerequisites:


Dataset iteration: ## Personal and Sensitive Information

The dataset is not anonymized, so individuals' names can be found in the dataset. Information about the original authors is included in the dataset where possible.
TAGS
#task_categories-text-generation #size_categories-1M<n<10M #language-Russian #region-us 
 # URL dataset ## Table of Contents
- Table of Contents
- Description
- Usage
- Personal and Sensitive Information ## Description

Summary: A subset if Taiga, uploaded here for convenience. Additional cleaning was performed.

Script: create_stihi.py

Point of Contact: Ilya Gusev

Languages: Russian. ## Usage

Prerequisites:


Dataset iteration: ## Personal and Sensitive Information

The dataset is not anonymized, so individuals' names can be found in the dataset. Information about the original authors is included in the dataset where possible.