
# RuNews dataset

## Table of Contents
- [Table of Contents](#table-of-contents)
- [Description](#description)
- [Usage](#usage)
- [Data Instances](#data-instances)
- [Personal and Sensitive Information](#personal-and-sensitive-information)

## Description

**Summary:** Dataset of news from several sources:
* [Lenta.ru by yutkin](https://github.com/yutkin/Lenta.Ru-News-Dataset)
* [Several sources by buriy](https://github.com/buriy/russian-nlp-datasets/releases)
* [ODS Newsviz Tass](https://github.com/newsviz/newsviz)
* [Taiga fontanka](https://tatianashavrina.github.io/taiga_site/)
* [News from Telegram contest](https://github.com/IlyaGusev/tgcontest)

**Script:** [create_ru_news.py](https://github.com/IlyaGusev/rulm/blob/master/data_processing/create_ru_news.py)

**Point of Contact:** [Ilya Gusev](ilya.gusev@phystech.edu)

**Languages:** Russian.


## Usage

Prerequisites:
```bash
pip install datasets zstandard jsonlines pysimdjson
```

Dataset iteration:
```python
from datasets import load_dataset
dataset = load_dataset('IlyaGusev/ru_news', split="train", streaming=True)
for example in dataset:
    print(example["text"])
```

## Data Instances

```
{
  "title": "Заместитель главы района в Якутии пожаловался на пьянство начальника",
  "text": "Заместитель главы Нерюнгринского района Якутии Геннадий Ленц пожаловался руководителю республики Егору Борисову на своего начальника. Как рассказал Ленц 'Интерфаксу', Андрей Фитисов пьет на рабочем месте и 'уходит в многодневные загулы'...",
  "timestamp": 1346284800,
  "url": "https://lenta.ru/news/2012/08/30/alco/",
  "source": "lenta"
}
```

## Personal and Sensitive Information

The dataset is not anonymized, so individuals' names can be found in the dataset. Information about the original authors is included in the dataset where possible.
task_categories:text-generation size_categories:1M<n<10M language:ru region:us
{"language": ["ru"], "size_categories": ["1M<n<10M"], "task_categories": ["text-generation"], "dataset_info": {"features": [{"name": "url", "dtype": "string"}, {"name": "text", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "source", "dtype": "string"}, {"name": "timestamp", "dtype": "uint64"}], "splits": [{"name": "train", "num_bytes": 12858731888, "num_examples": 4137525}], "download_size": 3669747077, "dataset_size": 12858731888}}
TAGS
#task_categories-text-generation #size_categories-1M<n<10M #language-Russian #region-us 


# RuNews dataset

## Table of Contents
- Table of Contents
- Description
- Usage
- Data Instances
- Personal and Sensitive Information

## Description

Summary: Dataset of news from several sources:
* URL by yutkin
* Several sources by buriy
* ODS Newsviz Tass
* Taiga fontanka
* News from Telegram contest

Script: create_ru_news.py

Point of Contact: Ilya Gusev

Languages: Russian.


## Usage

Prerequisites:


Dataset iteration:


## Data Instances



## Personal and Sensitive Information

The dataset is not anonymized, so individuals' names can be found in the dataset. Information about the original authors is included in the dataset where possible.
# RuNews dataset ## Table of Contents
- Table of Contents
- Description
- Usage
- Data Instances
- Personal and Sensitive Information ## Description

Summary: Dataset of news from several sources:
* URL by yutkin
* Several sources by buriy
* ODS Newsviz Tass
* Taiga fontanka
* News from Telegram contest

Script: create_ru_news.py

Point of Contact: Ilya Gusev

Languages: Russian. ## Usage

Prerequisites:


Dataset iteration: ## Data Instances ## Personal and Sensitive Information

The dataset is not anonymized, so individuals' names can be found in the dataset. Information about the original authors is included in the dataset where possible.
TAGS
#task_categories-text-generation #size_categories-1M<n<10M #language-Russian #region-us 
 # RuNews dataset ## Table of Contents
- Table of Contents
- Description
- Usage
- Data Instances
- Personal and Sensitive Information ## Description

Summary: Dataset of news from several sources:
* URL by yutkin
* Several sources by buriy
* ODS Newsviz Tass
* Taiga fontanka
* News from Telegram contest

Script: create_ru_news.py

Point of Contact: Ilya Gusev

Languages: Russian. ## Usage

Prerequisites:


Dataset iteration: ## Data Instances ## Personal and Sensitive Information

The dataset is not anonymized, so individuals' names can be found in the dataset. Information about the original authors is included in the dataset where possible.