
# Dataset Description

This dataset contains the first paragraph of cleaned Wikipedia articles in English. 

It was obtained by transorming the [Wikipedia](https://huggingface.co/datasets/wikipedia) "20220301.en" dataset as follows:
```python
from datasets import load_dataset

dataset = load_dataset("wikipedia", "20220301.en")["train"]

def get_first_paragraph(example):
  example["text"] = example['text'].split('\n\n')[0]
  return example

dataset = dataset.map(get_first_paragraph)
```

# Why use this dataset?
The size of the original English Wikipedia dataset is over 20GB. It takes 20min to load it on a Google Colab notebook and running computations on that dataset can be costly.

If you want to create a use case that mostly needs the information in the first paragraph of a Wikipedia article (which is the paragraph with the most important information), this 'wikipedia-first-paragraph' dataset is for you.

Its size is 1.39GB and it takes 5 min to load it on a Google colab notebook.


# How to load dataset

You can load it by runnning:
```python
from datasets import load_dataset

load_dataset("abokbot/wikipedia-first-paragraph")

```

# Dataset Structure
An example looks as follows:
```
{
  'id': '12',
  'url': 'https://en.wikipedia.org/wiki/Anarchism',
  'title': 'Anarchism',
  'text': 'Anarchism is a political philosophy and movement that is sceptical of authority and rejects \
          all involuntary, coercive forms of hierarchy. Anarchism calls for the abolition of the state, \
          which it holds to be unnecessary, undesirable, and harmful. As a historically left-wing movement, \
          placed on the farthest left of the political spectrum, it is usually described alongside communalism \
          and libertarian Marxism as the libertarian wing (libertarian socialism) of the socialist movement, and \
          has a strong historical association with anti-capitalism and socialism.'
}
```
language:en wikipedia region:us
{"language": ["en"], "tags": ["wikipedia"]}
TAGS
#language-English #wikipedia #region-us 


# Dataset Description

This dataset contains the first paragraph of cleaned Wikipedia articles in English. 

It was obtained by transorming the Wikipedia "URL" dataset as follows:


# Why use this dataset?
The size of the original English Wikipedia dataset is over 20GB. It takes 20min to load it on a Google Colab notebook and running computations on that dataset can be costly.

If you want to create a use case that mostly needs the information in the first paragraph of a Wikipedia article (which is the paragraph with the most important information), this 'wikipedia-first-paragraph' dataset is for you.

Its size is 1.39GB and it takes 5 min to load it on a Google colab notebook.


# How to load dataset

You can load it by runnning:


# Dataset Structure
An example looks as follows:

# Dataset Description

This dataset contains the first paragraph of cleaned Wikipedia articles in English. 

It was obtained by transorming the Wikipedia "URL" dataset as follows: # Why use this dataset?
The size of the original English Wikipedia dataset is over 20GB. It takes 20min to load it on a Google Colab notebook and running computations on that dataset can be costly.

If you want to create a use case that mostly needs the information in the first paragraph of a Wikipedia article (which is the paragraph with the most important information), this 'wikipedia-first-paragraph' dataset is for you.

Its size is 1.39GB and it takes 5 min to load it on a Google colab notebook. # How to load dataset

You can load it by runnning: # Dataset Structure
An example looks as follows:
TAGS
#language-English #wikipedia #region-us 
 # Dataset Description

This dataset contains the first paragraph of cleaned Wikipedia articles in English. 

It was obtained by transorming the Wikipedia "URL" dataset as follows: # Why use this dataset?
The size of the original English Wikipedia dataset is over 20GB. It takes 20min to load it on a Google Colab notebook and running computations on that dataset can be costly.

If you want to create a use case that mostly needs the information in the first paragraph of a Wikipedia article (which is the paragraph with the most important information), this 'wikipedia-first-paragraph' dataset is for you.

Its size is 1.39GB and it takes 5 min to load it on a Google colab notebook. # How to load dataset

You can load it by runnning: # Dataset Structure
An example looks as follows:
passage: TAGS
#language-English #wikipedia #region-us 
# Dataset Description

This dataset contains the first paragraph of cleaned Wikipedia articles in English. 

It was obtained by transorming the Wikipedia "URL" dataset as follows:# Why use this dataset?
The size of the original English Wikipedia dataset is over 20GB. It takes 20min to load it on a Google Colab notebook and running computations on that dataset can be costly.

If you want to create a use case that mostly needs the information in the first paragraph of a Wikipedia article (which is the paragraph with the most important information), this 'wikipedia-first-paragraph' dataset is for you.

Its size is 1.39GB and it takes 5 min to load it on a Google colab notebook.# How to load dataset

You can load it by runnning:# Dataset Structure
An example looks as follows: