
# xlsum

- Source: https://huggingface.co/datasets/GEM/xlsum
- Num examples: 
  - 306,521 (train)
  - 11,535 (validation)
  - 11,535 (test)
- Language: English

```python
from datasets import load_dataset

load_dataset("tdtunlp/xlsum_en")
```

- Format for Summarization task

```python
def preprocess(sample):
    title = sample['title']
    article = sample['text']
    summary = sample['target']
    return {'text': f'<|startoftext|><|title|>{title}<|article|>{article}<|summary|>{summary}<|endoftext|>'}

"""
<|startoftext|><|title|>Weather alert issued for gale force winds in Wales<|article|>The Met Office has issued a yellow weather warning for wind covering Wales and England, starting from 21:00 GMT on Wednesday evening.

Travel and power are both likely to be disrupted, with the warning to remain in place until 15:00 on Thursday.

Gusts of 55mph (88kmh) are likely and could hit up to 70mph on coasts and hills, with heavy and blustery showers.

<|summary|>Winds could reach gale force in Wales with stormy weather set to hit the whole of the country this week.<|endoftext|>
"""
```
task_categories:summarization size_categories:100K<n<1M language:en region:us
{"language": ["en"], "size_categories": ["100K<n<1M"], "task_categories": ["summarization"], "dataset_info": {"features": [{"name": "gem_id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "target", "dtype": "string"}, {"name": "references", "list": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 975957381, "num_examples": 306521}, {"name": "test", "num_bytes": 34940457, "num_examples": 11535}, {"name": "validation", "num_bytes": 35201391, "num_examples": 11535}], "download_size": 655141522, "dataset_size": 1046099229}}
TAGS
#task_categories-summarization #size_categories-100K<n<1M #language-English #region-us 


# xlsum

- Source: URL
- Num examples: 
  - 306,521 (train)
  - 11,535 (validation)
  - 11,535 (test)
- Language: English



- Format for Summarization task


# xlsum

- Source: URL
- Num examples: 
  - 306,521 (train)
  - 11,535 (validation)
  - 11,535 (test)
- Language: English



- Format for Summarization task
TAGS
#task_categories-summarization #size_categories-100K<n<1M #language-English #region-us 
 # xlsum

- Source: URL
- Num examples: 
  - 306,521 (train)
  - 11,535 (validation)
  - 11,535 (test)
- Language: English



- Format for Summarization task