
# ELI5 paired

This is a processed version of the [`eli5`](https://huggingface.co/datasets/eli5) dataset.
Compared to ["eli5_rlhf"](https://huggingface.co/datasets/vincentmin/eli5_rlhf), this dataset contains only QA pairs from the train split of the eli5 dataset and only from the subreddit explainlikeimfive.
Furthermore, the function
```
def get_question(example):
    title = example["title"]
    selftext = example["selftext"]
    if selftext:
        if selftext[-1] not in [".", "?", "!"]:
            seperator = ". "
        else:
            seperator = " "
        question = title + seperator + selftext
    else:
        question = title
    example["question"] = question
    return example
```
was applied to get the "question" column and the "title" and "selftext" columns were removed.
The dataset was created following very closely the steps in the [`stack-exchange-paired`](https://huggingface.co/datasets/lvwerra/stack-exchange-paired) dataset.
The following steps were applied:

- The "question" field is a concatenation of "title" with "selftext".
- Create pairs `(response_j, response_k)` where j was rated better than k
- Sample at most 10 pairs per question
- Shuffle the dataset globally

This dataset is designed to be used for preference learning. The processing notebook is in the repository as well.
task_categories:text-generation task_categories:question-answering size_categories:100K<n<1M language:en region:us
{"language": ["en"], "size_categories": ["100K<n<1M"], "task_categories": ["text-generation", "question-answering"], "pretty_name": "Reddit Explain Like I'm 5 for Reinforcement Learning Human Feedback"}
TAGS
#task_categories-text-generation #task_categories-question-answering #size_categories-100K<n<1M #language-English #region-us 


# ELI5 paired

This is a processed version of the 'eli5' dataset.
Compared to "eli5_rlhf", this dataset contains only QA pairs from the train split of the eli5 dataset and only from the subreddit explainlikeimfive.
Furthermore, the function

was applied to get the "question" column and the "title" and "selftext" columns were removed.
The dataset was created following very closely the steps in the 'stack-exchange-paired' dataset.
The following steps were applied:

- The "question" field is a concatenation of "title" with "selftext".
- Create pairs '(response_j, response_k)' where j was rated better than k
- Sample at most 10 pairs per question
- Shuffle the dataset globally

This dataset is designed to be used for preference learning. The processing notebook is in the repository as well.
# ELI5 paired

This is a processed version of the 'eli5' dataset.
Compared to "eli5_rlhf", this dataset contains only QA pairs from the train split of the eli5 dataset and only from the subreddit explainlikeimfive.
Furthermore, the function

was applied to get the "question" column and the "title" and "selftext" columns were removed.
The dataset was created following very closely the steps in the 'stack-exchange-paired' dataset.
The following steps were applied:

- The "question" field is a concatenation of "title" with "selftext".
- Create pairs '(response_j, response_k)' where j was rated better than k
- Sample at most 10 pairs per question
- Shuffle the dataset globally

This dataset is designed to be used for preference learning. The processing notebook is in the repository as well.
TAGS
#task_categories-text-generation #task_categories-question-answering #size_categories-100K<n<1M #language-English #region-us 
 # ELI5 paired

This is a processed version of the 'eli5' dataset.
Compared to "eli5_rlhf", this dataset contains only QA pairs from the train split of the eli5 dataset and only from the subreddit explainlikeimfive.
Furthermore, the function

was applied to get the "question" column and the "title" and "selftext" columns were removed.
The dataset was created following very closely the steps in the 'stack-exchange-paired' dataset.
The following steps were applied:

- The "question" field is a concatenation of "title" with "selftext".
- Create pairs '(response_j, response_k)' where j was rated better than k
- Sample at most 10 pairs per question
- Shuffle the dataset globally

This dataset is designed to be used for preference learning. The processing notebook is in the repository as well.