size_categories:100K<n<1M language:en license:mit region:us
---
license: mit
language:
- en
size_categories:
- 100K<n<1M
---

# RLAIF Dataset

## Schema:
```
{
    prompt: str,  // in format "\n\nHuman: ... \n\nAssistant: "
    answers: [
        {
            answer: str,
            model: str  // name of model that generated the response
        },
        ...
        {
            answer: str,
            model: str  // name of model that generated the response
        },
    ]
    turns: int,
    num_response: int,
    source: list[str] // list of the dataset sources for the prompt and answers.
}
```
## Process
  1. For each dataset, generate prompt and answer pairs.
  2. For each dataset, group by prompt.
  3. Concatenate datasets from (2), down sample according to the following.
     a. Take all ShareGPT prompts.
     b. Randomly sample without replacement 75,000 Anthropic HH prompts.
     c. Take all Ultrafeedback prompts.
     d. Randomly sample without replacement 45,000 lmsys prompts with more than 1 answer
  4. Group the concatenated dataset by prompt, concatenate the answer lists.
  5. Take all prompts that are longer than 40 characters and have more than 1 answer

Notes: 
  1. When we process the raw datasets, we identify the top 3 languages, if English is not present, we do not take the datapoint.
  2. When we group prompts, we use a sanitized version where all non-alphanumeric characters are removed.

## Stats

<img src="sources.png" alt="alt text" width="600"/>