task_categories:text-classification task_categories:question-answering task_categories:text-generation size_categories:100K<n<1M license:mit code art music legal finance biology chemistry arxiv:2306.02707 arxiv:2301.13688 region:us
---
license: mit
task_categories:
- text-classification
- question-answering
- text-generation
tags:
- code
- art
- music
- legal
- finance
- biology
- chemistry
pretty_name: SlimOrca Dedup
size_categories:
- 100K<n<1M
---
----

# Overview

----

"SlimOrca Dedup" is a deduplicated, unfiltered subset of the SlimOrca dataset, excluding RLHF instances, resulting in 363k unique examples.

# Key Features

- Removal of RLHF instances.
- Deduplication using minhash and Jaccard similarity techniques.

# Demo Models

Note: These models were trained on the full SlimOrca dataset, not the deduplicated, unfiltered version.
* https://huggingface.co/openaccess-ai-collective/jackalope-7b
* https://huggingface.co/Open-Orca/Mistral-7B-SlimOrca
  
----

# Dataset format

----

 **Basic Structure**

 This dataset uses basic sharegpt formatting. Example and explanation of the schema is below:
  
  ```json
  {
      "conversations": [
          {"from": "system", "value": "You are an AI assistant..."},
          {"from": "human", "value": "Write an article based on this..."},
          {"from": "gpt", "value": "Title: Tragedy Strikes in Sydney..."}
      ]
  }
  ```
  
   **Message Formatting**
  
  - **"from"**: A string indicating the sender of the message. Possible senders are "system", "human", and "gpt".
  - **"value"**: A string containing the message or instruction from the sender.
  
   **Message roles**
  
  - ** System: ** The system provides instructions or guidelines for the task to the large language model (LLM). 
  - ** Human: ** The human provides prompts or queries for the AI model to respond to.
  - ** GPT: ** The language model, generates responses or content based on the prompts or queries provided by the human. messages from this role only ever follow messages from the human role.



----

# Citation

----
```bibtex
@misc{SlimOrcaDedup,
  title = {SlimOrca Dedup: A Deduplicated Subset of SlimOrca},
  author = {Wing Lian and Guan Wang and Bleys Goodson and Eugene Pentland and Austin Cook and Chanvichet Vong and "Teknium" and Nathan Hoos},
  year = {2023},
  publisher = {HuggingFace},
  url = {https://huggingface.co/datasets/unaidedelf87777/SlimOrca-dedup-unfiltered/}
}

@misc{mukherjee2023orca,
      title={Orca: Progressive Learning from Complex Explanation Traces of GPT-4}, 
      author={Subhabrata Mukherjee and Arindam Mitra and Ganesh Jawahar and Sahaj Agarwal and Hamid Palangi and Ahmed Awadallah},
      year={2023},
      eprint={2306.02707},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

@misc{longpre2023flan,
      title={The Flan Collection: Designing Data and Methods for Effective Instruction Tuning}, 
      author={Shayne Longpre and Le Hou and Tu Vu and Albert Webson and Hyung Won Chung and Yi Tay and Denny Zhou and Quoc V. Le and Barret Zoph and Jason Wei and Adam Roberts},
      year={2023},
      eprint={2301.13688},
      archivePrefix={arXiv},
      primaryClass={cs.AI}
}

```