text
🦫 We have just released 
🦫 We have just released 
resource
https://huggingface.co/datasets/argilla/Capybara-Preferences
dataset
argilla/Capybara-Preferences
https://huggingface.co/datasets/argilla/Capybara-Preferences
text
 in collaboration with Kaist AI (
 in collaboration with Kaist AI (
mention
@JW17
JW17
text
, 
, 
mention
@nlee-208
nlee-208
text
) and Hugging Face (
) and Hugging Face (
mention
@lewtun
lewtun
text
)
)
new_line
new_line
text
A new synthetic preference dataset built using 
A new synthetic preference dataset built using 
inline_code
`distilabel`
distilabel
text
 on top of the awesome 
 on top of the awesome 
resource
https://huggingface.co/datasets/LDJnr/Capybara
dataset
LDJnr/Capybara
https://huggingface.co/datasets/LDJnr/Capybara
text
 from 
 from 
mention
@LDJnr
LDJnr
text
new_line
new_line
text
The current dataset combines the already generated alternative completions from 
The current dataset combines the already generated alternative completions from 
resource
https://huggingface.co/datasets/argilla/distilabel-capybara-dpo-7k-binarized
dataset
argilla/distilabel-capybara-dpo-7k-binarized
https://huggingface.co/datasets/argilla/distilabel-capybara-dpo-7k-binarized
text
, while also adding the remaining ones using the same approach!
, while also adding the remaining ones using the same approach!
new_line
new_line
text
Here are some key features on how we built it:
Here are some key features on how we built it:
new_line
new_line
text
- 🧹 Duplicate removal, keeping the conversation besides the last assistant response, and some slight pre-processing
- 🧹 Duplicate removal, keeping the conversation besides the last assistant response, and some slight pre-processing
new_line
new_line
text
- 🤖 Generation of alternative completions for the existing conversations (last turn only) with: 
- 🤖 Generation of alternative completions for the existing conversations (last turn only) with: 
resource
https://huggingface.co/mlabonne/NeuralBeagle14-7B
model
mlabonne/NeuralBeagle14-7B
https://huggingface.co/mlabonne/NeuralBeagle14-7B
text
, 
, 
resource
https://huggingface.co/argilla/notus-7b-v1
model
argilla/notus-7b-v1
https://huggingface.co/argilla/notus-7b-v1
text
, and 
, and 
resource
https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B
model
teknium/OpenHermes-2.5-Mistral-7B
https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B
new_line
new_line
text
- 👨🏻‍🏫 Running UltraFeedback via GPT-4 to generate the critique i.e. ratings and rationales, for the last assistant responses
- 👨🏻‍🏫 Running UltraFeedback via GPT-4 to generate the critique i.e. ratings and rationales, for the last assistant responses
new_line
new_line
text
- 🎉 Finally, we selected the chosen and rejected responses based on their UltraFeedback score, and applied some slight post-processing!
- 🎉 Finally, we selected the chosen and rejected responses based on their UltraFeedback score, and applied some slight post-processing!
new_line
new_line
text
Sounds simple right? Start building your own synthetic datasets with 
Sounds simple right? Start building your own synthetic datasets with 
link
https://github.com/argilla-io/distilabel
https://github.com/argilla-io/distilabel
text
 already!
 already!
🦫 We have just released https://huggingface.co/datasets/argilla/Capybara-Preferences in collaboration with Kaist AI (@JW17, @nlee-208) and Hugging Face (@lewtun)

A new synthetic preference dataset built using `distilabel` on top of the awesome https://huggingface.co/datasets/LDJnr/Capybara from @LDJnr 

The current dataset combines the already generated alternative completions from https://huggingface.co/datasets/argilla/distilabel-capybara-dpo-7k-binarized, while also adding the remaining ones using the same approach!

Here are some key features on how we built it:

- 🧹 Duplicate removal, keeping the conversation besides the last assistant response, and some slight pre-processing

- 🤖 Generation of alternative completions for the existing conversations (last turn only) with: https://huggingface.co/mlabonne/NeuralBeagle14-7B, https://huggingface.co/argilla/notus-7b-v1, and https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B

- 👨🏻‍🏫 Running UltraFeedback via GPT-4 to generate the critique i.e. ratings and rationales, for the last assistant responses

- 🎉 Finally, we selected the chosen and rejected responses based on their UltraFeedback score, and applied some slight post-processing!

Sounds simple right? Start building your own synthetic datasets with https://github.com/argilla-io/distilabel already!
🔥
lunarflu davanstrien tomaarsen qubvel-hf clem alielfilali01 AtAndDev gabrielmbmb
👍
mikeysilva808 AtAndDev gabrielmbmb