text
KTO offers an easier way to preference train LLMs (only 👍👎 ratings are required). As part of #DataIsBetterTogether, I've written a tutorial on creating a preference dataset using Argilla and Spaces. 
KTO offers an easier way to preference train LLMs (only 👍👎 ratings are required). As part of #DataIsBetterTogether, I've written a tutorial on creating a preference dataset using Argilla and Spaces. 
new_line
new_line
text
Using this approach, you can create a dataset that anyone with a Hugging Face account can contribute to 🤯
Using this approach, you can create a dataset that anyone with a Hugging Face account can contribute to 🤯
new_line
new_line
text
See an example of the kind of Space you can create following this tutorial here: 
See an example of the kind of Space you can create following this tutorial here: 
resource
https://huggingface.co/spaces/davanstrien/haiku-preferences
space
davanstrien/haiku-preferences
https://huggingface.co/spaces/davanstrien/haiku-preferences
new_line
new_line
text
🆕 New tutorial covers:
🆕 New tutorial covers:
new_line
text
💬 Generating responses with open models
💬 Generating responses with open models
new_line
text
👥 Collecting human feedback (do you like this model response? Yes/No)
👥 Collecting human feedback (do you like this model response? Yes/No)
new_line
text
🤖 Preparing a TRL-compatible dataset for training aligned models
🤖 Preparing a TRL-compatible dataset for training aligned models
new_line
new_line
text
Check it out here: 
Check it out here: 
link
https://github.com/huggingface/data-is-better-together/tree/main/kto-preference
https://github.com/huggingface/data-is-better-together/tree/main/kto-preference
KTO offers an easier way to preference train LLMs (only 👍👎 ratings are required). As part of #DataIsBetterTogether, I've written a tutorial on creating a preference dataset using Argilla and Spaces. 

Using this approach, you can create a dataset that anyone with a Hugging Face account can contribute to 🤯

See an example of the kind of Space you can create following this tutorial here: https://huggingface.co/spaces/davanstrien/haiku-preferences

🆕 New tutorial covers:
💬 Generating responses with open models
👥 Collecting human feedback (do you like this model response? Yes/No)
🤖 Preparing a TRL-compatible dataset for training aligned models

Check it out here: https://github.com/huggingface/data-is-better-together/tree/main/kto-preference
👍
merve transZ osanseviero AlekseiPravdin jaigouk JJhooww jayomb mmhamdy AtAndDev medmac01 pcuenq mathiasn1 michaelbenayoun dball ZennyKenny Cossale
🔥
AtAndDev dark-pen medmac01 michaelbenayoun clem
https://cdn-avatars.huggingface.co/v1/production/uploads/615c231c3a60fa8486f80634/t-kcY2gsYVcwrZrsTc0Fz.jpeg
David Faragó
dball
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1627505688463-60107b385ac3e86b3ea4fc34.jpeg
Daniel van Strien
davanstrien
user