text
🚨Exciting news for the Multilingual Synthetic Data Community!🚨
🚨Exciting news for the Multilingual Synthetic Data Community!🚨
new_line
new_line
text
I’ve taken inspiration from the MAGPIE paper on Llama-3-8B-instruct and extended its capabilities. Here’s what’s new!
I’ve taken inspiration from the MAGPIE paper on Llama-3-8B-instruct and extended its capabilities. Here’s what’s new!
new_line
new_line
text
🗞 The MAGPIE paper showcased that if you use the instruction-tuned version (
🗞 The MAGPIE paper showcased that if you use the instruction-tuned version (
inline_code
`Llama-3-8B-instruct`
Llama-3-8B-instruct
text
) to generate synthetic instructions and then fine-tune the base version (
) to generate synthetic instructions and then fine-tune the base version (
inline_code
`Llama-3-8B`
Llama-3-8B
text
) on this dataset, you can improve even the it-tuned version
) on this dataset, you can improve even the it-tuned version
new_line
new_line
text
🤔 While reading a script by Sebastian Raschka, PhD, I wondered: Could these advancements be replicated in other languages? Specifically, could they benefit non-English datasets?
🤔 While reading a script by Sebastian Raschka, PhD, I wondered: Could these advancements be replicated in other languages? Specifically, could they benefit non-English datasets?
new_line
new_line
text
🎉 And the answer is YES! At least for Spanish. I've successfully adapted the techniques for Spanish, proving the model's flexibility and multilingual capabilities.
🎉 And the answer is YES! At least for Spanish. I've successfully adapted the techniques for Spanish, proving the model's flexibility and multilingual capabilities.
new_line
new_line
text
👩‍💻 To make this accessible, I created a basic script (heavily inspired by the Sebastian Raschka one) that allows you to generate similar datasets using 
👩‍💻 To make this accessible, I created a basic script (heavily inspired by the Sebastian Raschka one) that allows you to generate similar datasets using 
inline_code
`ollama`
ollama
text
 models (initially phi and llama3) automatically and upload it to the Hugging Face Hub!
 models (initially phi and llama3) automatically and upload it to the Hugging Face Hub!
new_line
text
[Script](
[Script](
link
https://gist.github.com/mrm8488/4650a5e3cc45523798a527a3446eb312
https://gist.github.com/mrm8488/4650a5e3cc45523798a527a3446eb312
text
)
)
new_line
new_line
new_line
text
🔍 Explore the datasets 📚 generated using our new script! 
🔍 Explore the datasets 📚 generated using our new script! 
new_line
new_line
text
- [Llama-3-8B](
- [Llama-3-8B](
link
https://huggingface.co/datasets/mrm8488/dataset_llama3_5000_samples_es_4231_filtered
https://huggingface.co/datasets/mrm8488/dataset_llama3_5000_samples_es_4231_filtered
text
)
)
new_line
text
- [Phi-3-medium](
- [Phi-3-medium](
link
https://huggingface.co/datasets/mrm8488/dataset_phi3-medium_5000_samples_es_3906_filtered
https://huggingface.co/datasets/mrm8488/dataset_phi3-medium_5000_samples_es_3906_filtered
text
)
)
new_line
text
-  [Phi-3-mini](
-  [Phi-3-mini](
link
https://huggingface.co/datasets/mrm8488/dataset_phi3_5000_samples_es_3282_filtered
https://huggingface.co/datasets/mrm8488/dataset_phi3_5000_samples_es_3282_filtered
text
)
)
new_line
new_line
new_line
text
Note: These datasets have basic filtering. Apply additional quality filters before using them to fine-tune large language models.
Note: These datasets have basic filtering. Apply additional quality filters before using them to fine-tune large language models.
new_line
new_line
text
Inspiration and base script:
Inspiration and base script:
new_line
link
https://github.com/rasbt/LLMs-from-scratch/blob/main/ch07/05_dataset-generation/llama3-ollama.ipynb
https://github.com/rasbt/LLMs-from-scratch/blob/main/ch07/05_dataset-generation/llama3-ollama.ipynb
new_line
link
https://www.linkedin.com/feed/update/urn:li:activity:7210982019751661568/
https://www.linkedin.com/feed/update/urn:li:activity:7210982019751661568/
new_line
🚨Exciting news for the Multilingual Synthetic Data Community!🚨

I’ve taken inspiration from the MAGPIE paper on Llama-3-8B-instruct and extended its capabilities. Here’s what’s new!

🗞 The MAGPIE paper showcased that if you use the instruction-tuned version (`Llama-3-8B-instruct`) to generate synthetic instructions and then fine-tune the base version (`Llama-3-8B`) on this dataset, you can improve even the it-tuned version

🤔 While reading a script by Sebastian Raschka, PhD, I wondered: Could these advancements be replicated in other languages? Specifically, could they benefit non-English datasets?

🎉 And the answer is YES! At least for Spanish. I've successfully adapted the techniques for Spanish, proving the model's flexibility and multilingual capabilities.

👩‍💻 To make this accessible, I created a basic script (heavily inspired by the Sebastian Raschka one) that allows you to generate similar datasets using `ollama` models (initially phi and llama3) automatically and upload it to the Hugging Face Hub!
[Script](https://gist.github.com/mrm8488/4650a5e3cc45523798a527a3446eb312)


🔍 Explore the datasets 📚 generated using our new script! 

- [Llama-3-8B](https://huggingface.co/datasets/mrm8488/dataset_llama3_5000_samples_es_4231_filtered)
- [Phi-3-medium](https://huggingface.co/datasets/mrm8488/dataset_phi3-medium_5000_samples_es_3906_filtered)
-  [Phi-3-mini](https://huggingface.co/datasets/mrm8488/dataset_phi3_5000_samples_es_3282_filtered)


Note: These datasets have basic filtering. Apply additional quality filters before using them to fine-tune large language models.

Inspiration and base script:
https://github.com/rasbt/LLMs-from-scratch/blob/main/ch07/05_dataset-generation/llama3-ollama.ipynb
https://www.linkedin.com/feed/update/urn:li:activity:7210982019751661568/

❤️
Davipar osanseviero apol anakin87 davanstrien victor ucsahin floschne GPT007 vikas Marvin73 pavaldeveloper mrm8488 Taylor658 adorkin
https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png
Omar Sanseviero
osanseviero
user
https://cdn-avatars.huggingface.co/v1/production/uploads/5e4318d616b09a31220980d6/24rMJ_vPh3gW9ZEmj64xr.png
Manuel Romero
mrm8488
user
/avatars/53ebfcab852efd849a848a26dc65751c.svg
elsatch
elsatch
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1627505688463-60107b385ac3e86b3ea4fc34.jpeg
Daniel van Strien
davanstrien
user
https://cdn-avatars.huggingface.co/v1/production/uploads/5fc6879e1c5ee87b1164876d/Tjnm_lv0Bq0gPbFOTDH6E.jpeg
Huu Nguyen
huu-ontocord
user