text
We've just published a detailed blog post on the creation of Cosmopedia dataset. We hope this will provide insights about generating synthetic data at scale for pre-training. 
We've just published a detailed blog post on the creation of Cosmopedia dataset. We hope this will provide insights about generating synthetic data at scale for pre-training. 
new_line
link
https://huggingface.co/blog/cosmopedia
https://huggingface.co/blog/cosmopedia
new_line
new_line
text
Here are some key takeaways:  
Here are some key takeaways:  
new_line
text
🎯 Prompt curation is crucial: we want to cover many topics with few duplicates.
🎯 Prompt curation is crucial: we want to cover many topics with few duplicates.
new_line
text
📚 You can leverage various resources for diversity: using different seed data, generation formats, and target audiences.
📚 You can leverage various resources for diversity: using different seed data, generation formats, and target audiences.
new_line
text
⚙️ The importance of a good technical stack: for scalable generations with tools like llm-swarm and fast model training and evaluation.
⚙️ The importance of a good technical stack: for scalable generations with tools like llm-swarm and fast model training and evaluation.
new_line
new_line
text
Have a good read!
Have a good read!
We've just published a detailed blog post on the creation of Cosmopedia dataset. We hope this will provide insights about generating synthetic data at scale for pre-training. 
https://huggingface.co/blog/cosmopedia

Here are some key takeaways:  
🎯 Prompt curation is crucial: we want to cover many topics with few duplicates.
📚 You can leverage various resources for diversity: using different seed data, generation formats, and target audiences.
⚙️ The importance of a good technical stack: for scalable generations with tools like llm-swarm and fast model training and evaluation.

Have a good read!
🤗
alielfilali01 anton-l vwxyzjn kavalciemre samusenps osanseviero lvwerra xu3kev avinash02 thomwolf fpaupier manu muhtasham mjoshi malmenea AdinaY victor clefourrier molonelaveh
❤️
alielfilali01 anton-l vwxyzjn samusenps osanseviero xu3kev mmhamdy thomwolf muhtasham aloobun Salwa-Zeitoun AdinaY Dflare molonelaveh timpearce svallory
🔥
alielfilali01 anton-l vwxyzjn severo osanseviero xu3kev mmhamdy thomwolf muhtasham AdinaY ajibawa-2023 InferenceIllusionist mlabonne on1onmangoes molonelaveh
🤯
svallory
https://cdn-avatars.huggingface.co/v1/production/uploads/1651847561574-5fcaabed246881afd5b00167.jpeg
Muhtasham Oblokulov
muhtasham
user