text
Cohere for AI, Argilla, and Hugging Face are collaborating on an Open Science Project to enhance multilingual model evaluations. The project focuses on the widely-used MMLU dataset, which spans 57 subjects like mathematics, computer science, and law. However, existing translations often miss linguistic and cultural nuances, thus embedding biases. 🤔
Cohere for AI, Argilla, and Hugging Face are collaborating on an Open Science Project to enhance multilingual model evaluations. The project focuses on the widely-used MMLU dataset, which spans 57 subjects like mathematics, computer science, and law. However, existing translations often miss linguistic and cultural nuances, thus embedding biases. 🤔
new_line
new_line
text
To address this, they have annotated a subset of the MMLU test set and are inviting global perspectives to review prompts, highlighting cultural specifics and required knowledge. They have mentioned that insights will help shape future multilingual model evaluations, ensuring they are more inclusive and accurate. 🗺️ 📝 🙌 
To address this, they have annotated a subset of the MMLU test set and are inviting global perspectives to review prompts, highlighting cultural specifics and required knowledge. They have mentioned that insights will help shape future multilingual model evaluations, ensuring they are more inclusive and accurate. 🗺️ 📝 🙌 
new_line
new_line
text
▶️ To get started go to: 
▶️ To get started go to: 
resource
https://huggingface.co/spaces/CohereForAI/MMLU-evaluation
space
CohereForAI/MMLU-evaluation
https://huggingface.co/spaces/CohereForAI/MMLU-evaluation
new_line
new_line
text
🌍 They also have an Aya Discord server for collaboration with other participants: 
🌍 They also have an Aya Discord server for collaboration with other participants: 
link
https://discord.gg/9gVhdfnQMN
https://discord.gg/9gVhdfnQMN
new_line
Cohere for AI, Argilla, and Hugging Face are collaborating on an Open Science Project to enhance multilingual model evaluations. The project focuses on the widely-used MMLU dataset, which spans 57 subjects like mathematics, computer science, and law. However, existing translations often miss linguistic and cultural nuances, thus embedding biases. 🤔

To address this, they have annotated a subset of the MMLU test set and are inviting global perspectives to review prompts, highlighting cultural specifics and required knowledge. They have mentioned that insights will help shape future multilingual model evaluations, ensuring they are more inclusive and accurate. 🗺️ 📝 🙌 

▶️ To get started go to: https://huggingface.co/spaces/CohereForAI/MMLU-evaluation

🌍 They also have an Aya Discord server for collaboration with other participants: https://discord.gg/9gVhdfnQMN

👍
victor louisbrulenaudet lunarflu