text
Can we improve the quality of open LLMs for more languages?
Can we improve the quality of open LLMs for more languages?
new_line
new_line
text
Step 1: Evaluate current SOTA.
Step 1: Evaluate current SOTA.
new_line
new_line
text
The Data Is Better Together community has rated more than 10K prompts for quality. We now want to translate a subset of these to help address the language gap in model evals.
The Data Is Better Together community has rated more than 10K prompts for quality. We now want to translate a subset of these to help address the language gap in model evals.
new_line
new_line
text
The plan is roughly this:
The plan is roughly this:
new_line
new_line
text
- We started with 
- We started with 
link
https://huggingface.co/datasets/DIBT/10k_prompts_ranked
https://huggingface.co/datasets/DIBT/10k_prompts_ranked
text
 and took a subset of 500 high-quality prompts
 and took a subset of 500 high-quality prompts
new_line
text
- We're asking the community to translate these prompts into different languages
- We're asking the community to translate these prompts into different languages
new_line
text
- We'll evaluate the extent to which we can use AlpacaEval and similar approaches to rate the outputs of models across these different languages
- We'll evaluate the extent to which we can use AlpacaEval and similar approaches to rate the outputs of models across these different languages
new_line
text
- If it works well, we can more easily evaluate open LLMs across different languages by using a judge LLM to rate the quality of outputs from different models.
- If it works well, we can more easily evaluate open LLMs across different languages by using a judge LLM to rate the quality of outputs from different models.
new_line
new_line
text
You can find more details in our new GitHub repo: 
You can find more details in our new GitHub repo: 
link
https://github.com/huggingface/data-is-better-together
https://github.com/huggingface/data-is-better-together
text
 (don't forget to give it a ⭐!) 
 (don't forget to give it a ⭐!) 
Can we improve the quality of open LLMs for more languages?

Step 1: Evaluate current SOTA.

The Data Is Better Together community has rated more than 10K prompts for quality. We now want to translate a subset of these to help address the language gap in model evals.

The plan is roughly this:

- We started with https://huggingface.co/datasets/DIBT/10k_prompts_ranked and took a subset of 500 high-quality prompts
- We're asking the community to translate these prompts into different languages
- We'll evaluate the extent to which we can use AlpacaEval and similar approaches to rate the outputs of models across these different languages
- If it works well, we can more easily evaluate open LLMs across different languages by using a judge LLM to rate the quality of outputs from different models.

You can find more details in our new GitHub repo: https://github.com/huggingface/data-is-better-together (don't forget to give it a ⭐!) 
❤️
dvilasuero clefourrier samusenps sarahooker Joseph717171
🤗
dvilasuero sarahooker Joseph717171 seyf1elislam
🚀
dvilasuero sarahooker Joseph717171
🔥
dvilasuero sarahooker Joseph717171
🤝
victor sarahooker Joseph717171