text
Llama 3.1 405B Instruct beats GPT-4o on MixEval-Hard
Llama 3.1 405B Instruct beats GPT-4o on MixEval-Hard
new_line
new_line
text
Just ran MixEval for 405B, Sonnet-3.5 and 4o, with 405B landing right between the other two at 66.19
Just ran MixEval for 405B, Sonnet-3.5 and 4o, with 405B landing right between the other two at 66.19
new_line
new_line
text
The GPT-4o result of 64.7 replicated locally but Sonnet-3.5 actually scored 70.25/69.45 in my replications 🤔 Still well ahead of the other 2 though.
The GPT-4o result of 64.7 replicated locally but Sonnet-3.5 actually scored 70.25/69.45 in my replications 🤔 Still well ahead of the other 2 though.
new_line
new_line
text
Sammple of 1 of the eval calls here: 
Sammple of 1 of the eval calls here: 
link
https://wandb.ai/morgan/MixEval/weave/calls/07b05ae2-2ef5-4525-98a6-c59963b76fe1
https://wandb.ai/morgan/MixEval/weave/calls/07b05ae2-2ef5-4525-98a6-c59963b76fe1
new_line
new_line
text
Quick auto-logging tracing for openai-compatible clients and many more here: 
Quick auto-logging tracing for openai-compatible clients and many more here: 
link
https://wandb.github.io/weave/quickstart/
https://wandb.github.io/weave/quickstart/
new_line
new_line
Llama 3.1 405B Instruct beats GPT-4o on MixEval-Hard

Just ran MixEval for 405B, Sonnet-3.5 and 4o, with 405B landing right between the other two at 66.19

The GPT-4o result of 64.7 replicated locally but Sonnet-3.5 actually scored 70.25/69.45 in my replications 🤔 Still well ahead of the other 2 though.

Sammple of 1 of the eval calls here: https://wandb.ai/morgan/MixEval/weave/calls/07b05ae2-2ef5-4525-98a6-c59963b76fe1

Quick auto-logging tracing for openai-compatible clients and many more here: https://wandb.github.io/weave/quickstart/


👍
zenosai macsz Corvius
🔥
Rohitkhatri75436