text
🔥 New LLM leaderboard on the hub: NPHardEval!
🔥 New LLM leaderboard on the hub: NPHardEval!
new_line
new_line
text
It uses questions of logic, of different mathematical complexities, as a proxy for reasoning abilities. It notably removes questions relying on arithmetic, to really focus on logical abilities. 
It uses questions of logic, of different mathematical complexities, as a proxy for reasoning abilities. It notably removes questions relying on arithmetic, to really focus on logical abilities. 
new_line
text
What's interesting imo is the potential to really study a model performance at different levels of complexity.  
What's interesting imo is the potential to really study a model performance at different levels of complexity.  
new_line
new_line
text
Bonus: Since the questions can be generated automatically, it's going to be dynamic, updated monthly! 🚀
Bonus: Since the questions can be generated automatically, it's going to be dynamic, updated monthly! 🚀
new_line
resource
https://huggingface.co/spaces/NPHardEval/NPHardEval-leaderboard
space
NPHardEval/NPHardEval-leaderboard
https://huggingface.co/spaces/NPHardEval/NPHardEval-leaderboard
new_line
new_line
text
Read more about how their questions are generated in the intro blog: 
Read more about how their questions are generated in the intro blog: 
link
https://huggingface.co/blog/leaderboards-on-the-hub-nphardeval
https://huggingface.co/blog/leaderboards-on-the-hub-nphardeval
new_line
new_line
text
Congrats to 
Congrats to 
mention
@lizhouf
lizhouf
text
, 
, 
mention
@wenyueH
wenyueH
text
, 
, 
mention
@hyfrankl
hyfrankl
text
 and their teams!
 and their teams!
🔥 New LLM leaderboard on the hub: NPHardEval!

It uses questions of logic, of different mathematical complexities, as a proxy for reasoning abilities. It notably removes questions relying on arithmetic, to really focus on logical abilities. 
What's interesting imo is the potential to really study a model performance at different levels of complexity.  

Bonus: Since the questions can be generated automatically, it's going to be dynamic, updated monthly! 🚀
https://huggingface.co/spaces/NPHardEval/NPHardEval-leaderboard

Read more about how their questions are generated in the intro blog: https://huggingface.co/blog/leaderboards-on-the-hub-nphardeval

Congrats to @lizhouf, @wenyueH, @hyfrankl and their teams!
🤗
alvarobartt osanseviero alielfilali01 samusenps lizhouf diwank hyfrankl kramp joaogante hivaze Kukedlc sbrandeis
🤝
osanseviero
🤯
Kukedlc