Spaces:

open-llm-leaderboard

/

open_llm_leaderboard

like 12.3k

Running on CPU Upgrade

Hello @quincyqiang , if they don’t appear after they were supposedly “evaluated” then they encountered an error during the process and the evaluation did not push through.

If you’re looking for a particular model’s evaluation progress, it’s best to check out the requests page and look for the particular model’s eval request. If you see that the status is FAILED, then you report it back here so they can advise you on how to fix it or if it’s a problem on their end.

I checked some models, and they did fail. It seems that the tasks were all three days ago. Help me check if most of the tasks failed on July 31? Can you give me a specific reason, or how to fix it?

{"model": "LinkSoul/Chinese-Llama-2-7b", "base_model": "", "revision": "main", "private": false, "precision": "float16", "weight_type": "Original", "status": "FAILED", "submitted_time": "2023-07-31T01:52:02Z", "model_type": "fine-tuned", "job_id": "225981"}
https://huggingface.co/datasets/open-llm-leaderboard/requests/tree/main/LinkSoul

{"model": "golaxy/gowizardlm", "base_model": "", "revision": "main", "private": false, "precision": "float16", "weight_type": "Original", "status": "FAILED", "submitted_time": "2023-07-31T07:02:21Z", "model_type": "pretrained", "job_id": "226001"}

https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/golaxy/gowizardlm_eval_request_False_float16_Original.json

cc: @clefourrier

Is there any relevant information? Or can I delete these two models, and then I can submit the review again?

Hi @quincyqiang !
I'll investigate and come back to you today!

Thanks

@quincyqiang Just checked the logs, they were part of a faulty batch which has been removed since - you can resubmit :)

@quincyqiang Just checked the logs, they were part of a faulty batch which has been removed since - you can resubmit :)

Thanks you for effor，I would try again