![HuggingFace LeaderBoard](https://cdn-uploads.huggingface.co/production/uploads/6202a599216215a22221dea9/Uh5JX7Kq-rUxoVrdsV-M-.gif)
# Open LLM Leaderboard Requests

This repository contains the request files of models that have been submitted to the Open LLM Leaderboard.

You can take a look at the current status of your model by finding its request file in this dataset. If your model failed, feel free to open an issue on the Open LLM Leaderboard! (We don't follow issues in this repository as often)

## Evaluation Methodology
The evaluation process involves running your models against several benchmarks from the Eleuther AI Harness, a unified framework for measuring the effectiveness of generative language models. Below is a brief overview of each benchmark:

1. AI2 Reasoning Challenge (ARC) - Grade-School Science Questions (25-shot)
2. HellaSwag - Commonsense Inference (10-shot)
3. MMLU - Massive Multi-Task Language Understanding, knowledge on 57 domains (5-shot)
4. TruthfulQA - Propensity to Produce Falsehoods (0-shot)
5. Winogrande - Adversarial Winograd Schema Challenge (5-shot)
6. GSM8k - Grade School Math Word Problems Solving Complex Mathematical Reasoning (5-shot)

Together, these benchmarks provide an assessment of a model's capabilities in terms of knowledge, reasoning, and some math, in various scenarios.

## Accessing Your Results
To view the numerical results of your evaluated models, visit the dedicated Hugging Face Dataset at https://huggingface.co/datasets/open-llm-leaderboard/results. This dataset offers a thorough breakdown of each model's performance on the individual benchmarks.

## Exploring Model Details
For further insights into the inputs and outputs of specific models, locate the "📄" emoji associated with the desired model within this repository. Clicking on this icon will direct you to the respective GitHub page containing detailed information about the model's behavior during the evaluation process.

license:apache-2.0 region:us
{"license": "apache-2.0"}
TAGS
#license-apache-2.0 #region-us 

!HuggingFace LeaderBoard
# Open LLM Leaderboard Requests

This repository contains the request files of models that have been submitted to the Open LLM Leaderboard.

You can take a look at the current status of your model by finding its request file in this dataset. If your model failed, feel free to open an issue on the Open LLM Leaderboard! (We don't follow issues in this repository as often)

## Evaluation Methodology
The evaluation process involves running your models against several benchmarks from the Eleuther AI Harness, a unified framework for measuring the effectiveness of generative language models. Below is a brief overview of each benchmark:

1. AI2 Reasoning Challenge (ARC) - Grade-School Science Questions (25-shot)
2. HellaSwag - Commonsense Inference (10-shot)
3. MMLU - Massive Multi-Task Language Understanding, knowledge on 57 domains (5-shot)
4. TruthfulQA - Propensity to Produce Falsehoods (0-shot)
5. Winogrande - Adversarial Winograd Schema Challenge (5-shot)
6. GSM8k - Grade School Math Word Problems Solving Complex Mathematical Reasoning (5-shot)

Together, these benchmarks provide an assessment of a model's capabilities in terms of knowledge, reasoning, and some math, in various scenarios.

## Accessing Your Results
To view the numerical results of your evaluated models, visit the dedicated Hugging Face Dataset at URL This dataset offers a thorough breakdown of each model's performance on the individual benchmarks.

## Exploring Model Details
For further insights into the inputs and outputs of specific models, locate the "" emoji associated with the desired model within this repository. Clicking on this icon will direct you to the respective GitHub page containing detailed information about the model's behavior during the evaluation process.

# Open LLM Leaderboard Requests

This repository contains the request files of models that have been submitted to the Open LLM Leaderboard.

You can take a look at the current status of your model by finding its request file in this dataset. If your model failed, feel free to open an issue on the Open LLM Leaderboard! (We don't follow issues in this repository as often) ## Evaluation Methodology
The evaluation process involves running your models against several benchmarks from the Eleuther AI Harness, a unified framework for measuring the effectiveness of generative language models. Below is a brief overview of each benchmark:

1. AI2 Reasoning Challenge (ARC) - Grade-School Science Questions (25-shot)
2. HellaSwag - Commonsense Inference (10-shot)
3. MMLU - Massive Multi-Task Language Understanding, knowledge on 57 domains (5-shot)
4. TruthfulQA - Propensity to Produce Falsehoods (0-shot)
5. Winogrande - Adversarial Winograd Schema Challenge (5-shot)
6. GSM8k - Grade School Math Word Problems Solving Complex Mathematical Reasoning (5-shot)

Together, these benchmarks provide an assessment of a model's capabilities in terms of knowledge, reasoning, and some math, in various scenarios. ## Accessing Your Results
To view the numerical results of your evaluated models, visit the dedicated Hugging Face Dataset at URL This dataset offers a thorough breakdown of each model's performance on the individual benchmarks. ## Exploring Model Details
For further insights into the inputs and outputs of specific models, locate the "" emoji associated with the desired model within this repository. Clicking on this icon will direct you to the respective GitHub page containing detailed information about the model's behavior during the evaluation process.
TAGS
#license-apache-2.0 #region-us 
 # Open LLM Leaderboard Requests

This repository contains the request files of models that have been submitted to the Open LLM Leaderboard.

You can take a look at the current status of your model by finding its request file in this dataset. If your model failed, feel free to open an issue on the Open LLM Leaderboard! (We don't follow issues in this repository as often) ## Evaluation Methodology
The evaluation process involves running your models against several benchmarks from the Eleuther AI Harness, a unified framework for measuring the effectiveness of generative language models. Below is a brief overview of each benchmark:

1. AI2 Reasoning Challenge (ARC) - Grade-School Science Questions (25-shot)
2. HellaSwag - Commonsense Inference (10-shot)
3. MMLU - Massive Multi-Task Language Understanding, knowledge on 57 domains (5-shot)
4. TruthfulQA - Propensity to Produce Falsehoods (0-shot)
5. Winogrande - Adversarial Winograd Schema Challenge (5-shot)
6. GSM8k - Grade School Math Word Problems Solving Complex Mathematical Reasoning (5-shot)

Together, these benchmarks provide an assessment of a model's capabilities in terms of knowledge, reasoning, and some math, in various scenarios. ## Accessing Your Results
To view the numerical results of your evaluated models, visit the dedicated Hugging Face Dataset at URL This dataset offers a thorough breakdown of each model's performance on the individual benchmarks. ## Exploring Model Details
For further insights into the inputs and outputs of specific models, locate the "" emoji associated with the desired model within this repository. Clicking on this icon will direct you to the respective GitHub page containing detailed information about the model's behavior during the evaluation process.