![image/png](https://cdn-uploads.huggingface.co/production/uploads/6468ce47e134d050a58aa89c/u0jZZVpCxq3JN8VzFXVhV.png)

# Nous-Hermes-2-SUS-Chat-34B-Linear

This is the model for Nous-Hermes-2-SUS-Chat-34B-Linear. I used [mergekit](https://github.com/cg123/mergekit) to merge models.

# Yaml Config

```yaml

models:
  - model: Nous-Hermes-2-Yi-34B
    parameters:
      weight: 0.5
  - model: SUS-Chat-34B
    parameters:
      weight: 0.3
merge_method: linear
dtype: bfloat16

```
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Weyaxi__Nous-Hermes-2-SUS-Chat-34B-Linear)

|             Metric              |Value|
|---------------------------------|----:|
|Avg.                             |73.69|
|AI2 Reasoning Challenge (25-Shot)|66.38|
|HellaSwag (10-Shot)              |84.94|
|MMLU (5-Shot)                    |76.82|
|TruthfulQA (0-shot)              |59.19|
|Winogrande (5-shot)              |82.79|
|GSM8k (5-shot)                   |72.02|


{"license": "other", "tags": ["merge"], "license_name": "yi-34b", "license_link": "https://huggingface.co/01-ai/Yi-34B/blob/main/LICENSE", "model-index": [{"name": "Nous-Hermes-2-SUS-Chat-34B-Linear", "results": [{"task": {"type": "text-generation", "name": "Text Generation"}, "dataset": {"name": "AI2 Reasoning Challenge (25-Shot)", "type": "ai2_arc", "config": "ARC-Challenge", "split": "test", "args": {"num_few_shot": 25}}, "metrics": [{"type": "acc_norm", "value": 66.38, "name": "normalized accuracy"}], "source": {"url": "https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Nous-Hermes-2-SUS-Chat-34B-Linear", "name": "Open LLM Leaderboard"}}, {"task": {"type": "text-generation", "name": "Text Generation"}, "dataset": {"name": "HellaSwag (10-Shot)", "type": "hellaswag", "split": "validation", "args": {"num_few_shot": 10}}, "metrics": [{"type": "acc_norm", "value": 84.94, "name": "normalized accuracy"}], "source": {"url": "https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Nous-Hermes-2-SUS-Chat-34B-Linear", "name": "Open LLM Leaderboard"}}, {"task": {"type": "text-generation", "name": "Text Generation"}, "dataset": {"name": "MMLU (5-Shot)", "type": "cais/mmlu", "config": "all", "split": "test", "args": {"num_few_shot": 5}}, "metrics": [{"type": "acc", "value": 76.82, "name": "accuracy"}], "source": {"url": "https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Nous-Hermes-2-SUS-Chat-34B-Linear", "name": "Open LLM Leaderboard"}}, {"task": {"type": "text-generation", "name": "Text Generation"}, "dataset": {"name": "TruthfulQA (0-shot)", "type": "truthful_qa", "config": "multiple_choice", "split": "validation", "args": {"num_few_shot": 0}}, "metrics": [{"type": "mc2", "value": 59.19}], "source": {"url": "https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Nous-Hermes-2-SUS-Chat-34B-Linear", "name": "Open LLM Leaderboard"}}, {"task": {"type": "text-generation", "name": "Text Generation"}, "dataset": {"name": "Winogrande (5-shot)", "type": "winogrande", "config": "winogrande_xl", "split": "validation", "args": {"num_few_shot": 5}}, "metrics": [{"type": "acc", "value": 82.79, "name": "accuracy"}], "source": {"url": "https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Nous-Hermes-2-SUS-Chat-34B-Linear", "name": "Open LLM Leaderboard"}}, {"task": {"type": "text-generation", "name": "Text Generation"}, "dataset": {"name": "GSM8k (5-shot)", "type": "gsm8k", "config": "main", "split": "test", "args": {"num_few_shot": 5}}, "metrics": [{"type": "acc", "value": 72.02, "name": "accuracy"}], "source": {"url": "https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Nous-Hermes-2-SUS-Chat-34B-Linear", "name": "Open LLM Leaderboard"}}]}]}
transformers safetensors llama text-generation merge license:other model-index autotrain_compatible endpoints_compatible text-generation-inference region:us