Hugging Face's logo

RichardErkhov
/
allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf

GGUF
Inference Endpoints
conversational
Model card Files Community
allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf
File size: 10,609 Bytes 
21ca407
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
 
Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


Llama-3-Alpha-Ko-8B-Instruct - GGUF
- Model creator: https://huggingface.co/allganize/
- Original model: https://huggingface.co/allganize/Llama-3-Alpha-Ko-8B-Instruct/


| Name | Quant method | Size |
| ---- | ---- | ---- |
| [Llama-3-Alpha-Ko-8B-Instruct.Q2_K.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q2_K.gguf) | Q2_K | 2.96GB |
| [Llama-3-Alpha-Ko-8B-Instruct.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.IQ3_XS.gguf) | IQ3_XS | 3.28GB |
| [Llama-3-Alpha-Ko-8B-Instruct.IQ3_S.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.IQ3_S.gguf) | IQ3_S | 3.43GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q3_K_S.gguf) | Q3_K_S | 3.41GB |
| [Llama-3-Alpha-Ko-8B-Instruct.IQ3_M.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.IQ3_M.gguf) | IQ3_M | 3.52GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q3_K.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q3_K.gguf) | Q3_K | 3.74GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q3_K_M.gguf) | Q3_K_M | 3.74GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q3_K_L.gguf) | Q3_K_L | 4.03GB |
| [Llama-3-Alpha-Ko-8B-Instruct.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.IQ4_XS.gguf) | IQ4_XS | 4.18GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q4_0.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q4_0.gguf) | Q4_0 | 4.34GB |
| [Llama-3-Alpha-Ko-8B-Instruct.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.IQ4_NL.gguf) | IQ4_NL | 4.38GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q4_K_S.gguf) | Q4_K_S | 4.37GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q4_K.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q4_K.gguf) | Q4_K | 4.58GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q4_K_M.gguf) | Q4_K_M | 4.58GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q4_1.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q4_1.gguf) | Q4_1 | 4.78GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q5_0.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q5_0.gguf) | Q5_0 | 5.21GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q5_K_S.gguf) | Q5_K_S | 5.21GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q5_K.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q5_K.gguf) | Q5_K | 5.34GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q5_K_M.gguf) | Q5_K_M | 5.34GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q5_1.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q5_1.gguf) | Q5_1 | 5.65GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q6_K.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q6_K.gguf) | Q6_K | 6.14GB |
| [Llama-3-Alpha-Ko-8B-Instruct.Q8_0.gguf](https://huggingface.co/RichardErkhov/allganize_-_Llama-3-Alpha-Ko-8B-Instruct-gguf/blob/main/Llama-3-Alpha-Ko-8B-Instruct.Q8_0.gguf) | Q8_0 | 7.95GB |




Original model description:
---
license: other
license_name: llama3
language:
- ko
---
![Alpha-Instruct](./alpha-instruct.png)
We are thrilled to introduce **Alpha-Instruct**, our latest language model, which demonstrates exceptional capabilities in both Korean and English. Alpha-Instruct is developed using the **Evolutionary Model Merging** technique, enabling it to excel in complex language tasks and logical reasoning.

A key aspect of Alpha-Instruct's development is our **community-based approach**. We draw inspiration and ideas from various communities, shaping our datasets, methodologies, and the model itself. In return, we are committed to sharing our insights with the community, providing detailed information on the data, methods, and models used in Alpha-Instruct's creation.

Alpha-Instruct has achieved outstanding performance on the **LogicKor, scoring an impressive 6.62**. Remarkably, this performance rivals that of 70B models, showcasing the efficiency and power of our 8B model. This achievement highlights Alpha-Instruct's advanced computational and reasoning skills, making it a leading choice for diverse and demanding language tasks.

**For more information and technical details about Alpha-Instruct, stay tuned to our updates and visit our [website](https://allganize-alpha.github.io/) (Soon).**

---
## Overview
Alpha-Instruct is our latest language model, developed using 'Evolutionary Model Merging' technique. This method employs a 1:1 ratio of task-specific datasets from KoBEST and Haerae, resulting in a model with named 'Alpha-Ko-8B-Evo'. The following models were used for merging:
- [Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) (Base)
- [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) (Instruct)
- [Llama-3-Open-Ko-8B](beomi/Llama-3-Open-Ko-8B) (Continual Pretrained)

To refine and enhance Alpha-Instruct, we utilized a carefully curated high-quality datasets aimed at 'healing' the model's output, significantly boosting its human preference scores. We use [ORPO](https://arxiv.org/abs/2403.07691) specifically for this "healing" (RLHF) phase. The datasets* used include:
- [Korean-Human-Judgements](https://huggingface.co/datasets/HAERAE-HUB/Korean-Human-Judgements)
- [Orca-Math](https://huggingface.co/datasets/kuotient/orca-math-word-problems-193k-korean)
- [dpo-mix-7k](https://huggingface.co/datasets/argilla/dpo-mix-7k)

*Some of these datasets were partially used and translated for training, and we ensured there was no contamination during the evaluation process.

This approach effectively balances human preferences with the model's capabilities, making Alpha-Instruct well-suited for real-life scenarios where user satisfaction and performance are equally important.

## Benchmark Results
Results in [LogicKor](https://github.com/StableFluffy/LogicKor)* are as follows:

|         Model                  | Single turn* | Multi turn* | Overall* |
|:------------------------------:|:------------:|:-----------:|:--------:|
| MLP-KTLim/llama-3-Korean-Bllossom-8B | 4.238 | 3.404 | 3.821 |
| Alpha-Ko-Evo | 5.143 | 5.238 | 5.190 |
| Alpha-Ko-Instruct (alt)  |     7.095    |    **6.571**    |   **6.833**  |
| Alpha-Ko-Instruct |     **7.143**    |    6.065    |   6.620  |
| Alpha-Ko-Instruct-marlin (4bit) | 6.857 | 5.738 | 6.298 |

*Self report(Default settings with 'alpha' template, mean of 3).

Result in KoBEST(acc, num_shot=5) are as follows:

| Task  |	beomi/Llama-3-Open-Ko-8B-Instruct | maywell/Llama-3-Ko-8B-Instruct | **Alpha-Ko-Evo** | **Alpha-Ko-Instruct** |
| --- | --- | --- | --- | --- |
| kobest overall |	0.6220 | 0.6852 |0.7229|0.7055
| kobest_boolq|	0.6254 |	0.7208 | 0.8547 | 0.8369
| kobest_copa|	0.7110 |	0.7650 | 0.7420 | 0.7420
| kobest_hellaswag|	0.3840 |	0.4440 | 0.4220 | 0.4240
| kobest_sentineg|	0.8388 |	0.9194 |0.9471 | 0.9244
| kobest_wic|	0.5738|	0.6040 |0.6095 | 0.5730

*For reference, 'merged' models are chosen.

## How to use

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "allganize/Llama-3-Alpha-Ko-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "system", "content": "๋‹น์‹ ์€ ์ธ๊ณต์ง€๋Šฅ ์–ด์‹œ์Šคํ„ดํŠธ์ž…๋‹ˆ๋‹ค. ๋ฌป๋Š” ๋ง์— ์นœ์ ˆํ•˜๊ณ  ์ •ํ™•ํ•˜๊ฒŒ ๋‹ต๋ณ€ํ•˜์„ธ์š”."},
    {"role": "user", "content": "ํ”ผ๋ณด๋‚˜์น˜ ์ˆ˜์—ด์ด ๋ญ์•ผ? ๊ทธ๋ฆฌ๊ณ  ํ”ผ๋ณด๋‚˜์น˜ ์ˆ˜์—ด์— ๋Œ€ํ•ด ํŒŒ์ด์ฌ ์ฝ”๋“œ๋ฅผ ์งœ์ค˜๋ณผ๋ž˜?"},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

terminators = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = model.generate(
    input_ids,
    max_new_tokens=512,
    eos_token_id=terminators,
    do_sample=False,
    repetition_penalty=1.05,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
```

## Correspondence to
- Ji soo Kim ([email protected])
- Contributors
  - Sangmin Jeon ([email protected])
  - Seungwoo Ryu ([email protected])

## Special Thanks
- [@beomi](https://huggingface.co/beomi) for providing us with a great model!

## License
The use of this model is governed by the [META LLAMA 3 COMMUNITY LICENSE AGREEMENT](https://llama.meta.com/llama3/license/)


## Citation
If you use this model in your research, please cite it as follows:

```bibtex
@misc{alpha-instruct,
  author       = {Ji soo Kim},
  title        = {Alpha-Instruct: Allganize Bilingual Model},
  year         = {2024},
  publisher = {Hugging Face},
  journal = {Hugging Face repository},
  url          = {https://huggingface.co/allganize/Llama-3-Alpha-Ko-8B-Instruct},
}