README.md

---
license: other
license_name: qwen
language:
- th
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- openthaigpt
- qwen
---

# 🇹🇭 OpenThaiGPT 72b 1.5 Instruct
![OpenThaiGPT](https://1173516064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvvbWvIIe82Iv1yHaDBC5%2Fuploads%2Fb8eiMDaqiEQL6ahbAY0h%2Fimage.png?alt=media&token=6fce78fd-2cca-4c0a-9648-bd5518e644ce)  
[More Info](https://openthaigpt.aieat.or.th/)

🇹🇭 **OpenThaiGPT 72b Version 1.5** is an advanced 72-billion-parameter Thai language chat model based on Qwen v2.5 released on September 30, 2024. It has been specifically fine-tuned on over 2,000,000 Thai instruction pairs and is capable of answering Thai-specific domain questions.

## Highlights
- **State-of-the-art Thai language LLM**, achieving the highest average scores across various Thai language exams compared to other open-source Thai LLMs.
- **Multi-turn conversation support** for extended dialogues.
- **Retrieval Augmented Generation (RAG) compatibility** for enhanced response generation.
- **Impressive context handling**: Processes up to 131,072 tokens of input and generates up to 8,192 tokens, enabling detailed and complex interactions.

## Benchmark on [OpenThaiGPT Eval](https://huggingface.co/datasets/openthaigpt/openthaigpt_eval)
** Please take a look at ``openthaigpt/openthaigpt1.5-72b-instruct`` for this model's evaluation result.
| **Exam names**                 | **scb10x/llama-3-typhoon-v1.5x-70b-instruct** | **meta-llama/Llama-3.1-70B-Instruct** | **Qwen/Qwen2.5-72B-Instruct** | **openthaigpt/openthaigpt1.5-72b-instruct** |
|:------------------------------:|:---------------------------------------------:|:-------------------------------------:|:-----------------------------:|:----------------------------------:|
| **01_a_level**                 | 59.17%                                        | 61.67%                                | 75.00%                        | 76.67%                             |
| **02_tgat**                    | 46.00%                                        | 40.00%                                | 48.00%                        | 46.00%                             |
| **03_tpat1**                   | 52.50%                                        | 50.00%                                | 55.00%                        | 55.00%                             |
| **04_investment_consult**      | 60.00%                                        | 52.00%                                | 80.00%                        | 72.00%                             |
| **05_facebook_beleble_th_200** | 87.50%                                        | 88.00%                                | 90.00%                        | 90.00%                             |
| **06_xcopa_th_200**            | 84.50%                                        | 85.50%                                | 90.00%                        | 90.50%                             |
| **07_xnli2.0_th_200**          | 62.50%                                        | 63.00%                                | 65.50%                        | 70.50%                             |
| **08_onet_m3_thai**            | 76.00%                                        | 56.00%                                | 76.00%                        | 84.00%                             |
| **09_onet_m3_social**          | 95.00%                                        | 95.00%                                | 90.00%                        | 95.00%                             |
| **10_onet_m3_math**            | 43.75%                                        | 25.00%                                | 37.50%                        | 37.50%                             |
| **11_onet_m3_science**         | 53.85%                                        | 61.54%                                | 65.38%                        | 73.08%                             |
| **12_onet_m3_english**         | 93.33%                                        | 93.33%                                | 96.67%                        | 96.67%                             |
| **13_onet_m6_thai**            | 55.38%                                        | 60.00%                                | 60.00%                        | 56.92%                             |
| **14_onet_m6_math**            | 41.18%                                        | 58.82%                                | 23.53%                        | 41.18%                             |
| **15_onet_m6_social**          | 67.27%                                        | 76.36%                                | 63.64%                        | 65.45%                             |
| **16_onet_m6_science**         | 50.00%                                        | 57.14%                                | 64.29%                        | 67.86%                             |
| **17_onet_m6_english**         | 73.08%                                        | 82.69%                                | 86.54%                        | 90.38%                             |
| **Micro Average**              | 69.97%                                        | 71.09%                                | 75.02%                        | <b style="color:blue">76.73%</b>                           |


Thai language multiple choice exams, Test on unseen test set, Zero-shot learning. Benchmark source code and exams information: https://github.com/OpenThaiGPT/openthaigpt_eval

(Updated on: 30 September 2024)

## Benchmark on [scb10x/thai_exam](https://huggingface.co/datasets/scb10x/thai_exam)

| Models                                                    | **Thai Exam (Acc)** |
|:----------------------------------------------------------:|:-------------------:|
| **api/claude-3-5-sonnet-20240620**                         | 69.2                |
| <b style="color:blue">**openthaigpt/openthaigpt1.5-72b-instruct***</b>                        | <b style="color:blue">64.07</b>               |
| **api/gpt-4o-2024-05-13**                                  | 63.89               |
| **hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4**   | 63.54               |
| **Qwen/Qwen2-72B-Instruct**                                | 58.23               |
| **meta-llama/Meta-Llama-3.1-70B-Instruct**                 | 58.23               |
| **scb10x/llama-3-typhoon-v1.5x-70b-instruct**              | 58.76               |
| **Qwen/Qwen2.5-14B-Instruct**                              | 57.35               |
| **api/gpt-4o-mini-2024-07-18**                             | 54.51               |
| <b style="color:blue">**openthaigpt/openthaigpt1.5-7b-instruct***</b>                         | <b style="color:blue">52.04</b>               |
| **SeaLLMs/SeaLLMs-v3-7B-Chat**                             | 51.33               |
| **openthaigpt/openthaigpt-1.0.0-70b-chat**                 | 50.09               |

<b style="color:blue">*</b>  Evaluated by OpenThaiGPT team using [scb10x/thai_exam](https://huggingface.co/datasets/scb10x/thai_exam).
## Licenses
* Built with Qwen
* Qwen License: Allow **Research** and **Commercial uses** but if your user base exceeds 100 million monthly active users, you need to negotiate a separate commercial license. Please see LICENSE file for more information.<br>

## Sponsors
<img src="/static-proxy?url=https%3A%2F%2Fcdn-uploads.huggingface.co%2Fproduction%2Fuploads%2F5fcd9c426d942eaf4d1ebd30%2F3kjN6kuTzXDXQ6o1RFvHX.png%26quot%3B%3C%2Fspan%3E width="600px">

## Supports
- Official website: https://openthaigpt.aieat.or.th
- Facebook page: https://web.facebook.com/groups/openthaigpt
- A Discord server for discussion and support [here](https://discord.gg/rUTp6dfVUF)
- E-mail: [email protected]

## Prompt Format
Prompt format is based on ChatML.
```
<|im_start|>system\n{sytem_prompt}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n
```

### System prompt:
```
คุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์
```

### Examples

#### Single Turn Conversation Example
```
<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nสวัสดีครับ<|im_end|>\n<|im_start|>assistant\n
```

#### Single Turn Conversation with Context (RAG) Example
```
<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nกรุงเทพมหานคร เป็นเมืองหลวง นครและมหานครที่มีประชากรมากที่สุดของประเทศไทย กรุงเทพมหานครมีพื้นที่ทั้งหมด 1,568.737 ตร.กม. มีประชากรตามทะเบียนราษฎรกว่า 8 ล้านคน\nกรุงเทพมหานครมีพื้นที่เท่าไร่<|im_end|>\n<|im_start|>assistant\n
```

#### Multi Turn Conversation Example

##### First turn
```
<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nสวัสดีครับ<|im_end|>\n<|im_start|>assistant\n
```

##### Second turn
```
<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nสวัสดีครับ<|im_end|>\n<|im_start|>assistant\nสวัสดีครับ ยินดีต้อนรับครับ คุณต้องการให้ฉันช่วยอะไรครับ?<|im_end|>\n<|im_start|>user\nกรุงเทพมหานคร ชื่อเต็มยาวๆคืออะไร<|im_end|>\n<|im_start|>assistant\n
```

##### Result
```
<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nสวัสดีครับ<|im_end|>\n<|im_start|>assistant\nสวัสดีครับ ยินดีต้อนรับครับ คุณต้องการให้ฉันช่วยอะไรครับ?<|im_end|>\n<|im_start|>user\nกรุงเทพมหานคร ชื่อเต็มยาวๆคืออะไร<|im_end|>\n<|im_start|>assistant\nชื่อเต็มของกรุงเทพมหานครคือ \"กรุงเทพมหานคร อมรรัตนโกสินทร์ มหินทรายุธยา มหาดิลกภพ นพรัตนราชธานีบูรีรมย์ อุดมราชนิเวศน์มหาสถาน อมรพิมานอวตารสถิต สักกะทัตติยวิษณุกรรมประสิทธิ์\"
```

## How to use

### Huggingface
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "openthaigpt/openthaigpt1.5-72b-instruct"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "ประเทศไทยคืออะไร"
messages = [
    {"role": "system", "content": "คุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์"},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
```

### vLLM

1. Install VLLM (https://github.com/vllm-project/vllm)
   
2. Run server
```bash
vllm serve openthaigpt/openthaigpt1.5-72b-instruct --tensor-parallel-size 4
```
* Note, change ``--tensor-parallel-size 4`` to the amount of available GPU cards.
  
3. Run inference (CURL example)
```bash
curl -X POST 'http://127.0.0.1:8000/v1/completions' \
-H 'Content-Type: application/json' \
-d '{
  "model": ".",
  "prompt": "<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nสวัสดีครับ<|im_end|>\n<|im_start|>assistant\n",
  "max_tokens": 512,
  "temperature": 0.7,
  "top_p": 0.8,
  "top_k": 40,
  "stop": ["<|im_end|>"]
}'
```

### Processing Long Texts

The current `config.json` is set for context length up to 32,768 tokens.
To handle extensive inputs exceeding 32,768 tokens, we utilize [YaRN](https://arxiv.org/abs/2309.00071), a technique for enhancing model length extrapolation, ensuring optimal performance on lengthy texts.

For supported frameworks, you could add the following to `config.json` to enable YaRN:
```json
{
  ...,
  "rope_scaling": {
    "factor": 4.0,
    "original_max_position_embeddings": 32768,
    "type": "yarn"
  }
}
```

### GPU Memory Requirements
| **Number of Parameters** | **FP 16 bits** | **8 bits (Quantized)** | **4 bits (Quantized)** | **Example Graphic Card for 4 bits** |
|------------------|----------------|------------------------|------------------------|---------------------------------------------|
| **7b**           | 24 GB          | 12 GB                  | 6 GB                   | Nvidia RTX 4060 8GB                         |
| **13b**          | 48 GB          | 24 GB                  | 12 GB                  | Nvidia RTX 4070 16GB                        |
| **72b**          | 192 GB         | 96 GB                  | 48 GB                  | Nvidia RTX 4090 24GB x 2 cards              |

### Authors
* Sumeth Yuenyong ([email protected])
* Kobkrit Viriyayudhakorn ([email protected])
* Apivadee Piyatumrong ([email protected])
* Jillaphat Jaroenkantasima ([email protected])
* Thaweewat Rugsujarit ([email protected])
* Norapat Buppodom ([email protected])
* Koravich Sangkaew ([email protected])
* Peerawat Rojratchadakorn ([email protected])
* Surapon Nonesung ([email protected])
* Chanon Utupon ([email protected])
* Sadhis Wongprayoon ([email protected])
* Nucharee Thongthungwong ([email protected])
* Chawakorn Phiantham ([email protected])
* Patteera Triamamornwooth ([email protected])
* Nattarika Juntarapaoraya ([email protected])
* Kriangkrai Saetan ([email protected])
* Pitikorn Khlaisamniang ([email protected])

<i>Disclaimer: Provided responses are not guaranteed.</i>