Create README.md

README.md

@@ -0,0 +1,231 @@
+---
+license: other
+license_name: qwen
+language:
+- th
+- en
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- openthaigpt
+- qwen
+---
+
+# 🇹🇭 OpenThaiGPT 72b 1.5 Instruct
+![OpenThaiGPT](https://1173516064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvvbWvIIe82Iv1yHaDBC5%2Fuploads%2Fb8eiMDaqiEQL6ahbAY0h%2Fimage.png?alt=media&token=6fce78fd-2cca-4c0a-9648-bd5518e644ce)  
+[More Info](https://openthaigpt.aieat.or.th/)
+
+🇹🇭 **OpenThaiGPT 72b Version 1.5** is an advanced 72-billion-parameter Thai language chat model based on Qwen v2.5 released on September 30, 2024. It has been specifically fine-tuned on over 2,000,000 Thai instruction pairs and is capable of answering Thai-specific domain questions.
+
+## Highlights
+- **State-of-the-art Thai language LLM**, achieving the highest average scores across various Thai language exams compared to other open-source Thai LLMs.
+- **Multi-turn conversation support** for extended dialogues.
+- **Retrieval Augmented Generation (RAG) compatibility** for enhanced response generation.
+- **Impressive context handling**: Processes up to 131,072 tokens of input and generates up to 8,192 tokens, enabling detailed and complex interactions.
+
+## Benchmark on [OpenThaiGPT Eval](https://huggingface.co/datasets/openthaigpt/openthaigpt_eval)
+** Please take a look at ``openthaigpt/openthaigpt1.5-72b-instruct`` for this model's evaluation result.
+| **Exam names**                 | **scb10x/llama-3-typhoon-v1.5x-70b-instruct** | **meta-llama/Llama-3.1-70B-Instruct** | **Qwen/Qwen2.5-72B-Instruct** | **openthaigpt/openthaigpt1.5-72b-instruct** |
+|:------------------------------:|:---------------------------------------------:|:-------------------------------------:|:-----------------------------:|:----------------------------------:|
+| **01_a_level**                 | 59.17%                                        | 61.67%                                | 75.00%                        | 76.67%                             |
+| **02_tgat**                    | 46.00%                                        | 40.00%                                | 48.00%                        | 46.00%                             |
+| **03_tpat1**                   | 52.50%                                        | 50.00%                                | 55.00%                        | 55.00%                             |
+| **04_investment_consult**      | 60.00%                                        | 52.00%                                | 80.00%                        | 72.00%                             |
+| **05_facebook_beleble_th_200** | 87.50%                                        | 88.00%                                | 90.00%                        | 90.00%                             |
+| **06_xcopa_th_200**            | 84.50%                                        | 85.50%                                | 90.00%                        | 90.50%                             |
+| **07_xnli2.0_th_200**          | 62.50%                                        | 63.00%                                | 65.50%                        | 70.50%                             |
+| **08_onet_m3_thai**            | 76.00%                                        | 56.00%                                | 76.00%                        | 84.00%                             |
+| **09_onet_m3_social**          | 95.00%                                        | 95.00%                                | 90.00%                        | 95.00%                             |
+| **10_onet_m3_math**            | 43.75%                                        | 25.00%                                | 37.50%                        | 37.50%                             |
+| **11_onet_m3_science**         | 53.85%                                        | 61.54%                                | 65.38%                        | 73.08%                             |
+| **12_onet_m3_english**         | 93.33%                                        | 93.33%                                | 96.67%                        | 96.67%                             |
+| **13_onet_m6_thai**            | 55.38%                                        | 60.00%                                | 60.00%                        | 56.92%                             |
+| **14_onet_m6_math**            | 41.18%                                        | 58.82%                                | 23.53%                        | 41.18%                             |
+| **15_onet_m6_social**          | 67.27%                                        | 76.36%                                | 63.64%                        | 65.45%                             |
+| **16_onet_m6_science**         | 50.00%                                        | 57.14%                                | 64.29%                        | 67.86%                             |
+| **17_onet_m6_english**         | 73.08%                                        | 82.69%                                | 86.54%                        | 90.38%                             |
+| **Micro Average**              | 69.97%                                        | 71.09%                                | 75.02%                        | <b style="color:blue">76.73%</b>                           |
+
+
+Thai language multiple choice exams, Test on unseen test set, Zero-shot learning. Benchmark source code and exams information: https://github.com/OpenThaiGPT/openthaigpt_eval
+
+(Updated on: 30 September 2024)
+
+## Benchmark on [scb10x/thai_exam](https://huggingface.co/datasets/scb10x/thai_exam)
+
+| Models                                                    | **Thai Exam (Acc)** |
+|:----------------------------------------------------------:|:-------------------:|
+| **api/claude-3-5-sonnet-20240620**                         | 69.2                |
+| <b style="color:blue">**openthaigpt/openthaigpt1.5-72b-instruct***</b>                        | <b style="color:blue">64.07</b>               |
+| **api/gpt-4o-2024-05-13**                                  | 63.89               |
+| **hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4**   | 63.54               |
+| **Qwen/Qwen2-72B-Instruct**                                | 58.23               |
+| **meta-llama/Meta-Llama-3.1-70B-Instruct**                 | 58.23               |
+| **scb10x/llama-3-typhoon-v1.5x-70b-instruct**              | 58.76               |
+| **Qwen/Qwen2.5-14B-Instruct**                              | 57.35               |
+| **api/gpt-4o-mini-2024-07-18**                             | 54.51               |
+| <b style="color:blue">**openthaigpt/openthaigpt1.5-7b-instruct***</b>                         | <b style="color:blue">52.04</b>               |
+| **SeaLLMs/SeaLLMs-v3-7B-Chat**                             | 51.33               |
+| **openthaigpt/openthaigpt-1.0.0-70b-chat**                 | 50.09               |
+
+<b style="color:blue">*</b>  Evaluated by OpenThaiGPT team using [scb10x/thai_exam](https://huggingface.co/datasets/scb10x/thai_exam).
+## Licenses
+* Built with Qwen
+* Qwen License: Allow **Research** and **Commercial uses** but if your user base exceeds 100 million monthly active users, you need to negotiate a separate commercial license. Please see LICENSE file for more information.<br>
+
+## Sponsors
+<img src="https://cdn-uploads.huggingface.co/production/uploads/5fcd9c426d942eaf4d1ebd30/3kjN6kuTzXDXQ6o1RFvHX.png" width="600px">
+
+## Supports
+- Official website: https://openthaigpt.aieat.or.th
+- Facebook page: https://web.facebook.com/groups/openthaigpt
+- A Discord server for discussion and support [here](https://discord.gg/rUTp6dfVUF)
+- E-mail: [email protected]
+
+## Prompt Format
+Prompt format is based on ChatML.
+```
+<|im_start|>system\n{sytem_prompt}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n
+```
+
+### System prompt:
+```
+คุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์
+```
+
+### Examples
+
+#### Single Turn Conversation Example
+```
+<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nสวัสดีครับ<|im_end|>\n<|im_start|>assistant\n
+```
+
+#### Single Turn Conversation with Context (RAG) Example
+```
+<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nกรุงเทพมหานคร เป็นเมืองหลวง นครและมหานครที่มีประชากรมากที่สุดของประเทศไทย กรุงเทพมหานครมีพื้นที่ทั้งหมด 1,568.737 ตร.กม. มีประชากรตามทะเบียนราษฎรกว่า 8 ล้านคน\nกรุงเทพมหานครมีพื้นที่เท่าไร่<|im_end|>\n<|im_start|>assistant\n
+```
+
+#### Multi Turn Conversation Example
+
+##### First turn
+```
+<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nสวัสดีครับ<|im_end|>\n<|im_start|>assistant\n
+```
+
+##### Second turn
+```
+<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nสวัสดีครับ<|im_end|>\n<|im_start|>assistant\nสวัสดีครับ ยินดีต้อนรับครับ คุณต้องการให้ฉันช่วยอะไรครับ?<|im_end|>\n<|im_start|>user\nกรุงเทพมหานคร ชื่อเต็มยาวๆคืออะไร<|im_end|>\n<|im_start|>assistant\n
+```
+
+##### Result
+```
+<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉล���ดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nสวัสดีครับ<|im_end|>\n<|im_start|>assistant\nสวัสดีครับ ยินดีต้อนรับครับ คุณต้องการให้ฉันช่วยอะไรครับ?<|im_end|>\n<|im_start|>user\nกรุงเทพมหานคร ชื่อเต็มยาวๆคืออะไร<|im_end|>\n<|im_start|>assistant\nชื่อเต็มของกรุงเทพมหานครคือ \"กรุงเทพมหานคร อมรรัตนโกสินทร์ มหินทรายุธยา มหาดิลกภพ นพรัตนราชธานีบูรีรมย์ อุดมราชนิเวศน์มหาสถาน อมรพิมานอวตารสถิต สักกะทัตติยวิษณุกรรมประสิทธิ์\"
+```
+
+## How to use
+
+### Huggingface
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "openthaigpt/openthaigpt1.5-72b-instruct"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+prompt = "ประเทศไทยคืออะไร"
+messages = [
+    {"role": "system", "content": "คุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์"},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=512
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+```
+
+### vLLM
+
+1. Install VLLM (https://github.com/vllm-project/vllm)
+   
+2. Run server
+```bash
+vllm serve openthaigpt/openthaigpt1.5-72b-instruct --tensor-parallel-size 4
+```
+* Note, change ``--tensor-parallel-size 4`` to the amount of available GPU cards.
+  
+3. Run inference (CURL example)
+```bash
+curl -X POST 'http://127.0.0.1:8000/v1/completions' \
+-H 'Content-Type: application/json' \
+-d '{
+  "model": ".",
+  "prompt": "<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nสวัสดีครับ<|im_end|>\n<|im_start|>assistant\n",
+  "max_tokens": 512,
+  "temperature": 0.7,
+  "top_p": 0.8,
+  "top_k": 40,
+  "stop": ["<|im_end|>"]
+}'
+```
+
+### Processing Long Texts
+
+The current `config.json` is set for context length up to 32,768 tokens.
+To handle extensive inputs exceeding 32,768 tokens, we utilize [YaRN](https://arxiv.org/abs/2309.00071), a technique for enhancing model length extrapolation, ensuring optimal performance on lengthy texts.
+
+For supported frameworks, you could add the following to `config.json` to enable YaRN:
+```json
+{
+  ...,
+  "rope_scaling": {
+    "factor": 4.0,
+    "original_max_position_embeddings": 32768,
+    "type": "yarn"
+  }
+}
+```
+
+### GPU Memory Requirements
+| **Number of Parameters** | **FP 16 bits** | **8 bits (Quantized)** | **4 bits (Quantized)** | **Example Graphic Card for 4 bits** |
+|------------------|----------------|------------------------|------------------------|---------------------------------------------|
+| **7b**           | 24 GB          | 12 GB                  | 6 GB                   | Nvidia RTX 4060 8GB                         |
+| **13b**          | 48 GB          | 24 GB                  | 12 GB                  | Nvidia RTX 4070 16GB                        |
+| **72b**          | 192 GB         | 96 GB                  | 48 GB                  | Nvidia RTX 4090 24GB x 2 cards              |
+
+### Authors
+* Sumeth Yuenyong ([email protected])
+* Kobkrit Viriyayudhakorn ([email protected])
+* Apivadee Piyatumrong ([email protected])
+* Jillaphat Jaroenkantasima ([email protected])
+* Thaweewat Rugsujarit ([email protected])
+* Norapat Buppodom ([email protected])
+* Koravich Sangkaew ([email protected])
+* Peerawat Rojratchadakorn ([email protected])
+* Surapon Nonesung ([email protected])
+* Chanon Utupon ([email protected])
+* Sadhis Wongprayoon ([email protected])
+* Nucharee Thongthungwong ([email protected])
+* Chawakorn Phiantham ([email protected])
+* Patteera Triamamornwooth ([email protected])
+* Nattarika Juntarapaoraya ([email protected])
+* Kriangkrai Saetan ([email protected])
+* Pitikorn Khlaisamniang ([email protected])
+
+<i>Disclaimer: Provided responses are not guaranteed.</i>