prithivMLmods commited on
Commit
4fb1502
·
verified ·
1 Parent(s): a65fd43

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +39 -0
README.md CHANGED
@@ -25,3 +25,42 @@ With its robust natural language processing capabilities, **Deepthink-Reasoning-
25
  - **Long-context Support** up to 128K tokens and can generate up to 8K tokens.
26
  - **Multilingual support** for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.
27
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
  - **Long-context Support** up to 128K tokens and can generate up to 8K tokens.
26
  - **Multilingual support** for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.
27
 
28
+ # **Demo Start**
29
+
30
+ Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
31
+
32
+ ```python
33
+ from transformers import AutoModelForCausalLM, AutoTokenizer
34
+
35
+ model_name = "prithivMLmods/Deepthink-Reasoning-7B"
36
+
37
+ model = AutoModelForCausalLM.from_pretrained(
38
+ model_name,
39
+ torch_dtype="auto",
40
+ device_map="auto"
41
+ )
42
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
43
+
44
+ prompt = "Give me a short introduction to large language model."
45
+ messages = [
46
+ {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
47
+ {"role": "user", "content": prompt}
48
+ ]
49
+ text = tokenizer.apply_chat_template(
50
+ messages,
51
+ tokenize=False,
52
+ add_generation_prompt=True
53
+ )
54
+ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
55
+
56
+ generated_ids = model.generate(
57
+ **model_inputs,
58
+ max_new_tokens=512
59
+ )
60
+ generated_ids = [
61
+ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
62
+ ]
63
+
64
+ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
65
+ ```
66
+