Update README.md
Browse files
README.md
CHANGED
@@ -7,4 +7,5 @@ language:
|
|
7 |
base_model:
|
8 |
- opencsg/csg-wukong-ablation-chinese-fineweb-edu
|
9 |
---
|
10 |
-
Using ``opencsg/csg-wukong-
|
|
|
|
7 |
base_model:
|
8 |
- opencsg/csg-wukong-ablation-chinese-fineweb-edu
|
9 |
---
|
10 |
+
* Using ``opencsg/csg-wukong-2b-chinese-fineweb-edu`` as base model, we fine-tune it on ``smoltalk-chinese`` for 2 epoch
|
11 |
+
* learning rate = 3e-4 ; global batch size = 32 ; lr scheduler=cosine
|