| license: apache-2.0 | |
| datasets: | |
| - opencsg/smoltalk-chinese | |
| language: | |
| - zh | |
| base_model: | |
| - opencsg/csg-wukong-ablation-chinese-fineweb-edu | |
| * Using ``opencsg/csg-wukong-2b-chinese-fineweb-edu`` as base model, we fine-tune it on ``smoltalk-chinese`` for 2 epoch | |
| * learning rate = 3e-4 ; global batch size = 32 ; lr scheduler=cosine |