README.md

metadata

license: apache-2.0
datasets:
  - opencsg/smoltalk-chinese
language:
  - zh
base_model:
  - opencsg/csg-wukong-ablation-chinese-fineweb-edu

• Using opencsg/csg-wukong-2b-chinese-fineweb-edu as base model, we fine-tune it on smoltalk-chinese for 2 epoch
• learning rate = 3e-4 ; global batch size = 32 ; lr scheduler=cosine