transformer-16M-test / configs /deepspeed.yaml
zaydzuhri's picture
Training in progress, step 2500
51f88fe verified
raw
history blame contribute delete
246 Bytes
compute_environment: LOCAL_MACHINE
distributed_type: DEEPSPEED
deepspeed_config:
deepspeed_config_file: configs/ds_config.json
zero3_init_flag: true
machine_rank: 0
main_training_function: main
num_machines: 1
num_processes: 1
use_cpu: false