- set offload_param.device to "none" or completely remove the offload_param section if you don't
- want CPU offload

- if using offload_param you can manually finetune stage3_param_persistence_threshold to control
- which params should remain on gpus - the larger the value the smaller the offload size