- enable bf16 if you use Ampere or higher GPU - this will run in mixed precision and will be
faster.

- for older GPUs you can enable fp16, but it'll only work for non-bf16 pretrained models - e.g.
all official t5 models are bf16-pretrained

- set offload_param.device to "none" or completely remove the offload_param section if you don't
- want CPU offload