DeepSpeed requires a distributed environment even when only one process is used.
This emulates a launcher in the notebook
import os
os.environ["MASTER_ADDR"] = "localhost"
os.environ["MASTER_PORT"] = "9994"  # modify if RuntimeError: Address already in use
os.environ["RANK"] = "0"
os.environ["LOCAL_RANK"] = "0"
os.environ["WORLD_SIZE"] = "1"
Now proceed as normal, plus pass the DeepSpeed config file
training_args = TrainingArguments(, deepspeed="ds_config_zero3.json")
trainer = Trainer()
trainer.train()