import numpy as np
from datasets import Dataset
seq_len, dataset_size = 512, 512
dummy_data = {
     "input_ids": np.random.randint(100, 30000, (dataset_size, seq_len)),
     "labels": np.random.randint(0, 1, (dataset_size)),
 }
ds = Dataset.from_dict(dummy_data)
ds.set_format("pt")

To print summary statistics for the GPU utilization and the training run with the [Trainer] we define two helper functions: