NEFTune
NEFTune is a technique that can improve performance by adding noise to the embedding vectors during training. To enable it in [Trainer], set the neftune_noise_alpha parameter in [TrainingArguments] to control how much noise is added.

from transformers import TrainingArguments, Trainer
training_args = TrainingArguments(, neftune_noise_alpha=0.1)
trainer = Trainer(, args=training_args)