Different Trainers, when to use which?
There’s a few*Trainerobjects available fromtransformers,trlandsetfit.Other than the standard answer of “it depends on the task and which library you want to use”,what is the best practice or general guidelines when choosing which*Trainerobject to use to train/tune our models?Together with the*Trainerobject, sometimes we see suggestions to use*TrainingArgumentsor the vanillaTrainingArguments.For reference, we have:fine-tuning llama,https://github.com/philschmid/huggingface-llama-2-samples/blob/master/training/scripts/run_clm.pysuggests vanillaTrainerfew-shot learning,SetFit: Efficient Few-Shot Learning Without Promptssuggestsfrom setfit import SetFitTrainermachine translationhttps://github.com/huggingface/transformers/blob/main/examples/pytorch/translation/run_translation.pysuggestsSeq2SeqTrainerreinforcement learningSupervised Fine-tuning Trainersuggestsfrom trl import SFTTrainer
2024-10-12T14:39:56Z
@alvationsis there any update on this?@philschmidAre there any resources you could recommend? Or people we could tag?