Training on 'free' Googe Colab
I am pretty sure this question has been answered before, but I could not find it.Basically, I am learning how to train a Bert classifier from scratch to classify a set of e-mails as ‘spam’ or ‘not spam’ on Google colab using T4 GPUMost of the code is pretty basic and standard, i.e., loading ‘sms_spam’ data set, tokening loading the model (“distilbert-base-uncased”), and Unfreezing all the model parameters.And here is the instance of the trainer class:import numpy as npfrom transformers import DataCollatorWithPadding, Trainer, TrainingArgumentsfrom accelerate import Acceleratoraccelerator = Accelerator()def compute_metrics(eval_pred):predictions, labels = eval_predpredictions = np.argmax(predictions, axis=1)return {“accuracy”: (predictions == labels).mean()}The HuggingFace Trainer class handles the training and eval loop for PyTorch for us.Read more about it hereTrainertrainer = Trainer(model=model,args=TrainingArguments(output_dir=“/content/drive/My Drive/DATA/spam_not_spam”,learning_rate=2e-3,per_device_train_batch_size=4,per_device_eval_batch_size=4,evaluation_strategy=“epoch”,save_strategy=“epoch”,num_train_epochs=2,weight_decay=0.01,load_best_model_at_end=True,),train_dataset=tokenized_dataset["train"],
eval_dataset=tokenized_dataset["test"],
tokenizer=tokenizer,
data_collator=DataCollatorWithPadding(tokenizer=tokenizer),
compute_metrics=compute_metrics,accelerator=accelerator,)trainer.train()Note: First time, I did not have ‘accelerate’. Colab threw an error and suggested I install accelerator and create an instance which I did with “!pip install accelerate -U”But even after that, I keep getting the following error:<<ImportError                               Traceback (most recent call last)in <cell line: 18>()18 trainer = Trainer(19     model=model,—> 20     args=TrainingArguments(21      #   output_dir=“./data/spam_not_spam”,22         output_dir=“/content/drive/My Drive/DATA/spam_not_spam”,4 frames/usr/local/lib/python3.10/dist-packages/transformers/training_args.py in _setup_devices(self)1785         if not is_sagemaker_mp_enabled():1786             if not is_accelerate_available(min_version=“0.20.1”):→ 1787                 raise ImportError(1788                     “Using theTrainerwithPyTorchrequiresaccelerate>=0.20.1: Please runpip install transformers[torch]orpip install accelerate -U”1789                 )ImportError: Using theTrainerwithPyTorchrequiresaccelerate>=0.20.1: Please runpip install transformers[torch]orpip install accelerate -UCan anyone offer any suggestions on what I am doing wrong and how to fix this? Cheers
2024-02-13T22:41:29Z
You need to restart your runtime as well
2024-02-14T03:11:14Z
@muellerzr, Can you elaborate please? At what point do I need to restart the runtime?
2024-02-14T18:13:45Z
After installing accelerate (Or just make it the first thing you do)
2024-03-07T21:34:16Z
Hi,https://huggingface.co/docs/accelerate/basic_tutorials/notebookhttps://huggingface.co/docs/accelerate/v0.27.2/en/package_reference/accelerator#accelerate.Accelerator.prepare