Huggingface / Pytorch versions on Sagemaker
When trying to use Huggingface estimator on sagemaker,Run training on Amazon SageMakere.g.# create the Estimator
huggingface_estimator = HuggingFace(
        entry_point='train.py',
        source_dir='./scripts',
        instance_type='ml.p3.2xlarge',
        instance_count=1,
        role=role,
        transformers_version='4.17',
        pytorch_version='1.10',
        py_version='py38',
        hyperparameters = hyperparameters
)When I tried to increase the version totransformers_version='4.24', it throws an error where the maximum version supported is 4.17.Is there a page that lists the version that Sagemaker supports?What are the possible versions for the following arguments?transformers_versionpytorch_versionpy_versionAlso, are CPU instances supported by the Huggingface estimator on sagemaker?
2022-11-18T15:25:03Z
Hi@alvations- thisthreadshould hopefully answer all your questions
2022-11-18T17:48:24Z
Thank you for the prompt reply!There’s a pointer in the thread torequirements.txtfor deployment when loading the model, is there some documentation to usingrequirements.txtwhen trying to use the huggingface estimator for training?
2022-11-18T18:24:12Z
Yup, that would be in thispost
2022-11-22T01:17:02Z
@marshmellow77, I’ve tried following the postGitHub - aws/sagemaker-huggingface-inference-toolkitand tried a few combinations.Is it right that the only way for me to use transformers version>=4.17with aTrainerobject, I’ll have tofirst load the model in pythonsave to disk in a directoryadd therequirements.txt, in the directoryload the model with therequirements.txtIn code, something like:import torch

from datasets import load_dataset
from transformers import EncoderDecoderModel
from transformers import AutoTokenizer
from transformers import Seq2SeqTrainer, Seq2SeqTrainingArguments

multibert = EncoderDecoderModel.from_encoder_decoder_pretrained(
    "bert-base-multilingual-uncased", "bert-base-multilingual-uncased"
)

multibert.save_to_disk("my-model/")

with open("my-model/code/requirements.txt", "w") as fout:
    fout.write("transformers==4.24")Then inside my./scripts/train-v4-24.py, load the model like:# Instead of 
#  multibert = EncoderDecoderModel.from_encoder_decoder_pretrained(
#     "bert-base-multilingual-uncased", "bert-base-multilingual-uncased"
# )
# load the model like this

def train(): 
    multibert = EncoderDecoderModel.load_from_disk("my-model/")

    trainer = Trainer(
    model=model,
    args=training_args,
    compute_metrics=compute_metrics,
    train_dataset=train_dataset,
    eval_dataset=test_dataset,
    tokenizer=tokenizer,
    )

    trainer.train()

def __main__():
    train()Then in the Sagemaker, I can do this without the version:huggingface_estimator = HuggingFace(
        entry_point='train-v4-24.py',
        source_dir='./scripts',
        instance_type='ml.p3.2xlarge',
        instance_count=1,
        role=role,
        hyperparameters = hyperparameters
)Is the above the only way to use the Trainer object with other versions>=4.17?
2022-11-23T14:10:07Z
Also asked ondocker - How to use AWS Sagemaker with newer version of Huggingface Estimator? - Stack Overflowfor wider audience
2022-11-29T17:06:13Z
I published a repo that shows how you can extend the existing HF DLCs in order to usetransformersversions higher than 4.17.0.The repo shows how to extend the Inference Container, but the same applies to the Training DLC. Hope that helps!GitHubGitHub - marshmellow77/sm-extend-container: Extend a pre-built container to...Extend a pre-built container to run latest transformer workloads - GitHub - marshmellow77/sm-extend-container: Extend a pre-built container to run latest transformer workloads
2022-12-16T12:23:59Z
Pardon the hiatus. After trying out different methods, I’ve managed to get different transformers versions to work using either:Custom Docker withimage_uriReference:Adapting Your Own Training Container - Amazon SageMakerSupported version of Huggingface:Any version that you use to create your Docker image.from sagemaker.huggingface import HuggingFace
import sagemaker
import boto3

client = boto3.client('sts')
account = client.get_caller.identit()['Account']
sess = boto3.sessions.Session()
role = sagemaker.get_execution_role()

region = sess.region_name
image_name = "huggingface-custom"
tag = "latest"

ecr_uri = f"{account}.dkr.ecr.{region}.amazonaws.com/{image_name}:{tag}"

estimator = HuggingFace(
  entry_point="train.py",
  source_dir="./scripts",
  instance_type="ml.p3.8large", 
  instance_count=1
  role=role,
  image_uri=ecr_uri,  # Custom image.
  py_version="py38"  # Somehow this is needed.
)

estimator.fit()Using Sagmaker Training Compiler ContainersReference:Amazon SageMaker Training Compiler - Amazon SageMakeramazon-sagemaker-examples/gpt-2.ipynb at main · aws/amazon-sagemaker-examples · GitHubUsing images fromdeep-learning-containers/available_images.md at master · aws/deep-learning-containers · GitHubSupported version of Huggingface:4.21.1from sagemaker.pytorch import PyTorch, TrainingCompilerConfig

estimator = HuggingFace(
  entry_point="train.py",
  source_dir="./scripts",
  instance_type="ml.p3.8large", 
  instance_count=1
  role=role,
  pytorch_version='1.11.0',
  transformers_version="4.21",
  py_version='py3',                  
  compiler_config=TrainingCompilerConfig(),  # Needed to use the Training Compiler
  distribution ={'pytorchxla' : { 'enabled': True }}  # To enable optimized training.
)
2022-12-17T07:28:10Z
I also meant to share this blog post I wrote around the repo I mentioned earlier:Unlock the Latest Transformer Models with Amazon SageMaker | by Heiko Hotz | Dec, 2022 | Towards Data Science
2022-12-20T09:41:32Z
Thank you for the share. Great job on the blogpost!