Deploying HG Pipelines on AWS Sagemaker
I need some help with deploying HG Pipelines on AWS SagemakerThis is how I deploy a pre-trained model in a pipeline in my local environment and get a prediction.local_model = BertForSequenceClassification.from_pretrained(“local_model_path”)
tokenizer = BertTokenizer.from_pretrained(“"bert-base-uncased", padding=True, truncation=True)
pipe = pipeline("text-classification", model= local_model, tokenizer=.tokenizer)
output = pipe(input)I could not find a way to deploy this HG Pipeline on Sagemaker. I followed the following documentationDeploy models to Amazon SageMakerand tried to use HuggingFaceModel. However, I am not sure how I can provide the other pipeline parameters.huggingface_model = HuggingFaceModel(
   model_data="s3://model.tar.gz",
   role=role,
   transformers_version="4.12",
   pytorch_version="1.9",
   py_version='py38'
)Where and how can I provide the task (text-classification) and tokenizer information? If this is not possible, should I first tokenize my data and give that as an input to predictor? predictor.predict(data_tokenized)?Any other suggestions? Thanks so much!!!
2022-01-15T17:40:18Z
Hi Tom, I think you are on the right path. Provided your model is already on S3 and packaged as tar.gz file you can just use the deploy() method to deploy the model. To get predictions you can use predictor.predict, as you have already pointed out.This notebook by@philschmidshould be useful:notebooks/deploy_transformer_model_from_s3.ipynb at master · huggingface/notebooks · GitHubHope that helps, let me know if there are any questions.CheersHeiko
2022-01-16T15:15:33Z
Thanks for the note. I followed@philschmid’s notebook. However, I am still wondering where and how I should be providing the tokenizer parameter that I usually provide in a HG pipeline. Please see below:pipe = pipeline(“text-classification”, model= local_model, tokenizer=.tokenizer)any ideas?
2022-01-16T16:22:21Z
The tokenizer goes into the model.tar.gz file, together with the model and all other required files for your model to run.
2022-01-17T07:38:52Z
Hello@TomKc,You need to create amodel.tar.gzincluding your tokenizer and model files. Instructions can be found here:Deploy models to Amazon SageMakeror below.{repository}must include those files.cd {repository}
tar zcvf model.tar.gz *
aws s3 cp model.tar.gz <s3://{my-s3-path}>and then you can use theenvparameter to provide your task.huggingface_model = HuggingFaceModel(
   model_data="s3://{my-s3-path}/model.tar.gz",
   role=role,
   transformers_version="4.12",
   pytorch_version="1.9",
   py_version='py38', 
   env = {"HF_TASK":"text-classification"}
)