InternalServerException when running a model loaded on S3
Hi there,I am trying to deploy a model loaded on S3, following the steps found mainly on this video:[Deploy a Hugging Face Transformers Model from S3 to Amazon SageMaker](https://www.youtube.com/watch?v=pfBGgSGnYLs).For that I have downloaded a model into a S3 bucket and use this image URI for DLC: image_uri = “763104351884.dkr.ecr.eu-west-1.amazonaws.com/huggingface-pytorch-inference:1.7.1-transformers4.6.1-cpu-py36-ubuntu18.04”When I run thepredictor.predict(data)command, I get this error:Captura de pantalla 2021-08-05 a las 18.13.381168×1127 179 KBThe model I use fot these tests is this one: dccuchile/bert-base-spanish-wwm-uncased, and I could not find the way for letting the model know which action should perform.I am pretty new with HuggingFace technology, and probably I am missing the point for fixing that.Please, could you let me know what should I do for informing the model about what to do?Thank you!
2021-08-05T16:32:47Z
Hey@vicente-m47,Could you please provide the whole code you executed?If you want to deploy a model fromModels - Hugging Faceyou can use the “deploy” button on each of the model pages.image1456×382 55.7 KBThis will generate a code snippet for youimage1455×1159 178 KBFrom reading the code you attached you are trying to send an input forquestion-answeringto a model (dccuchile/bert-base-spanish-wwm-uncased), which is not fine-tuned forquestion-answering, also you are sending an English input to a Spanish model.A good starting point for new Hugging Facer is our course atIntroduction - Hugging Face Course.You can find more information about deploying to sagemaker in the documentation hereDeploy models to Amazon SageMaker
2021-08-06T07:04:42Z
Hi,@philschmid,Sorry, you are right, my question was awfully written. Here is the code I used:!pip install sagemaker --upgradefrom sagemaker.huggingface import HuggingFaceModel
import sagemaker

role = sagemaker.get_execution_role()

# This model was supposedly trained from the dccuchile Bert model for QA.
model = "s3://xxxxxxx/hg_model.tar.gz"
image_uri = "763104351884.dkr.ecr.eu-west-1.amazonaws.com/huggingface-pytorch-inference:1.7.1-transformers4.6.1-cpu-py36-ubuntu18.04"

huggingface_model = HuggingFaceModel(
    model_data=model,
    transformers_version='4.6.1',
    pytorch_version='1.7.1',
    py_version='py36',
    image_uri=image_uri,
    role=role, 
)predictor = huggingface_model.deploy(
    initial_instance_count=1,
    instance_type="ml.m5.xlarge"
)data = {
    "inputs": {
        "question": "¿Cómo me llamo?",
        "context": "Mi nombre es Juan y vivo en Francia"
    }
}

predictor.predict(data)Reading your comment, I understand that the problem should be on the trained model. I would review the course you recommend and check the training I did.Thank you!
2021-08-06T07:53:04Z
You don’t need to provide animage_urithe HuggingFaceModel will select the right one based onpytorch_versionandtransformers_version.
2021-08-06T09:39:19Z
Yes, you’re right. Thank you,@philschmid.