Error deploying endpoint on Aws
I’m trying to deploy my finetuned LLama3 model on Aws, so the first step to create an endpointI used instance_type=“ml.g5.4xlarge”,This is my code:import jsonimport sagemakerimport boto3from sagemaker.huggingface import HuggingFaceModel, get_huggingface_llm_image_uritry:role = sagemaker.get_execution_role()except ValueError:iam = boto3.client(‘iam’)role = iam.get_role(RoleName=‘sagemaker_execution_role’)[‘Role’][‘Arn’]Hub Model configuration.https://huggingface.co/modelshub = {‘HF_MODEL_ID’:‘Guepard/knaine_llama3.1_v0’,‘HF_TASK’: ‘text-generation’}create Hugging Face Model Classhuggingface_model = HuggingFaceModel(image_uri=get_huggingface_llm_image_uri(“huggingface”,version=“2.0.2”),env=hub,role=role,)deploy model to SageMaker Inferencepredictor = huggingface_model.deploy(initial_instance_count=1,instance_type=“ml.g5.4xlarge”,container_startup_health_check_timeout=1200,)send requestpredictor.predict({“inputs”: “My name is Julien and I like to”,})I get the following error :UnexpectedStatusException: Error hosting endpoint huggingface-pytorch-tgi-inference-2024-08-23-08-25-25-823: Failed. Reason: The primary container for production variant AllTraffic did not pass the ping health check. Please check CloudWatch logs for this endpoint… Try changing the instance type or reference the troubleshooting pageTroubleshooting - Amazon SageMakerAny solution?
2024-08-23T09:12:54Z
hi@Knour13Did you check CloudWatch logs?github.com/aws/amazon-sagemaker-examplesFailed Reason:  The primary container for production variant AllTraffic did not pass the ping health check.opened04:26PM - 08 Mar 18 UTCclosed09:46PM - 19 Mar 18 UTCdtsukiyamaI am trying to deploy a BYOB (bring your own model) keras model. I pushed the im…age to ECR with the 'latest' tag. All local testing passed, and I am able to successfully train the model e.g.:

```python

image = '{}.dkr.ecr.{}.amazonaws.com/my-model:latest'.format(account, region)

dl = sage.estimator.Estimator(image,
                       role, 1, 'ml.c4.2xlarge',
                       output_path="s3://{}/output".format(sess.default_bucket()),
                       sagemaker_session=sess)

```

However attempting to deploy  gives me the error:

```
Failed Reason:  The primary container for production variant AllTraffic did not pass the ping health check.
```
I am not quite sure where this stems from given local health check passed. Any insight would be great! Thanks.
2024-08-23T11:29:50Z
I found in cloud watch these errors:image1336×77 3.32 KBimage1435×91 5.13 KB
2024-08-23T11:31:29Z
@philschmidHi Do you have any solution ?Thanks
2024-08-23T11:52:11Z
Try TGI 2.2.0
2024-08-23T12:20:55Z
Same errorimage1268×66 6.2 KB
2024-08-23T12:42:38Z
image1386×235 9.45 KBThis is cloud watch