Calling Image Classification Model Deployed in SageMaker Endpoint
Hi Community!I am deploying this HF model (EdBianchi/vit-fire-detection · Hugging Face) on AWS SageMaker.Once deployed, I have an S3 bucket where I am uploading .jpg images too. This S3 bucket triggers a Lambda function to invoke the SageMaker endpoint.However, I am running into difficulty getting this endpoint working! Playing around, I have seen the endpoint can handle a maximum file size of 5 MB, which I have accounted for.I have tried calling the endpoint with the image/content type set to ‘image/jpeg’ and passing the file as is from S3. I have tried converting the image file to bytes also. Nothing I have attempted is allowing me to successfully call the endpoint and return the classification.Any help in this regard would be greatly appreciated!
2023-02-21T20:22:56Z
Hi Ross, what’s the error that you’re getting when trying to invoke the endpoint?
2023-02-21T20:34:07Z
Hi!Here is the error currently:“message”: “cannot identify image file \u003c_io.BytesIO object at 0x7f3d78578810\u003e”
2023-02-21T20:46:39Z
Hm, ok, I probably need some more info. What transformers version are you using, what does your inference script look like, how do you invoke the endpoint, what serializer are you using, etc. Anything than can help
2023-02-21T20:52:51Z
So I am using the example Deployment Code on HuggingFace.from sagemaker.huggingface import HuggingFaceModel
import sagemaker

role = sagemaker.get_execution_role()
# Hub Model configuration. https://huggingface.co/models
hub = {
	'HF_MODEL_ID':'EdBianchi/vit-fire-detection',
	'HF_TASK':'image-classification'
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
	transformers_version='4.17.0',
	pytorch_version='1.10.2',
	py_version='py38',
	env=hub,
	role=role, 
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
	initial_instance_count=1, # number of instances
	instance_type='ml.m5.xlarge' # ec2 instance type
)
})For the Lambda Function I have a trigger attached to S3 when the JPEG File lands and here is the code to call the SageMaker endpoint:import os
import io
import boto3
import json
import base64

# grab environment variables
ENDPOINT_NAME = os.environ['ENDPOINT_NAME']
runtime= boto3.client('runtime.sagemaker')

# output variables
output_bucket = 'output-wildfire-images'

def lambda_handler(event, context):
    try:
        # Get the object from the S3 event
        s3 = boto3.client("s3")
        bucket = event['Records'][0]['s3']['bucket']['name']
        key = event['Records'][0]['s3']['object']['key']
        
        # Download the image file from S3
        response = s3.get_object(Bucket=bucket, Key=key)
        body = response['Body'].read()
        
        # Encode the image as a base64-encoded string
        img_b64 = base64.b64encode(body).decode('utf-8')
        
        # Convert the base64-encoded string back to bytes
        img_bytes = bytes(img_b64, 'utf-8')
        
        # Invoke the SageMaker endpoint
        response = runtime.invoke_endpoint(
            EndpointName=ENDPOINT_NAME,
            Body=img_bytes,
            ContentType='image/jpeg')
        print(response)

        # Parse the JSON response from the SageMaker endpoint
        result = json.loads(response['Body'].read().decode())
        print(result)
        
        # Upload the result to the output S3 bucket
        output_key = key.split('.')[0] + '.json'
        s3_client = boto3.client('s3')
        s3_client.put_object(Body=json.dumps(result), Bucket=output_bucket, Key=output_key)

        return {
            'statusCode': 200,
            'body': result
        }
    except Exception as e:
        print('Exception:', str(e))
        return {
            'statusCode': 500,
            'body': str(e)
        }I am not sure if the .jpg file needs to be converted to bytes for the Hugging Face model to make the prediction on it…Any help would be appreciated
2023-02-22T04:04:36Z
Ah ok, that is helpful. I believe the problem is that Sagemaker predictor uses (I believe) the JSON serializer when not specified otherwise.Have a look at this sample notebook that also handles images:notebooks/sagemaker-notebook.ipynb at main · huggingface/notebooks · GitHubYou can see that  this notebook  uses aDataSerializerinstead:# create a serializer for the data
image_serializer = DataSerializer(content_type='image/x-image') # using x-image to support multiple image formats

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
	initial_instance_count=1, # number of instances
	instance_type='ml.g4dn.xlarge', # ec2 instance type
    serializer=image_serializer, # serializer for image data.
)Hopefully that fixes your issue. You can learn more about Sagemaker SerializershereCheersHeiko
2023-03-03T19:30:44Z
Hi, I am still having issues getting this wildifre model endpoint to call correctly and return the result.I have worked with the example you sent for image segmentation, which works fine -notebooks/sagemaker-notebook.ipynb at main · huggingface/notebooks · GitHub.Here is the code for deploying the wildifre model in SageMaker Notebook that I am using.import base64
import json
import sagemaker

# Role
role = sagemaker.get_execution_role()
print(role)

# SageMaker 
from sagemaker.huggingface import HuggingFaceModel
from sagemaker.serializers import DataSerializer

role = sagemaker.get_execution_role()
# Hub Model configuration. https://huggingface.co/models
hub = {
	'HF_MODEL_ID':'EdBianchi/vit-fire-detection',
	'HF_TASK':'image-classification'
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
    transformers_version='4.17.0',
	pytorch_version='1.10.2',
	py_version='py38',
	env=hub,
	role=role, 
)

# create a serializer for the data
image_serializer = DataSerializer(content_type='image/x-image') # using x-image to support multiple image formats

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
	initial_instance_count=1, # number of instances
	instance_type='ml.m5.xlarge', # ec2 instance type
        serializer=image_serializer, # serializer for image data.
)

# download the image 
!wget https://huggingface.co/datasets/hf-internal-testing/fixtures_ade20k/raw/main/ADE_val_00000001.jpg

# File path
image_path = "ADE_val_00000001.jpg"

# Call the endpoint
with open(image_path, "rb") as data_file:
  image_data = data_file.read()
  res = predictor.predict(data=image_data)
  print(res)I run into this error…ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
  "code": 400,
  "type": "InternalServerException",
  "message": "an integer is required (got type str)"
}
2023-03-06T08:45:23Z
Hello@TBRE,can you share the logs from cloudwatch?
2023-03-06T20:47:07Z
Hi@philschmid,Here is the CloudWatch log response."message": "cannot identify image file \u003c_io.BytesIO object at 0x7f3d783d4a40\u003e"

2023-02-21T15:46:24,565 [INFO ] W-EdBianchi__vit-fire-detec-4-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - mms.service.PredictionException: an integer is required (got type str) : 400
2023-03-06T20:59:12Z
It seems that the container couldn’t parse your image correctly. Could you try as mime-typeimage/jpeginstead ofx-image?
2023-03-08T02:26:53Z
Tried changing toimage/jpegand still not working
2023-03-08T07:59:26Z
can you try a different image? it seems something is off with the request payload
2023-03-22T11:59:47Z
I’m having the same issue - I’ve tried appending “data:image/jpeg;base64,” and just sending the raw B64 image - however no approach works - did you find a way around this?
2023-03-23T01:03:30Z
Hi@Neesh,In short, no I wasn’t able to.I had a change in priorities as this was taking too long to set up.I tried similar techniques to you and was not able to get the inference working.
2023-04-13T19:27:03Z
I am having a similar issue. was anyone able to fix this?
2023-04-13T21:39:03Z
Not yet! Can anyone from hugging face directly test this model and share some insight?
2023-04-14T06:54:07Z
Have you checked out this example?notebooks/sagemaker-notebook.ipynb at main · huggingface/notebooks · GitHubIt is usingimage-segementationbut the process should be the same and it is tested and works.
2023-07-11T13:59:23Z
I am also having issues setting up on inference api. Im using stablediffusion and follow phillipps examples.Keep running into this cannot identify image file <_io.BytesIO error. The handler.py file works perfectly in colab.
2024-04-02T18:18:27Z
TBRE:with open(image_path, "r") as data_file:
  image_data = data_file.read()
  res = predictor.predict(data=image_data)
  print(res)try with “r” instead of “rb”
2024-04-04T23:06:41Z
Make sure your image not only meets the maximum file size, but is also encoded correctly. Make sure you are using the correct image format when calling the SageMaker endpoint. It’s also worth checking your S3 permissions and Lambda settings to make sure they’re configured correctly for file access.