How can I create lambda function code after deploying bart transformer for summarization in sagemaker?
I deployed bart-large-cnn model in aws sagemaker for summarization task by following the code.from sagemaker.huggingface import HuggingFaceModelimport sagemakerrole = sagemaker.get_execution_role()Hub Model configuration.Models - Hugging Facehub = {‘HF_MODEL_ID’:‘facebook/bart-large-cnn’, # model_id fromModels - Hugging Face‘HF_TASK’:‘summarization’ # NLP task you want to use for predictions}create Hugging Face Model Classhuggingface_model = HuggingFaceModel(env=hub,role=role, # iam role with permissions to create an Endpointtransformers_version=“4.6”, # transformers version usedpytorch_version=“1.7”, # pytorch version usedpy_version=“py36”, # python version of the DLC)deploy model to SageMaker Inferencepredictor = huggingface_model.deploy(initial_instance_count=1,instance_type=“ml.m5.xlarge”)Then tested with text to extract summaries and it performed well.What code i needed to add in lambda function, for creating a rest api to call the summariZation model endpoint using aws lambda and api gateway.
2021-09-27T07:43:41Z
Hello@vihaary,Create to hear it worked as expected. Invoking your endpoint from an AWS Lambda function is pretty easy you can either use theboto3sagemaker-runtimeor install thesagemakersdk into your lambda function.boto3snippet:import boto3
import json
client = boto3.client('sagemaker-runtime')

response = client.invoke_endpoint(
EndpointName=ENDPOINT_NAME,
ContentType="application/json",
Accept="application/json",
Body=json.dumps(payload),
)
print(response['Body'].read().decode())sagemakersnippetfrom sagemaker.huggingface import HuggingFacePredictor

predictor = HuggingFacePredictor(ENDPOINT_NAME)
response = predictor.predict(payload)
print(response)
2021-09-27T07:54:04Z
Hello@philschmid, thanks for your response. Will I get summary as response output for text paragraphs as input with this lambda code?.
2021-09-27T12:45:41Z
No that is just how you can invoke your endpoint in your AWS Lambda function.
2021-09-28T04:26:09Z
@philschmid{“errorMessage”: “An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from model with message “{\n  “code”: 400,\n  “type”: “InternalServerException”,\n  “message”: “\u0027numpy.ndarray\u0027 object has no attribute \u0027pop\u0027”\n}\n”. Seehttps://us-west-2.console.aws.amazon.com/cloudwatch/home?region=us-west-2#logEventViewer:group=/aws/sagemaker/Endpoints/huggingface-pytorch-inference-2021-09-27-12-40-12-385in account 215401044306 for more information.”,“errorType”: “ModelError”,“stackTrace”: ["  File “/var/task/lambda_function.py”, line 20, in lambda_handler\n    Body=payload)\n","  File “/var/runtime/botocore/client.py”, line 386, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n","  File “/var/runtime/botocore/client.py”, line 705, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n"]}I am getting this error. Also can i get summary as output for text paragraphs as input after invoking my endpoint in aws lambda function?
2021-09-28T07:06:10Z
vihaary:“\u0027numpy.ndarray\u0027 object has no attribute \u0027pop\u0027”\n}\n”You are not sending ajsoninput to the endpoint.vihaary:I am getting this error. Also can i get summary as output for text paragraphs as input after invoking my endpoint in aws lambda function?Yes
2021-09-28T08:02:57Z
@philschmidThanks for the reply.I tried this oneimport osimport ioimport boto3import jsonimport csvgrab environment variablesENDPOINT_NAME = os.environ[‘ENDPOINT_NAME’]runtime= boto3.client(‘runtime.sagemaker’)def lambda_handler(event, context):print("Received event: " + json.dumps(event, indent=2))data = json.loads(json.dumps(event))
payload = data['data']
print(payload)

response = runtime.invoke_endpoint(EndpointName=ENDPOINT_NAME,
                                   ContentType='text/csv',
                                   Body=payload)
print(response)
result = json.loads(response['Body'].read().decode())
print(result)But i am not getting the summary output. Can you provide a sample code for lambda function where i can give paragraph as input and i should get summary as output for my huggingface bart model endpoint.
2021-09-28T11:25:12Z
YourContentTypeis wrong if you compare it to what I have shared. I am going to create a sample for this but not before next week.
2021-09-30T04:10:07Z
Thanks@philschmid. Will be waiting for your response
2021-10-05T05:01:16Z
Hi@philschmid. Can you try to create a lambda function sample for the above bart model summarization task?. Thanks in advance.
2021-10-05T06:52:32Z
Hey,I’ll post about it later on Social Media (Twitter, LinkedIn) but here is the example:cdk-samples/aws-lambda-sagemaker-endpoint-huggingface at master · philschmid/cdk-samples · GitHubIt is an example using the CDK so if you want to deploy a BART model for summarization you can follow the readme.clone the repositorygit clone https://github.com/philschmid/cdk-samples.git
cd aws-lambda-sagemaker-endpoint-huggingfaceInstall the cdk required dependencies. Make your you have thecdkinstalled.pip3 install -r requirements.txtBootstrapyour application in the cloud.cdk bootstrapDeploy your Hugging Face Transformer model to Amazon SageMakercdk deploy \
-c model="facebook/bart-large-cnn" \
-c task="summarization"clean upcdk destroy \
-c model="facebook/bart-large-cnn" \
-c task="summarization"If you only want to know/see the lambda code you can look herecdk-samples/handler.py at master · philschmid/cdk-samples · GitHub
2022-01-21T06:25:17Z
Hi@vihaary,Do you able to solve that problem ? If yes then please share the code with me . I need that code as I am working on a project regarding that things…Thanks
2022-01-21T07:36:07Z
Yes, as mentioned above I created an example on how to do do this using AWS CDK.philschmid:If you only want to know/see the lambda code you can look herecdk-samples/handler.py at master · philschmid/cdk-samples · GitHub