Qwen/Qwen1.5-72B-Chat
import sagemakerimport boto3from sagemaker.huggingface import HuggingFaceModeltry:role = sagemaker.get_execution_role()except ValueError:iam = boto3.client(‘iam’)role = iam.get_role(RoleName=‘sagemaker_execution_role’)[‘Role’][‘Arn’]Hub Model configuration.Models - Hugging Facehub = {‘HF_MODEL_ID’:‘Qwen/Qwen1.5-72B-Chat’,‘HF_TASK’:‘text-generation’}create Hugging Face Model Classhuggingface_model = HuggingFaceModel(transformers_version=‘4.26.0’,pytorch_version=‘1.13.1’,py_version=‘py39’,env=hub,role=role,)deploy model to SageMaker Inferencepredictor = huggingface_model.deploy(initial_instance_count=1, # number of instancesinstance_type=‘ml.m5.xlarge’ # ec2 instance type)predictor.predict({“inputs”: "Can you please let us know more details about your ",})for this I am getting this errorModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{“code”: 400,“type”: “InternalServerException”,“message”: “\u0027qwen2\u0027”}
2024-03-19T13:30:40Z
you can not depoly a 72B model to a m5.xlarge…