Can text-to-image models be deployed to a SageMaker endpoint?
I have had trouble finding any resources pertaining to deploying text-to-image models on SageMaker. I’m working with DALLE-Mega and I am able to run it in a notebook on SageMaker but when it comes to deploying it with theSageMaker Hugging Face Inference ToolkitI am having trouble.This is from my SageMaker notebook to deploy the endpoint:from sagemaker.huggingface import HuggingFaceModel
import sagemaker

sess = sagemaker.Session()
role = sagemaker.get_execution_role()

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
    transformers_version='4.6',
    pytorch_version='1.7',
    py_version='py36',
    model_data='s3://example/dalle-mega',
    role=role,
    env={ 'HF_TASK':'text-to-image' },
)
# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(initial_instance_count=1,instance_type="ml.g4dn.xlarge")Because text-to-image is not a valid HF_TASK I get this error:ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
  "code": 400,
  "type": "InternalServerException",
  "message": "\"Unknown task text-to-image, available tasks are [\u0027feature-extraction\u0027, \u0027text-classification\u0027, \u0027token-classification\u0027, \u0027question-answering\u0027, \u0027table-question-answering\u0027, \u0027fill-mask\u0027, \u0027summarization\u0027, \u0027translation\u0027, \u0027text2text-generation\u0027, \u0027text-generation\u0027, \u0027zero-shot-classification\u0027, \u0027conversational\u0027, \u0027image-classification\u0027, \u0027translation_XX_to_YY\u0027]\""
}Removing the “env={ ‘HF_TASK’:‘text-to-image’ },” line will allow the endpoint to be created without error, but whenever I attempt to call the endpoint it generates an error about not knowing what kind of task it is.All that to say, the possible tasks listed in the error clearly do not include “text-to-image” so is it even possible to deploy a model like DALLE mini/mega to SageMaker?
2022-07-08T08:23:19Z
Hey@ColinPM,The Inference toolkit supports 0 code deployments for transformerspipelinescurrently there is notext-to-imagepipeline support.So to get your model working you need to create ainference.pysimilar to the one here:notebooks/sagemaker-notebook.ipynb at main · huggingface/notebooks · GitHub