Sagemaker Huggingface zero shot classification with candidate_labels in predict method
Hi,I am using HF for zero shot classification for doc classification. I successfully implemented code with native HF library. After working code I deploy the model using SM, but I am not able to find anywhere how to pass the custom labels to the predict method?Using HF library:classifier = pipeline("zero-shot-classification")
a = classifier(
    "Sample Vaccination Record Card Mary Major M Last Name First Name MI 1/6/58 012345abcd67  blah blah blah….",
    candidate_labels=["Medical Card", "ID card", "Passport" ,"education", "politics", "business"],
)I got the prediction score for each candidate label. So all good until hereNow I deployed the model using our SM:hub_v1 = {
	'HF_MODEL_ID':'cross-encoder/nli-distilroberta-base',
	'HF_TASK':'text-classification'
}
huggingface_model_v1 = HuggingFaceModel(
	transformers_version='4.17.0',
	pytorch_version='1.10.2',
	py_version='py38',
	env=hub_v1,
	role=role, 
)
serverless_config_v1 = ServerlessInferenceConfig(
    memory_size_in_mb=4096, max_concurrency=10,
)
predictor_v1 = huggingface_model_v1.deploy(
    serverless_inference_config=serverless_config_v1
)I called the endpoint like thispredictor_v1.predict({
	'inputs': "Sample Vaccination Record Card Mary Major M Last Name First Name MI 1/6/58 012345abcd67  blah blah blah…",
    'candidate_labels' : ['WILL', 'ID', 'P60', 'OTHER']
})i get output like this:[{'label': 'ENTAILMENT', 'score': 0.5859580636024475}]I need the prediction score for the input string per each of the four categories.Any idea? Am i missing something here? Appreciate your help
2022-06-16T07:04:11Z
@dineshmaneyou deploy your model on sagemaker with the tasktext-classificationand in the pipeline you usezero-shot-classification. If you change your sagemaker task it should work.Also you payload was a bit off. An example of azero-shot-classifcationpayload can be found in the documentation and I also added a json below.Reference{
  "inputs": "Hi, I recently bought a device from your company but it is not working as advertised and I would like to get reimbursed!",
  "parameters": {
    "candidate_labels": ["refund", "legal", "faq"]
  }
}
2022-06-16T14:17:24Z
You the best@philschmid! Thank youIt is working now.working code basefrom sagemaker.huggingface.model import HuggingFaceModel
from sagemaker.serverless import ServerlessInferenceConfig

# Hub Model configuration. <https://huggingface.co/models>
hub_v1 = {
	'HF_MODEL_ID':'cross-encoder/nli-distilroberta-base',
	'HF_TASK':'zero-shot-classification'
}

# create Hugging Face Model Class
huggingface_model_v1 = HuggingFaceModel(
	transformers_version='4.17.0',
	pytorch_version='1.10.2',
	py_version='py38',
	env=hub_v1,
	role=role, 
)

# Specify MemorySizeInMB and MaxConcurrency in the serverless config object
serverless_config_v1 = ServerlessInferenceConfig(
    memory_size_in_mb=4096, max_concurrency=10,
)

# deploy the endpoint endpoint
predictor_v1 = huggingface_model_v1.deploy(
    serverless_inference_config=serverless_config_v1
)

predictor_v1.predict({
	'inputs': " 
Sample Vaccination Record Card Mary Major M Last Name First Name MI 1/6/58 012345abcd67  blah blah ",
    "parameters":{'candidate_labels' : label_list}
})output was like below:{'sequence': '   Sample Vaccination Record Card Mary Major M Last Name First Name MI 1/6/58 012345abcd67 blah blah  ',
 'labels': ['RENT',
  'Medical_Document',
  'p60 form',
  'uk_driving_licence',
  'WILL',
  'passport'],
 'scores': [0.27988865971565247,
  0.17847613990306854,
  0.16656287014484406,
  0.15448175370693207,
  0.125006765127182,
  0.09558377414941788]}