[text classification] different result format for inference API and inference endpoint
Hi all!I am trying to deploy an inference endpoint for theDeBERTa Base MNLImodel. While the endpoint deploys successfully, I am finding that the result format is different the free inference API as opposed to querying the inference endpoint. See the below results:Inference endpoint:>>> requests.post("<MY INFERENCE ENDPOINT>", headers={"Authorization": f"Bearer <HF API KEY>"}, json={"inputs": "[CLS] I love you. [SEP] I like you. [SEP]"}).json()
[{'label': 'ENTAILMENT', 'score': 0.9248048663139343}]Inference API:>>> requests.post("https://api-inference.huggingface.co/models/microsoft/deberta-base-mnli", headers={"Authorization": f"Bearer <HF API KEY>"}, json={"inputs": "[CLS] I love you. [SEP] I like you. [SEP]"}).json()
[[{'label': 'ENTAILMENT', 'score': 0.9248047471046448}, {'label': 'NEUTRAL', 'score': 0.07485755532979965}, {'label': 'CONTRADICTION', 'score': 0.00033764145337045193}]]Note that through the inference API, I receiveENTAILMENT,NEUTRAL, andCONTRADICTIONscores, where as with the inference endpoint, I only receiveENTAILMENT. For my application, I need all three scores.It’s not immediately clear to me what’s going on here. When creating the inference endpoint, I did so by clicking the deploy dropdown on thedeberta-base-mnlipage, so I would expect for the deployed model to yield the same result format. I am able to replicate this when creating both CPU and GPU inference endpoints for thedeberta-base-mnlimodel and when creating an inference endpoint for thedeberta-large-mnlimodel.Thanks in advance for the help!