How to find confidence score in QA model in the pipeline module
Hi folks, I would like to know how Hugging Face estimates the confidence score, displayed when we use QA model from the Hugging Face “pipeline”. What I know is, QA model tries to predict 2 tokens (starting and ending index). The span between starting and ending index represents the answer.For each token, we understand that the confidence score is represented by the max probability in the output vector. But how to estimate confidence scores when there are 2 output vectors (one for starting token and the other one for ending token) ?
2021-03-30T12:35:44Z
That score is obtained by multiplying the probabilities for start and end tokens: taking the softmax ofstart_logitsto get probabilities, the softmax ofend_logitsthen the value corresponding to the predicted start index end index respectively, and finally multiplying those two numbers.
2021-03-30T12:46:03Z
Okay, got it. Thanks a lot@sgugger.