text
So, I have this idea to (potentially) improve uncertainty quantification for LLM hallucination detection.
So, I have this idea to (potentially) improve uncertainty quantification for LLM hallucination detection.
new_line
new_line
text
The premise is that not all output tokens of a generated response share the same importance. Hallucinations are more dangerous in the form of a noun, date, number, etc.
The premise is that not all output tokens of a generated response share the same importance. Hallucinations are more dangerous in the form of a noun, date, number, etc.
new_line
new_line
text
The idea is to have a "token selection" layer that filters the output token probabilities sequence. Then, we use only the probabilities of the relevant tokens to calculate uncertainty quantification metrics.
The idea is to have a "token selection" layer that filters the output token probabilities sequence. Then, we use only the probabilities of the relevant tokens to calculate uncertainty quantification metrics.
new_line
new_line
text
The big question is how we know which tokens are the relevant ones. 🤔
The big question is how we know which tokens are the relevant ones. 🤔
new_line
new_line
text
My idea is to use the output sequence (decoded one) and use an NLP model (it doesn't need to be a fancy one) to do entity recognition and part-of-speech tagging to the output sequence and then do uncertainty quantification only on the entities that we have set as relevant (nouns, dates, numbers, etc).
My idea is to use the output sequence (decoded one) and use an NLP model (it doesn't need to be a fancy one) to do entity recognition and part-of-speech tagging to the output sequence and then do uncertainty quantification only on the entities that we have set as relevant (nouns, dates, numbers, etc).
new_line
new_line
text
What are your thoughts? Have you seen anyone try this before? 
What are your thoughts? Have you seen anyone try this before? 
new_line
new_line
text
Curious to see if anyone has tried this before and know if this would have an impact on the correlation with human-annotated evaluations.
Curious to see if anyone has tried this before and know if this would have an impact on the correlation with human-annotated evaluations.
So, I have this idea to (potentially) improve uncertainty quantification for LLM hallucination detection.

The premise is that not all output tokens of a generated response share the same importance. Hallucinations are more dangerous in the form of a noun, date, number, etc.

The idea is to have a "token selection" layer that filters the output token probabilities sequence. Then, we use only the probabilities of the relevant tokens to calculate uncertainty quantification metrics.

The big question is how we know which tokens are the relevant ones. 🤔

My idea is to use the output sequence (decoded one) and use an NLP model (it doesn't need to be a fancy one) to do entity recognition and part-of-speech tagging to the output sequence and then do uncertainty quantification only on the entities that we have set as relevant (nouns, dates, numbers, etc).

What are your thoughts? Have you seen anyone try this before? 

Curious to see if anyone has tried this before and know if this would have an impact on the correlation with human-annotated evaluations.
❤️
gsarti osanseviero cnmoro clem santiviquez
https://cdn-avatars.huggingface.co/v1/production/uploads/1657144463525-629a173153a72d997d3f57d0.jpeg
Santiago Viquez
santiviquez
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1670231290373-5e7749883d77a72421292d07.jpeg
Gabriele Sarti
gsarti
user