text
Confidence * may be * all you need.
Confidence * may be * all you need.
new_line
new_line
text
A simple average of the log probabilities of the output tokens from an LLM might be all it takes to tell if the model is hallucinating.🫨
A simple average of the log probabilities of the output tokens from an LLM might be all it takes to tell if the model is hallucinating.🫨
new_line
new_line
text
The idea is that if a model is not confident (low output token probabilities), the model may be inventing random stuff.
The idea is that if a model is not confident (low output token probabilities), the model may be inventing random stuff.
new_line
new_line
text
In these two papers:
In these two papers:
new_line
text
1. 
1. 
link
https://aclanthology.org/2023.eacl-main.75/
https://aclanthology.org/2023.eacl-main.75/
new_line
text
2. 
2. 
link
https://arxiv.org/abs/2303.08896
https://arxiv.org/abs/2303.08896
new_line
new_line
text
The authors claim that this simple method is the best heuristic for detecting hallucinations. The beauty is that it only uses the generated token probabilities, so it can be implemented at inference time ⚡
The authors claim that this simple method is the best heuristic for detecting hallucinations. The beauty is that it only uses the generated token probabilities, so it can be implemented at inference time ⚡
Confidence * may be * all you need.

A simple average of the log probabilities of the output tokens from an LLM might be all it takes to tell if the model is hallucinating.🫨

The idea is that if a model is not confident (low output token probabilities), the model may be inventing random stuff.

In these two papers:
1. https://aclanthology.org/2023.eacl-main.75/
2. https://arxiv.org/abs/2303.08896

The authors claim that this simple method is the best heuristic for detecting hallucinations. The beauty is that it only uses the generated token probabilities, so it can be implemented at inference time ⚡
❤️
clem leegao19 osanseviero victor s3nh abbasm2 gsarti rreed-pha stolsvik
https://cdn-avatars.huggingface.co/v1/production/uploads/1661497922734-62f4ac43567dbf9a39f75474.jpeg
Daniel Huynh
dhuynh95
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1657144463525-629a173153a72d997d3f57d0.jpeg
Santiago Viquez
santiviquez
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1670231290373-5e7749883d77a72421292d07.jpeg
Gabriele Sarti
gsarti
user