text
"What Evidence Do Language Models Find Convincing?" is a new paper that explores what types of evidence and argumentation techniques language models find convincing when presented with ambiguous, open-domain questions that have conflicting answers online. 
"What Evidence Do Language Models Find Convincing?" is a new paper that explores what types of evidence and argumentation techniques language models find convincing when presented with ambiguous, open-domain questions that have conflicting answers online. 
new_line
new_line
text
Keypoints:
Keypoints:
new_line
text
* Dataset: It introduces "ConflictingQA," a dataset of controversial questions and real-world evidence paragraphs supporting both "yes" and "no" answers.
* Dataset: It introduces "ConflictingQA," a dataset of controversial questions and real-world evidence paragraphs supporting both "yes" and "no" answers.
new_line
text
* Convincingness Metric: It uses the "paragraph win rate" - when shown two conflicting paragraphs, this measures how often a model predicts the answer that aligns with a given paragraph's stance.
* Convincingness Metric: It uses the "paragraph win rate" - when shown two conflicting paragraphs, this measures how often a model predicts the answer that aligns with a given paragraph's stance.
new_line
text
* Current models rely on the relevance of the content to the query, while largely ignoring stylistic features such as whether a text contains scientific references or if it is written with a neutral tone.
* Current models rely on the relevance of the content to the query, while largely ignoring stylistic features such as whether a text contains scientific references or if it is written with a neutral tone.
new_line
new_line
text
Congrats to the authors for their work!
Congrats to the authors for their work!
new_line
new_line
text
Paper: 
Paper: 
resource
https://huggingface.co/papers/2402.11782
paper
2402.11782
https://huggingface.co/papers/2402.11782
What Evidence Do Language Models Find Convincing? (2402.11782)
new_line
text
Code: 
Code: 
link
https://github.com/AlexWan0/rag-convincingness
https://github.com/AlexWan0/rag-convincingness
"What Evidence Do Language Models Find Convincing?" is a new paper that explores what types of evidence and argumentation techniques language models find convincing when presented with ambiguous, open-domain questions that have conflicting answers online. 

Keypoints:
* Dataset: It introduces "ConflictingQA," a dataset of controversial questions and real-world evidence paragraphs supporting both "yes" and "no" answers.
* Convincingness Metric: It uses the "paragraph win rate" - when shown two conflicting paragraphs, this measures how often a model predicts the answer that aligns with a given paragraph's stance.
* Current models rely on the relevance of the content to the query, while largely ignoring stylistic features such as whether a text contains scientific references or if it is written with a neutral tone.

Congrats to the authors for their work!

Paper: https://huggingface.co/papers/2402.11782
Code: https://github.com/AlexWan0/rag-convincingness
❤️
samusenps dillfrescott osanseviero gsarti victor clem Kukedlc victorshenbr sbrandeis