text
A new paper from Google DeepMind explores the effect of premise ordering on large language models (LLMs) in reasoning tasks. Despite the logical principle that the sequence of premises should not influence the conclusion's validity, the study finds LLMs' performance varies with different premise arrangements. Here's a summary:
A new paper from Google DeepMind explores the effect of premise ordering on large language models (LLMs) in reasoning tasks. Despite the logical principle that the sequence of premises should not influence the conclusion's validity, the study finds LLMs' performance varies with different premise arrangements. Here's a summary:
new_line
new_line
text
The research investigates how the order of premises affects LLMs in logical and mathematical reasoning tasks, challenging the assumption that premise sequence is irrelevant to the outcome.
The research investigates how the order of premises affects LLMs in logical and mathematical reasoning tasks, challenging the assumption that premise sequence is irrelevant to the outcome.
new_line
new_line
text
Key Findings:
Key Findings:
new_line
text
* Logical Reasoning: LLMs perform best when premises are in a forward order that aligns with the proof's progression. Deviations from this order result in significant performance drops.
* Logical Reasoning: LLMs perform best when premises are in a forward order that aligns with the proof's progression. Deviations from this order result in significant performance drops.
new_line
text
* Mathematical Reasoning: The introduction of the R-GSM benchmark shows a similar sensitivity in LLMs.
* Mathematical Reasoning: The introduction of the R-GSM benchmark shows a similar sensitivity in LLMs.
new_line
new_line
text
Congrats to the authors for their work!
Congrats to the authors for their work!
new_line
new_line
text
Paper: 
Paper: 
resource
https://huggingface.co/papers/2402.08939
paper
2402.08939
https://huggingface.co/papers/2402.08939
Premise Order Matters in Reasoning with Large Language Models (2402.08939)
text
.
.
A new paper from Google DeepMind explores the effect of premise ordering on large language models (LLMs) in reasoning tasks. Despite the logical principle that the sequence of premises should not influence the conclusion's validity, the study finds LLMs' performance varies with different premise arrangements. Here's a summary:

The research investigates how the order of premises affects LLMs in logical and mathematical reasoning tasks, challenging the assumption that premise sequence is irrelevant to the outcome.

Key Findings:
* Logical Reasoning: LLMs perform best when premises are in a forward order that aligns with the proof's progression. Deviations from this order result in significant performance drops.
* Mathematical Reasoning: The introduction of the R-GSM benchmark shows a similar sensitivity in LLMs.

Congrats to the authors for their work!

Paper: https://huggingface.co/papers/2402.08939.
👍
osanseviero jjoker999 alielfilali01 flippers clem victor Dlbk cosmojg