text
🔍 Today's pick in Interpretability & Analysis of LMs: Do language models plan ahead for future tokens? by W. Wu 
🔍 Today's pick in Interpretability & Analysis of LMs: Do language models plan ahead for future tokens? by W. Wu 
mention
@jxm
jxm
text
mention
@lionellevine
lionellevine
new_line
new_line
text
This work aims to evaluate whether language models exhibit implicit planning during generation.
This work aims to evaluate whether language models exhibit implicit planning during generation.
new_line
new_line
text
Authors propose two hypotheses that could result in planning-like behavior: 
Authors propose two hypotheses that could result in planning-like behavior: 
new_line
text
- Pre-caching: the model engages in computation that is functional to future, but not current, predictions. 
- Pre-caching: the model engages in computation that is functional to future, but not current, predictions. 
new_line
text
- Breadcrumbs: Features contributing to the current prediction happen to also be the ones improving future ones.
- Breadcrumbs: Features contributing to the current prediction happen to also be the ones improving future ones.
new_line
new_line
text
To validate which behavior is observed in practice, authors note that off-diagonal gradients for weight matrices across the model are the ones responsible for pre-caching, and craft a variant of gradient descent (myopic descent) to remove such terms from the optimization procedure.
To validate which behavior is observed in practice, authors note that off-diagonal gradients for weight matrices across the model are the ones responsible for pre-caching, and craft a variant of gradient descent (myopic descent) to remove such terms from the optimization procedure.
new_line
new_line
text
Using a synthetic dataset, authors demonstrate that pre-caching occurs in Transformers language models. However, for natural language settings the LM is observed to leverage breadcrumbs from previous passes even in the case of myopic training, rendering the latter hypothesis more plausible to account for model behavior.
Using a synthetic dataset, authors demonstrate that pre-caching occurs in Transformers language models. However, for natural language settings the LM is observed to leverage breadcrumbs from previous passes even in the case of myopic training, rendering the latter hypothesis more plausible to account for model behavior.
new_line
new_line
text
📄 Paper: 
📄 Paper: 
resource
https://huggingface.co/papers/2404.00859
paper
2404.00859
https://huggingface.co/papers/2404.00859
Do language models plan ahead for future tokens? (2404.00859)
new_line
new_line
text
🔍 All daily picks: 
🔍 All daily picks: 
link
https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9
https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9
🔍 Today's pick in Interpretability & Analysis of LMs: Do language models plan ahead for future tokens? by W. Wu @jxm @lionellevine

This work aims to evaluate whether language models exhibit implicit planning during generation.

Authors propose two hypotheses that could result in planning-like behavior: 
- Pre-caching: the model engages in computation that is functional to future, but not current, predictions. 
- Breadcrumbs: Features contributing to the current prediction happen to also be the ones improving future ones.

To validate which behavior is observed in practice, authors note that off-diagonal gradients for weight matrices across the model are the ones responsible for pre-caching, and craft a variant of gradient descent (myopic descent) to remove such terms from the optimization procedure.

Using a synthetic dataset, authors demonstrate that pre-caching occurs in Transformers language models. However, for natural language settings the LM is observed to leverage breadcrumbs from previous passes even in the case of myopic training, rendering the latter hypothesis more plausible to account for model behavior.

📄 Paper: https://huggingface.co/papers/2404.00859

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9
❤️
samusenps jxm zoratrox lordvader31 NisargBhavsar25