text
🔍 Today's pick in Interpretability & Analysis of LMs: Information Flow Routes: Automatically Interpreting Language Models at Scale by 
🔍 Today's pick in Interpretability & Analysis of LMs: Information Flow Routes: Automatically Interpreting Language Models at Scale by 
mention
@javifer
javifer
text
mention
@lena-voita
lena-voita
new_line
new_line
text
This work presents a novel method to identify salient components in Transformer-based language models by decomposing the contribution of various model components into the residual stream.
This work presents a novel method to identify salient components in Transformer-based language models by decomposing the contribution of various model components into the residual stream.
new_line
new_line
text
This method is more efficient and scalable than previous techniques such as activation patching, as it only requires a single forward pass through the model to identify critical information flow paths. Moreover, it can be applied without a contrastive template, which is observed to produce results dependent on the selected contrastive example for activation patching.
This method is more efficient and scalable than previous techniques such as activation patching, as it only requires a single forward pass through the model to identify critical information flow paths. Moreover, it can be applied without a contrastive template, which is observed to produce results dependent on the selected contrastive example for activation patching.
new_line
new_line
text
Information flow routes are applied to Llama 2, showing that:
Information flow routes are applied to Llama 2, showing that:
new_line
new_line
text
1. Models show “typical” information flow routes for non-content words, while content words don’t exhibit such patterns.
1. Models show “typical” information flow routes for non-content words, while content words don’t exhibit such patterns.
new_line
text
2. Feedforward networks are more active in the bottom layers of the network (where e.g. subject enrichment is performed) and in very last layer.
2. Feedforward networks are more active in the bottom layers of the network (where e.g. subject enrichment is performed) and in very last layer.
new_line
text
3. Positional and subword-merging attention heads are among the most active and important throughout the network.
3. Positional and subword-merging attention heads are among the most active and important throughout the network.
new_line
text
4. Periods can be treated by the model as BOS tokens by leaving their residual representation mostly untouched during the forward pass.
4. Periods can be treated by the model as BOS tokens by leaving their residual representation mostly untouched during the forward pass.
new_line
new_line
text
Finally, the paper also demonstrates that some model components are specialized for specific domains, such as coding or multilingual texts, suggesting a high degree of modularity in the network. The contribution of domain-specific heads obtained by projecting right singular values of the OV circuit to the unembedding matrix show highly interpretable concepts being handled in granular model components.
Finally, the paper also demonstrates that some model components are specialized for specific domains, such as coding or multilingual texts, suggesting a high degree of modularity in the network. The contribution of domain-specific heads obtained by projecting right singular values of the OV circuit to the unembedding matrix show highly interpretable concepts being handled in granular model components.
new_line
new_line
text
📄 Paper: 
📄 Paper: 
resource
https://huggingface.co/papers/2403.00824
paper
2403.00824
https://huggingface.co/papers/2403.00824
Information Flow Routes: Automatically Interpreting Language Models at
  Scale (2403.00824)
new_line
new_line
text
🔍 All daily picks: 
🔍 All daily picks: 
resource
https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9
collection
gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9
https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9
🔍 Today's pick in Interpretability & Analysis of LMs: Information Flow Routes: Automatically Interpreting Language Models at Scale by @javifer @lena-voita

This work presents a novel method to identify salient components in Transformer-based language models by decomposing the contribution of various model components into the residual stream.

This method is more efficient and scalable than previous techniques such as activation patching, as it only requires a single forward pass through the model to identify critical information flow paths. Moreover, it can be applied without a contrastive template, which is observed to produce results dependent on the selected contrastive example for activation patching.

Information flow routes are applied to Llama 2, showing that:

1. Models show “typical” information flow routes for non-content words, while content words don’t exhibit such patterns.
2. Feedforward networks are more active in the bottom layers of the network (where e.g. subject enrichment is performed) and in very last layer.
3. Positional and subword-merging attention heads are among the most active and important throughout the network.
4. Periods can be treated by the model as BOS tokens by leaving their residual representation mostly untouched during the forward pass.

Finally, the paper also demonstrates that some model components are specialized for specific domains, such as coding or multilingual texts, suggesting a high degree of modularity in the network. The contribution of domain-specific heads obtained by projecting right singular values of the OV circuit to the unembedding matrix show highly interpretable concepts being handled in granular model components.

📄 Paper: https://huggingface.co/papers/2403.00824

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9
❤️
Kukedlc samusenps javifer CristianJD Theli
🔥
eramax mmhamdy