text
Good folks from Universitat Politècnica de Catalunya, University of Groningen, and Meta have released "A Primer on the Inner Workings of Transformer-based Language Models."
Good folks from Universitat Politècnica de Catalunya, University of Groningen, and Meta have released "A Primer on the Inner Workings of Transformer-based Language Models."
new_line
new_line
text
They don't make survey papers like they used to, but this is an exciting new survey on Transformer LM interpretability!
They don't make survey papers like they used to, but this is an exciting new survey on Transformer LM interpretability!
new_line
new_line
text
This comprehensive survey provides a technical deep dive into:
This comprehensive survey provides a technical deep dive into:
new_line
new_line
text
• Transformer architecture components (attention, FFN, residual stream)
• Transformer architecture components (attention, FFN, residual stream)
new_line
text
• Methods for localizing model behavior:
• Methods for localizing model behavior:
new_line
text
  - Input attribution (gradient & perturbation-based)
  - Input attribution (gradient & perturbation-based)
new_line
text
  - Component importance (logit attribution, causal interventions)
  - Component importance (logit attribution, causal interventions)
new_line
text
• Information decoding techniques:
• Information decoding techniques:
new_line
text
  - Probing, linear feature analysis
  - Probing, linear feature analysis
new_line
text
  - Sparse autoencoders for disentangling features
  - Sparse autoencoders for disentangling features
new_line
text
• Key insights on model internals:
• Key insights on model internals:
new_line
text
  - Attention mechanisms (induction heads, copy suppression)
  - Attention mechanisms (induction heads, copy suppression)
new_line
text
  - FFN neuron behaviors 
  - FFN neuron behaviors 
new_line
text
  - Residual stream properties
  - Residual stream properties
new_line
text
  - Multi-component emergent behaviors
  - Multi-component emergent behaviors
new_line
new_line
text
The paper offers a unified notation and connects insights across different areas of interpretability research. It's a must-read for anyone working on understanding large language models!
The paper offers a unified notation and connects insights across different areas of interpretability research. It's a must-read for anyone working on understanding large language models!
new_line
new_line
text
Some fascinating technical highlights:
Some fascinating technical highlights:
new_line
text
- Detailed breakdowns of attention head circuits (e.g., IOI task)
- Detailed breakdowns of attention head circuits (e.g., IOI task)
new_line
text
- Analysis of factual recall mechanisms 
- Analysis of factual recall mechanisms 
new_line
text
- Overview of polysemanticity and superposition
- Overview of polysemanticity and superposition
new_line
text
- Discussion of grokking as circuit emergence
- Discussion of grokking as circuit emergence
new_line
new_line
text
What interpretability insights do you find most intriguing?
What interpretability insights do you find most intriguing?
Good folks from Universitat Politècnica de Catalunya, University of Groningen, and Meta have released "A Primer on the Inner Workings of Transformer-based Language Models."

They don't make survey papers like they used to, but this is an exciting new survey on Transformer LM interpretability!

This comprehensive survey provides a technical deep dive into:

• Transformer architecture components (attention, FFN, residual stream)
• Methods for localizing model behavior:
  - Input attribution (gradient & perturbation-based)
  - Component importance (logit attribution, causal interventions)
• Information decoding techniques:
  - Probing, linear feature analysis
  - Sparse autoencoders for disentangling features
• Key insights on model internals:
  - Attention mechanisms (induction heads, copy suppression)
  - FFN neuron behaviors 
  - Residual stream properties
  - Multi-component emergent behaviors

The paper offers a unified notation and connects insights across different areas of interpretability research. It's a must-read for anyone working on understanding large language models!

Some fascinating technical highlights:
- Detailed breakdowns of attention head circuits (e.g., IOI task)
- Analysis of factual recall mechanisms 
- Overview of polysemanticity and superposition
- Discussion of grokking as circuit emergence

What interpretability insights do you find most intriguing?
👀
John6666