text
💥 Today's pick in Interpretability & Analysis of LMs: 🩺 Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models by 
💥 Today's pick in Interpretability & Analysis of LMs: 🩺 Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models by 
mention
@asmadotgh
asmadotgh
text
, 
, 
mention
@codevan
codevan
text
, 
, 
mention
@1wheel
1wheel
text
, 
, 
mention
@iislucas
iislucas
text
 & 
 & 
mention
@mega
mega
new_line
new_line
text
Patchscopes is a generalized framework for verbalizing information contained in LM representations. This is achieved via a mid-forward patching operation inserting the information into an ad-hoc prompt aimed at eliciting model knowledge. Patchscope instances for vocabulary projection, feature extraction and entity resolution in model representation are show to outperform popular interpretability approaches, often resulting in more robust and expressive information.
Patchscopes is a generalized framework for verbalizing information contained in LM representations. This is achieved via a mid-forward patching operation inserting the information into an ad-hoc prompt aimed at eliciting model knowledge. Patchscope instances for vocabulary projection, feature extraction and entity resolution in model representation are show to outperform popular interpretability approaches, often resulting in more robust and expressive information.
new_line
new_line
text
🌐 Website: 
🌐 Website: 
link
https://pair-code.github.io/interpretability/patchscopes/
https://pair-code.github.io/interpretability/patchscopes/
new_line
text
📄 Paper: 
📄 Paper: 
resource
https://huggingface.co/papers/2401.06102
paper
2401.06102
https://huggingface.co/papers/2401.06102
Patchscope: A Unifying Framework for Inspecting Hidden Representations
  of Language Models (2401.06102)
💥 Today's pick in Interpretability & Analysis of LMs: 🩺 Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models by @asmadotgh, @codevan, @1wheel, @iislucas & @mega

Patchscopes is a generalized framework for verbalizing information contained in LM representations. This is achieved via a mid-forward patching operation inserting the information into an ad-hoc prompt aimed at eliciting model knowledge. Patchscope instances for vocabulary projection, feature extraction and entity resolution in model representation are show to outperform popular interpretability approaches, often resulting in more robust and expressive information.

🌐 Website: https://pair-code.github.io/interpretability/patchscopes/
📄 Paper: https://huggingface.co/papers/2401.06102
🤗
s3nh osanseviero victor codevan iislucas asmadotgh merve Kernel samusenps anjutv clem Dlbk