text
Mixture of experts: beware 🛡️⚔️
Mixture of experts: beware 🛡️⚔️
new_line
new_line
text
New paper by DeepMind: Buffer Overflow in MoE 
New paper by DeepMind: Buffer Overflow in MoE 
resource
https://huggingface.co/papers/2402.05526
paper
2402.05526
https://huggingface.co/papers/2402.05526
Buffer Overflow in Mixture of Experts (2402.05526)
new_line
new_line
text
The paper shows an adversarial attack strategy in which a user sends malicious queries that can affect the output of other user queries from the same batch. 
The paper shows an adversarial attack strategy in which a user sends malicious queries that can affect the output of other user queries from the same batch. 
new_line
new_line
text
So if in the same batch we have 
So if in the same batch we have 
new_line
text
- User A benign query
- User A benign query
new_line
text
- User B malicious query
- User B malicious query
new_line
text
The response for A might be altered!😱
The response for A might be altered!😱
new_line
new_line
text
How is this possible?
How is this possible?
new_line
text
One approach is to fill the token buffers with adversarial data, hence forcing the gating to use the non-ideal experts or to entirely drop the bening tokens (in the case of finite limit size). 
One approach is to fill the token buffers with adversarial data, hence forcing the gating to use the non-ideal experts or to entirely drop the bening tokens (in the case of finite limit size). 
new_line
new_line
text
This assumes that the adversary can use the model as a black-box but can observe the logit outputs + ensure that the data is always grouped in the same batch. 
This assumes that the adversary can use the model as a black-box but can observe the logit outputs + ensure that the data is always grouped in the same batch. 
new_line
new_line
text
How to mitigate this?
How to mitigate this?
new_line
text
- Randomize batch order (and even run twice if some queries are very sensitive)
- Randomize batch order (and even run twice if some queries are very sensitive)
new_line
text
- Use a large capacity slack
- Use a large capacity slack
new_line
text
- Sample from gate weights instead of top-k (not great IMO, as that require more memory for inference)
- Sample from gate weights instead of top-k (not great IMO, as that require more memory for inference)
new_line
new_line
text
Very cool paper!!
Very cool paper!!
Mixture of experts: beware 🛡️⚔️

New paper by DeepMind: Buffer Overflow in MoE https://huggingface.co/papers/2402.05526

The paper shows an adversarial attack strategy in which a user sends malicious queries that can affect the output of other user queries from the same batch. 

So if in the same batch we have 
- User A benign query
- User B malicious query
The response for A might be altered!😱

How is this possible?
One approach is to fill the token buffers with adversarial data, hence forcing the gating to use the non-ideal experts or to entirely drop the bening tokens (in the case of finite limit size). 

This assumes that the adversary can use the model as a black-box but can observe the logit outputs + ensure that the data is always grouped in the same batch. 

How to mitigate this?
- Randomize batch order (and even run twice if some queries are very sensitive)
- Use a large capacity slack
- Sample from gate weights instead of top-k (not great IMO, as that require more memory for inference)

Very cool paper!!
👍
NickyNicky julien-c macadeliccc ivanfioravanti clem samusenps victor JosephGesnouin pratjz blancsw marinaretik juancopi81 Dlbk djmcflush EmelMk surfiend Yasbok 0xe69756
❤️
macadeliccc clem samusenps JosephGesnouin blancsw santiviquez Dlbk abunchofrandomwords 0xe69756
🤯
meefs blancsw davanstrien Dlbk Yhyu13 Epiculous
🤝
Dlbk Epiculous
/avatars/19f9eeb9281b47b34f68c312092ca468.svg
Yu
Yhyu13
user
https://cdn-avatars.huggingface.co/v1/production/uploads/64adfd277b5ff762771e4571/HVmAmGOtBPYd72492XGar.png
Epiculous
Epiculous
user