text
How talkative is your chatbot about your internal data? 😬
How talkative is your chatbot about your internal data? 😬
new_line
new_line
text
As more chatbots get deployed in production, with access to internal databases, we need to make sure they don't leak private information to anyone interacting with them.
As more chatbots get deployed in production, with access to internal databases, we need to make sure they don't leak private information to anyone interacting with them.
new_line
new_line
text
The Lighthouz AI team therefore introduced the Chatbot Guardrails Arena to stress test models and see how well guarded your private information is.
The Lighthouz AI team therefore introduced the Chatbot Guardrails Arena to stress test models and see how well guarded your private information is.
new_line
text
Anyone can try to make models reveal information they should not share 😈
Anyone can try to make models reveal information they should not share 😈
new_line
text
(which is quite fun to do for the strongest models)!
(which is quite fun to do for the strongest models)!
new_line
new_line
text
The votes will then be gathered to create an Elo ranking of the safest models with respect to PII. 
The votes will then be gathered to create an Elo ranking of the safest models with respect to PII. 
new_line
new_line
text
In the future, with the support of the community, this arena could inform safety choices that company make, when choosing models and guardrails on their resistance to adversarial attacks.
In the future, with the support of the community, this arena could inform safety choices that company make, when choosing models and guardrails on their resistance to adversarial attacks.
new_line
text
It's also a good way to easily demonstrate the limitations of current systems!
It's also a good way to easily demonstrate the limitations of current systems!
new_line
new_line
text
Check out the arena: 
Check out the arena: 
resource
https://huggingface.co/spaces/lighthouzai/guardrails-arena
space
lighthouzai/guardrails-arena
https://huggingface.co/spaces/lighthouzai/guardrails-arena
new_line
text
Learn more in the blog: 
Learn more in the blog: 
link
https://huggingface.co/blog/arena-lighthouz
https://huggingface.co/blog/arena-lighthouz
How talkative is your chatbot about your internal data? 😬

As more chatbots get deployed in production, with access to internal databases, we need to make sure they don't leak private information to anyone interacting with them.

The Lighthouz AI team therefore introduced the Chatbot Guardrails Arena to stress test models and see how well guarded your private information is.
Anyone can try to make models reveal information they should not share 😈
(which is quite fun to do for the strongest models)!

The votes will then be gathered to create an Elo ranking of the safest models with respect to PII. 

In the future, with the support of the community, this arena could inform safety choices that company make, when choosing models and guardrails on their resistance to adversarial attacks.
It's also a good way to easily demonstrate the limitations of current systems!

Check out the arena: https://huggingface.co/spaces/lighthouzai/guardrails-arena
Learn more in the blog: https://huggingface.co/blog/arena-lighthouz
🚀
osanseviero m-ric zelus82 julien-c