text
🔍 Today's pick in Interpretability & Analysis of LMs: Black-Box Access is Insufficient for Rigorous AI Audits by 
🔍 Today's pick in Interpretability & Analysis of LMs: Black-Box Access is Insufficient for Rigorous AI Audits by 
mention
@stecas
stecas
text
mention
@carsonezell
carsonezell
text
 et al.
 et al.
new_line
new_line
text
Audits conducted on AI systems can identify potential risks and ensure their compliance to safety requirements. Authors categorise audits based on the access to model-related resources (black, grey, white and out-of-the box) and highlight how levels of transparency on audited AI system enable broader and more effective auditing procedures. Technical, physical, and legal safeguards for performing audits are also introduced to ensure minimal security risks for audited companies. Authors conclude that transparency on the type of auditors’ access and methods is a pre-requisite to correctly interpret audit results, and white- and outside-the-box access allow for substantially more scrutiny than black-box access alone.
Audits conducted on AI systems can identify potential risks and ensure their compliance to safety requirements. Authors categorise audits based on the access to model-related resources (black, grey, white and out-of-the box) and highlight how levels of transparency on audited AI system enable broader and more effective auditing procedures. Technical, physical, and legal safeguards for performing audits are also introduced to ensure minimal security risks for audited companies. Authors conclude that transparency on the type of auditors’ access and methods is a pre-requisite to correctly interpret audit results, and white- and outside-the-box access allow for substantially more scrutiny than black-box access alone.
new_line
new_line
text
📄 Paper: 
📄 Paper: 
resource
https://huggingface.co/papers/2401.14446
paper
2401.14446
https://huggingface.co/papers/2401.14446
Black-Box Access is Insufficient for Rigorous AI Audits (2401.14446)
new_line
new_line
text
🔍 Further readings:
🔍 Further readings:
new_line
new_line
text
📄Taxonomy of AI system access: 
📄Taxonomy of AI system access: 
link
https://bit.ly/struct-access
https://bit.ly/struct-access
new_line
text
💻An API for transparent science on Black-box AI (NNsight): 
💻An API for transparent science on Black-box AI (NNsight): 
link
https://nnsight.net/about
https://nnsight.net/about
🔍 Today's pick in Interpretability & Analysis of LMs: Black-Box Access is Insufficient for Rigorous AI Audits by @stecas @carsonezell et al.

Audits conducted on AI systems can identify potential risks and ensure their compliance to safety requirements. Authors categorise audits based on the access to model-related resources (black, grey, white and out-of-the box) and highlight how levels of transparency on audited AI system enable broader and more effective auditing procedures. Technical, physical, and legal safeguards for performing audits are also introduced to ensure minimal security risks for audited companies. Authors conclude that transparency on the type of auditors’ access and methods is a pre-requisite to correctly interpret audit results, and white- and outside-the-box access allow for substantially more scrutiny than black-box access alone.

📄 Paper: https://huggingface.co/papers/2401.14446

🔍 Further readings:

📄Taxonomy of AI system access: https://bit.ly/struct-access
💻An API for transparent science on Black-box AI (NNsight): https://nnsight.net/about
❤️
clem skrishna osanseviero merve stecas
👍
dhuynh95 skrishna osanseviero stecas