Explainable ai
214265509
From local explanations to global understanding with explainable AI for trees
Tree based machine learning models such as random forests, decision trees and gradient boosted trees are popular nonlinear predictive models, yet comparatively little attention has been paid to explaining their predictions. Here we improve the interpretability of tree based models through three main contributions. (1) A polynomial time algorithm to compute optimal explanations based on game theory. (2) A new type of explanation that directly measures local feature interaction effects. (3) A new set of tools for understanding global model structure based on combining many local explanations of each prediction. We apply these tools to three medical machine learning problems and show how combining many high quality local explanations allows us to represent global structure while retaining local faithfulness to the original model. These tools enable us to (1) identify high magnitude but low frequency nonlinear mortality risk factors in the US population, (2) highlight distinct population subgroups with shared risk characteristics, (3) identify nonlinear interaction effects among risk factors for chronic kidney disease and (4) monitor a machine learning model deployed in a hospital by identifying which features are degrading the model's performance over time. Given the popularity of tree based machine learning models, these improvements to their interpretability have implications across a broad set of domains. Tree based machine learning models are widely used in domains such as healthcare, finance and public services. The authors present an explanation method for trees that enables the computation of optimal local explanations for individual predictions, and demonstrate their method on three medical datasets.
Nature machine intelligence
Scott M Lundberg Gabriel G Erion Hugh Chen Alex J DeGrave Jordan M Prutkin Bala G Nair Ronit Katz Jonathan Himmelfarb Nisha Bansal Su-In Lee
210064344
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
A surge of interest in explainable AI (XAI) has led to a vast collection of algorithmic work on the topic. While many recognize the necessity to incorporate explainability features in AI systems, how to address real world user needs for understanding AI remains an open question. By interviewing 20 UX and design practitioners working on various AI products, we seek to identify gaps between the current XAI algorithmic work and practices to create explainable AI products. To do so, we develop an algorithm informed XAI question bank in which user needs for explainability are represented as prototypical questions users might ask about the AI, and use it as a study probe. Our work contributes insights into the design space of XAI, informs efforts to support design practices in this space, and identifies opportunities for future XAI work. We also provide an extended XAI question bank and discuss how it can be used for creating user centered XAI.
CHI
Qingzi Vera Liao Dan Gruen Sarah Miller
218971737
Good Counterfactuals and Where to Find Them: A Case Based Technique for Generating Counterfactuals for Explainable AI (XAI)
Recently, a groundswell of research has identified the use of counterfactual explanations as a potentially significant solution to the Explainable AI (XAI) problem. It is argued that (a) technically, these counterfactual cases can be generated by permuting problem features until a class change is found, (b) psychologically, they are much more causally informative than factual explanations, (c) legally, they are GDPR compliant. However, there are issues around the finding of good counterfactuals using current techniques (e.g. sparsity and plausibility) We show that many commonly used datasets appear to have few good counterfactuals for explanation purposes. So, we propose a new case based approach for generating counterfactuals using novel ideas about the counterfactual potential and explanatory coverage of a case base. The new technique reuses patterns of good counterfactuals, present in a case base, to generate analogous counterfactuals that can explain new problems and their solutions. Several experiments show how this technique can improve the counterfactual potential and explanatory coverage of case bases that were previously found wanting.
ICCBR
Mark T Keane Barry Smyth
210837324
Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Explainable artificially intelligent (XAI) systems form part of sociotechnical systems, e.g. human+AI teams tasked with making decisions. Yet, current XAI systems are rarely evaluated by measuring the performance of human+AI teams on actual decision making tasks. We conducted two online experiments and one in person think aloud study to evaluate two currently common techniques for evaluating XAI systems: (1) using proxy, artificial tasks such as how well humans predict the AI's decision from the given explanations, and (2) using subjective measures of trust and preference as predictors of actual performance. The results of our experiments demonstrate that evaluations with proxy tasks did not predict the results of the evaluations with the actual decision making tasks. Further, the subjective measures on evaluations with actual decision making tasks did not predict the objective performance on those same tasks. Our results suggest that by employing misleading evaluation methods, our field may be inadvertently slowing its progress toward developing human+AI teams that can reliably perform better than humans or AIs alone.
IUI
Zana Buccinca Phoebe Lin Krzysztof Z Gajos Elena L Glassman
204950036
Feature relevance quantification in explainable AI: A causality problem
We discuss promising recent contributions on quantifying feature relevance using Shapley values, where we observed some confusion on which probability distribution is the right one for dropped features. We argue that the confusion is based on not carefully distinguishing between observational and interventional conditional probabilities and try a clarification based on Pearl's seminal work on causality. We conclude that unconditional rather than conditional expectations provide the right notion of dropping features in contradiction to the theoretical justification of the software package SHAP. Parts of SHAP are unaffected because unconditional expectations (which we argue to be conceptually right) are used as approximation for the conditional ones, which encouraged others to `improve' SHAP in a way that we believe to be flawed.
AISTATS
Dominik Janzing Lenon Minorics Patrick Blobaum
211506204
The Emerging Landscape of Explainable AI Planning and Decision Making
In this paper, we provide a comprehensive outline of the different threads of work in Explainable AI Planning (XAIP) that has emerged as a focus area in the last couple of years and contrast that with earlier efforts in the field in terms of techniques, target users, and delivery mechanisms. We hope that the survey will provide guidance to new researchers in automated planning towards the role of explanations in the effective design of human in the loop systems, as well as provide the established researcher with some perspective on the evolution of the exciting world of explainable planning.
ArXiv
Tathagata Chakraborti Sarath Sreedharan S Kambhampati
218502350
Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?
Algorithmic approaches to interpreting machine learning models have proliferated in recent years. We carry out human subject tests that are the first of their kind to isolate the effect of algorithmic explanations on a key aspect of model interpretability, simulatability, while avoiding important confounding experimental factors. A model is simulatable when a person can predict its behavior on new inputs. Through two kinds of simulation tests involving text and tabular data, we evaluate five explanations methods: (1) LIME, (2) Anchor, (3) Decision Boundary, (4) a Prototype model, and (5) a Composite approach that combines explanations from each method. Clear evidence of method effectiveness is found in very few cases: LIME improves simulatability in tabular classification, and our Prototype method is effective in counterfactual simulation tests. We also collect subjective ratings of explanations, but we do not find that ratings are predictive of how helpful explanations are. Our results provide the first reliable and comprehensive estimates of how explanations influence simulatability across a variety of explanation methods and data domains. We show that (1) we need to be careful about the metrics we use to evaluate explanation methods, and (2) there is significant room for improvement in current methods.
ACL
Peter Hase Mohit Bansal
222125099
A Survey of the State of Explainable AI for Natural Language Processing
Recent years have seen important advances in the quality of state of the art models, but this has come at the expense of models becoming less interpretable. This survey presents an overview of the current state of Explainable AI (XAI) considered within the domain of Natural Language Processing (NLP) We discuss the main categorization of explanations, as well as the various ways explanations can be arrived at and visualized. We detail the operations and explainability techniques currently available for generating explanations for NLP model predictions, to serve as a resource for model developers in the community. Finally, we point out the current gaps and encourage directions for future work in this important research area.
AACL
Marina Danilevsky Kun Qian Ranit Aharonov Yannis Katsis Ban Kawas Prithviraj Sen
220734948
Explainable AI and Mass Surveillance System Based Healthcare Framework to Combat COVID I9 Like Pandemics
Tactile edge technology that focuses on 5G or beyond 5G reveals an exciting approach to control infectious diseases such as COVID 19 internationally. The control of epidemics such as COVID 19 can be managed effectively by exploiting edge computation through the 5G wireless connectivity network. The implementation of a hierarchical edge computing system provides many advantages, such as low latency, scalability, and the protection of application and training model data, enabling COVID 19 to be evaluated by a dependable local edge server. In addition, many deep learning (DL) algorithms suffer from two crucial disadvantages: first, training requires a large COVID 19 dataset consisting of various aspects, which will pose challenges for local councils; second, to acknowledge the outcome, the findings of deep learning require ethical acceptance and clarification by the health care sector, as well as other contributors. In this article, we propose a B5G framework that utilizes the 5G network's low latency, high bandwidth functionality to detect COVID 19 using chest X ray or CT scan images, and to develop a mass surveillance system to monitor social distancing, mask wearing, and body temperature. Three DL models, ResNet50, Deep tree, and Inception v3, are investigated in the proposed framework. Furthermore, blockchain technology is also used to ensure the security of healthcare data.
IEEE Network
M Shamim Hossain Muhammad Ghulam Nadra Guizani
211021069
Human centered Explainable AI: Towards a Reflective Sociotechnical Approach
Explanations a form of post hoc interpretability play an instrumental role in making systems accessible as AI continues to proliferate complex and sensitive sociotechnical systems. In this paper, we introduce Human centered Explainable AI (HCXAI) as an approach that puts the human at the center of technology design. It develops a holistic understanding of "who" the human is by considering the interplay of values, interpersonal dynamics, and the socially situated nature of AI systems. In particular, we advocate for a reflective sociotechnical approach. We illustrate HCXAI through a case study of an explanation system for non technical end users that shows how technical advancements and the understanding of human factors co evolve. Building on the case study, we lay out open research questions pertaining to further refining our understanding of "who" the human is and extending beyond 1 to 1 human computer interactions. Finally, we propose that a reflective HCXAI paradigm mediated through the perspective of Critical Technical Practice and supplemented with strategies from HCI, such as value sensitive design and participatory design not only helps us understand our intellectual blind spots, but it can also open up new design and research spaces.
HCI
Upol Ehsan Mark O Riedl