PERFEX: Classifier Performance Explanations for Trustworthy AI Systems

Explainability of a classiﬁcation model is crucial when de- ployed in real-world decision support systems. Explanations make predictions actionable to the user and should inform about the capabilities and limitations of the system. Existing explanation methods, however, typically only provide expla- nations for individual predictions. Information about conditions under which the classiﬁer is able to support the deci- sion maker is not available, while for instance information about when the system is not able to differentiate classes can be very helpful. In the development phase it can support the search for new features or combining models, and in the oper- ational phase it supports decision makers in deciding e.g. not to use the system. This paper presents a method to explain the qualities of a trained base classiﬁer, called PERFormance EXplainer (PERFEX). Our method consists of a meta tree learning algorithm that is able to predict and explain under which conditions the base classiﬁer has a high or low error or any other classiﬁcation performance metric. We evaluate PERFEX using several classiﬁers and datasets, including a case study with urban mobility data. It turns out that PERFEX typically has high meta prediction performance even if the base classiﬁer is hardly able to differentiate classes, while giving compact performance explanations.