Diagnostic Measures for Model Criticism
Abstract We discuss the problem of model criticism, with emphasis on developing summary diagnostic measures. We approach model criticism by identifying possible troublesome features of the currently entertained model, embedding the model in an elaborated model, and measuring the value of elaborating. This requires three elements: a model elaboration, a prior distribution, and a utility function. Each triplet generates a different diagnostic measure. We focus primarily on the measure given by a Kullback—Leibler divergence between the marginal prior and posterior distributions on the elaboration parameter. We also develop a linearized version of this diagnostic and use it to show that our procedure is related to other tools commonly used for model diagnostics, such as Bayes factors and the score function. One attraction of this approach is that it allows model criticism to be performed jointly with parameter inference and prediction. Also, this diagnostic approach aims at maintaining an exploratory nature t...