Theoretical Foundations of Equitability and the Maximal Information Coefficient
The maximal information coecient (MIC) is a tool for nding the strongest pairwise relationships in a data set with many variables [1]. MIC is useful because it gives similar scores to equally noisy relationships of dierent types. This property, called equitability, is important for analyzing high-dimensional data sets. Here we formalize the theory behind both equitability and MIC in the language of estimation theory. This formalization has a number of advantages. First, it allows us to show that equitability is a generalization of power against statistical independence. Second, it allows us to compute and discuss the population value of MIC, which we call MIC . In doing so we generalize and strengthen the mathematical results proven in [1] and clarify the relationship between MIC and mutual information. Introducing MIC also enables us to reason about the properties of MIC more abstractly: for instance, we show that MIC is continuous and that there is a sense in which it is a canonical \smoothing" of mutual information. We also prove an alternate, equivalent characterization of MIC that we use to state new estimators of it as well as an algorithm for explicitly computing it when the joint probability density function of a pair of random variables is known. Our hope is that this paper provides a richer theoretical foundation for MIC and equitability going forward. This paper will be accompanied by a forthcoming companion paper that performs extensive empirical analysis and comparison to other methods and discusses the practical aspects of both equitability and the use of MIC and its related statistics.