I have two Gaussian random variables $X$ and $Y$, each of which is an estimator of an underlying quantity. I need to measure whether $Y$ is estimating something different than $X$. So if the mean of $Y$ is very different to the mean of $X$, that is good. However, if the variance of $Y$ is also much higher than the variance of $X$ it might just be that $Y$ is a poor estimator and the different mean is not meaningful (if you'll excuse the pun).


So for a fixed $X$, I need a function of $Y$ that increases with $\left|\mu\_Y-\mu\_X\right|$ but that decreases with $\sigma\_Y$ (and is zero if and only if $\sigma\_Y = \sigma\_X$ and $\mu\_Y = \mu\_X$). I don't really know whether to call this a measure, a divergence or something else, since it will be negative when $\mu\_Y = \mu\_X$ and $\sigma\_Y > \sigma\_X$.


I thought that the Kullback-Leibler divergence might be the way to go but it responds the wrong way to changes in the variance of $Y$. Also, it can't be negative.


Clearly, something like $\left|\mu\_Y-\mu\_X\right| + \sigma\_X - \sigma\_Y$ has the above properties but I'm looking for something a little more grounded in information theory.


Ideally, this would generalize to distributions other than Gaussian.



Divergence between two random variables
Let $0<\alpha<1$ (typically $\alpha=0.05$) and choose $\varepsilon>0$ such that


$$\alpha = \mathbb P(\left |X-\mu\_X\right | > \varepsilon)$$


Now let 


$$f = \mathbb P(\left |Y-\mu\_X\right | > \varepsilon) - \alpha$$


Then 


1. $f$ is increasing with $|\mu\_X-\mu\_Y|$,
2. $f$ is decreasing with $\sigma\_Y$, and
3. $f=0$ when $\mu\_X=\mu\_Y$ and $\sigma\_X=\sigma\_Y$.


This approach comes not so much from information theory as from hypothesis testing, in particular the notion of [statistical power](http://en.wikipedia.org/wiki/Statistical_power). Some generalizing from the case of the normal distribution seems possible, but note that in (3) we need the distribution to be determined by $\mu\_Y$ and $\sigma\_Y$.


