I asked this [question on stats.stackexchange](https://stats.stackexchange.com/questions/188371/negative-population-variable-importance) and even elsewhere, but it never received an answer. 


I just state the probabilistic problem here. It is about the optimality of the conditional expectation. 


Consider a random vector $(X\_1,X\_2,X\_3,Y)$ with $Y\in L^1$, and let $f$ be the Borelian function such that $E[Y\mid X\_1, X\_2, X\_3]=f(X\_1, X\_2, X\_3)$.


Define 
$$I = E\left[{\bigl(Y-f(X'\_1,X\_2,X\_3)\bigr)}^2\right] - E\left[{\bigl(Y-f(X\_1,X\_2,X\_3)\bigr)}^2\right]$$
where $X'\_1$ is a random variable having the same distribution as $X\_1$ but is independent of all other random variables $X\_2,X\_3,Y$. 


***The question: "is it possible that $I < 0$?"*** 


In case you are interested in a simple example, the paper [Correlation and variable importance in random forests (Gregorutti & al)](http://arxiv.org/abs/1310.5726) 
investigates some particular cases such as the additive case $f(X\_1,X\_2,X\_3)=f\_1(X\_1)+f\_2(X\_2)+f\_3(X\_3)$. It is not difficult to get that $I=2\textrm{Var}\bigl(f\_1(X\_1)\bigr)$ in this case. 



Negative population variable importance
That is not possible: $I$ cannot be negative. 


Indeed, let $X:=X\_1$, $X':=X'\_1$, $V:=(X\_2,X\_3)$. Let $E\_Z$ denote the conditional expectation given a random variable/random vector $Z$. Then $E\_{(X',V)}Y=E\_V Y$, since $X'$ is independent of $(Y,V)$ (see details below). So,
$$E(Y-f(X',V))^2\ge E(Y-E\_{(X',V)}Y)^2=E(Y-E\_V Y)^2
\ge E(Y-E\_{(X,V)} Y)^2=E(Y-f(X,V))^2. 
$$
The two inequalities above hold because $E(Y-g(Z))^2\ge E(Y-E\_Z Y)^2$ for any Borel function $g$. 


**Details on the equality $E\_{(X',V)}Y=E\_V Y$:** Let $h(V):=E\_V Y$. Let $I\{\cdot\}$ denote the indicator function. Let $A$ and $B$ denote Borel sets in appropriate spaces. 
Then, by the independence of $X'$ from $(Y,V)$, 
$$EYI\{X'\in A\}I\{V\in B\}=EI\{X'\in A\}\,EYI\{V\in B\}$$
$$=EI\{X'\in A\}\,Eh(V)I\{V\in B\}
=Eh(V)I\{X'\in A\}I\{V\in B\}, 
$$
whence $E\_{(X',V)}Y=h(V)=E\_V Y$. 


