Suppose that *X* and *Z* are matrices with the same number of rows. Let
$$ D = \left[\begin{array}{cc} X' X & X'Z \\ Z'X & Z'Z \end{array} \right]^{-1} - \left[\begin{array}{cc} (X' X)^{-1} & 0 \\ 0 & 0 \end{array} \right],$$ where all inverses are assumed to exist and the zeros represent zero matrices of suitable dimensions. How can we prove that $D$ is positive semidefinite?



Positive semidefinite ordering for covariance matrices
Let 
$$A:=X' X,\quad B:=X'Z,$$  

\begin{equation}
 \left[\begin{array}{cc} U & V \\ V' & T \end{array} \right]:=
 \left[\begin{array}{cc} X' X & X'Z \\ Z'X & Z'Z \end{array} \right]^{-1}.
\end{equation}
Then $AU+BV'=I$, $AV+BT=0$, whence 
\begin{equation}
 V=-A^{-1}BT,\quad U=A^{-1}+A^{-1}BTB'A^{-1}, 
\end{equation}
\begin{equation}
 D=\left[\begin{array}{cc} A^{-1}BTB'A^{-1} & -A^{-1}BT \\ -(A^{-1}BT)' & T \end{array} \right]. 
\end{equation}
Also, $T$ is positive definite, as a diagonal block of the positive definite matrix $\left[\begin{array}{cc} X' X & X'Z \\ Z'X & Z'Z \end{array} \right]^{-1}$. 
Therefore, for any column matrix $\left[\begin{array}{cc} x \\ z \end{array} \right]$ with sub-columns $x$ and $z$ of appropriate heights, straightforward calculations yield
\begin{equation}
 \left[\begin{array}{cc} x \\ z \end{array} \right]'D\left[\begin{array}{cc} x \\ z \end{array} \right]
 =(w-z)'T(w-z)\ge0, 
\end{equation}
where 
$w:=B'A^{-1}x$. 
Thus, $D$ is indeed nonnegative definite. 


