Let $a, b \in \mathbb{R}^k$ be two normalized vectors such that $a^T b << 1$. Define matrix $C$ such that $[a, b, C]$ is full column rank, and let matrix $D$ be positive definite. Define projection matrix $P\_A:=A (A^T A)^{−1}A^T$. Can we say the following?


$$\frac{a^T D^{-1/2}\left(I - P\_{D^{-1/2}C}\right)D^{-1/2}b}{
\left\|\left(I - P\_{D^{-1/2}C}\right)D^{-1/2}b\right\|\_2
\left\|\left(I - P\_{D^{-1/2}C}\right)D^{-1/2}a\right\|\_2
}<< 1$$
where $<<$ is the little-$o$ notation.


Also [posted](https://math.stackexchange.com/q/2698493/195455) on MSE.



An inequality regarding projection
The answer is no. E.g., let $a=-[1,0,0]^T$, $b=[0,1,0]^T$, $D=I\_3$, and $C=[1,1,t]^T$ for $t>0$. Then $a^Tb=0<<1$ and the orthoprojector matrix onto the column space of $A=D^{-1/2}C$ is $P\_A=CC^T/(2+t^2)$, whereas your ratio is $\frac1{1+t^2}\to1$ as $t\to0$, which is not $<<1$. 


