Suppose $X, U \in \mathbb{R}^{n \times r}$, $n>r$, where $U$ is a fixed matrix and $X$ is a variable, and both are of full column ranks. Let $\mathfrak{R} = \{ \Psi \in \mathbb{R}^{r \times r}: \Psi \Psi^\top = \Psi^\top \Psi = I\_r \}$ be the set of rotation matrices in dimension $r$. Also suppose there exists constants $c\_1,c\_2 >0$ such that $X$ satisifes:


(a) $\|X\|\_2 > c\_1$: this implies that $X$ is away from the zero matrix.


(b) $\min\_{\Psi \in \mathfrak{R}} \|X-U \Psi\|\_F > c\_2$: this implies that $X$ is away from $U$ (after the best possible rotation).


$\textbf{Question}$:


Given condition (a) and (b), can we find a constant $c\_3 > 0$ (probably in terms of $c\_1$ and $c\_2$) such that the following holds?
\begin{align}
&\|(XX^\top - UU^\top)X\|\_F > c\_3 
\end{align}


Remark: We have from (b) that $\|XX^\top - UU^\top \|\_F > c\_4$. You may use this instead of (b). You may also use other norms/singular value conditions in (a) and (b) if that helps.



lower bound on the norm of (correlated) matrix multiplication
No, there is no lower bound. Take for $\epsilon\neq 0$ arbitrarily small:
$$ X=\left( \begin{matrix} 1 & 0 \\ 0 & \epsilon \\ 0 & 0\end{matrix} \right) \ \ \mbox{and} \ \ U=\left( \begin{matrix} 1 & 0 \\ 0 & 1 \\ 0 & 0\end{matrix} \right) .$$
However, if you add a condition on $X^T X$ having a uniformly bounded inverse then your claim is correct.


Let $f(X)=\|(X X^T-U U^T)X\|\_F$. 
If we add the constraint 
$\|(X^T X)^{-1}\|\leq M<+\infty$ to the others then we claim that $f$
will have a non-zero lower bound. Suppose the contrary. 
Then we may find a sequence $X\_n$ satisfying the
criteria and such that $f(X\_n) \rightarrow 0$.
It is easy to see that $X\_n$ must be bounded. Using compactness
we may assume that $X\_n$ converges to $X$ which therefore verifies:
$$ (X X^T-U U^T)X = 0 .$$ 
Because of our auxilary condition
 $X^T X$ is invertible so
$$ X = U \left(U^T X 
(X^T X)^{-1} \right) \equiv U A$$
with $A$ an $r\times r$ matrix.
Both $X$ and $U$ have full rank
so $A$ is invertible and then 
$$ U (A A^T-I) U^T X= 0 $$
from which (using again full rank) $I=A A^T$ which contradicts the
condition regarding rotations.


