Let $X = [x\_1 \cdots x\_N] \in \mathbb{R}^{d \times N}$ and $ T= [t\_1 \cdots t\_N] \in \mathbb{R}^{1 \times N}$. Define
$$
E(w) = \text{tr}(T - w^TX)(T - w^TX)^T 
$$
as least square energy. When defining 
$$
\epsilon = T- w\_{LS}^T X,
$$
with $w\_{LS}^T$ being the solution that minimizes $E$, can we find a simple bound for $\epsilon$ in terms of $X$ and $T$ without involving inverse operation? And additionally, can I get a bound (needs to be as small as possible) for each $\epsilon\_i$ whereas $\epsilon = [\epsilon\_1 \cdots \epsilon\_N]$?


In addition to the above, regarding the vectors mentioned above as random ones, would there be any way to impose more conditions on the above to derive that 
$$
\epsilon\_i \sim \mathcal{N}(0,I/\sigma^2) 
$$
with $0<\sigma<<1$.



error bound for least square minimization
$\epsilon$ is the orthogonal projection of $T$ onto the orthogonal complement of the space spanned by the rows of $X$. Hence we have
$$
\|\epsilon\|\_\infty=\frac{\|\epsilon\|\_\infty}{\|T\|\_\infty}\|T\|\_\infty \leq \sup\_{(u,v)=0,u,v\neq 0} \frac{\|u\|\_\infty}{\|u+v\|\_\infty} \|T\|\_\infty\leq \frac{\sqrt{N}+1}{2}\|T\|\_\infty.
$$ 
Here we have used that the supremum is attained at


$u=\left(\frac{\sqrt{N}+1}{2},\frac{1}{2},\dots,\frac{1}{2}\right)$ and $v=\left(\frac{-\sqrt{N}+1}{2},\frac{1}{2},\dots,\frac{1}{2}\right)$.


Regarding the probability distribution: if $X$ is fix and the entries of $T$ are independent and have standard distribution $t\_1,\dots,t\_N$ then the covariance matrix of the $\epsilon's$ is given by $C\_{ij}=(I-X^T(XX^T)^{-1}X)\_{ij}t\_it\_j$. You can derive this from the explicit formula $\epsilon=T\left(I-X^T(XX^T)^{-1}X \right)$.


