Hi, I know they are related questions on the board but mine is more specific. Although the answer for any non-singular matrix would be also interesting. Thanks!


UPDATE: I am sorry I though this was clear, but as I know it the $||\cdot||\_{2}$ norm is defined as follow: Let be $A\in \mathbb{R}^{m\times n};\;||A||\_{2} = \displaystyle{max\_{x \in \mathbb{R}^n} \frac{||Ax||\_{2}}{||x||\_{2}}}$



How to compute the induced $||\cdot||_{2} $ matrix norm of an SPD matrix
To lay the question to rest, let me do two things: (i) restate it; (ii) answer it.


By $\|x\|$, we mean the Euclidean 2-norm throughout.



> 
> Show that the *induced* 2-norm $$\max\_{\|x\|\not= 0} \frac{\|Ax\|}{\|x\|}$$ is given by $\sqrt{\lambda\_{\max}(A^TA)}$
> 
> 
> 


The proof is textbook material. For the lazy, here is an informal sketch.


Notice that since without loss of generality, we may rescale vector $x$, hence we may equivalently consider maximizing $\|Ax\|$ such that $\|x\|=1$.


Consider, $\|Ax\|^2 = x^TA^TAx$. The matrix $A^TA$ is SPD, so it has the eigendecomposition $V\Lambda V^T$, where $\Lambda$ is a nonnegative diagonal matrix. Thus, we have $x^TA^TAx = x^TV\Lambda V^Tx = y^T\Lambda y = \sum\_i \lambda\_i y\_i^2$. This, implies that $\|Ax\|^2 \le \lambda\_{\max}y^Ty = \lambda\_{\max}x^TV^TVx=\lambda\_{\max}$ because $V^TV=I$ and $x^Tx=1$.


To conclude the proof we now need to show that in fact $\|Ax\|^2 = \lambda\_{\max}$. But this is trivial, because picking $x=v\_{\max}$ (eigenvector corr to max eigenvalue), we attain this equality.


PS: Other proofs based on Lagrange multipliers etc. can also be given, but ultimately one needs to invoke something like $A^TAx=\lambda x$ at some point.


