Given a linear system $Ax=b$, the pseudoinverse of $A$ is found as the matrix $A^+$ such that $x=A^+ b$ where $x$ solves the least squares problem $\min \| Ax - b \|^2 $ and $x \perp \mathcal{N}(A)$. That is, $x$ is the shortest vector in the solution space.


That is, find $x$ :
\begin{eqnarray}
\min \| x \| \text{ such that } x \text{ minimizes } \| Ax - b \|^2 .
\end{eqnarray}


This is similar to the regularization problem of minimizing
\begin{eqnarray}
 \| Ax - b \|^2 + \lambda \| x \|^2
\end{eqnarray}
I do not quite can get from one to the other? Are these two things equivalent?
Is there a way to post this as the same problem? Is there a connection?



Is the pseudoinverse the same as least squares with regularization?
Here's one way to see this directly. I will assume that $A$ is $m \times n$. Let
$$ A = U \Sigma V^T$$
be the SVD for A. Recall that the regularized solution to the least squares problem $Ax = b$ is given by
$$\hat{x} = (A^TA + \lambda I)^{-1}A^Tb.$$
Now substitute the SVD of $A$ in place of $A$. Simplifying algebra, we get that
$$\hat{x} = V(\Sigma^T \Sigma + \lambda I)^{-1} \Sigma^T U^T b. \hspace{1cm} (\ast) $$
OTOH, the solution by considering the pseudoinverse is given by
$$ A^+b = V\Sigma^+ U^Tb.\hspace{3cm} (\ast\ast) $$
Therefore, $(\ast)$ and $(\ast \ast)$ are equivalent (in the limit) if
$$\Sigma^+ = \lim\_{\lambda \to 0} (\Sigma^T \Sigma + \lambda I)^{-1} \Sigma^T.$$


Without referring to the arxiv preprint in G. Fougeron's answer, let us show this as follows. Let $\sigma\_1,\ldots, \sigma\_r$ be the singular values of $A$. By direct computation, we find that $\Sigma^T \Sigma + \lambda I $ is the $n \times n$ diagonal matrix given by
$$ \Sigma^T \Sigma + \lambda I = \begin{pmatrix} \sigma\_1^2 + \lambda & \\ 
 & \ddots \\
&& \sigma\_r^2 + \lambda \\
&&& \lambda \\
&&&& \ddots \\
&&&&&\lambda \end{pmatrix}$$


Since this is diagonal, it is easy to invert and therefore $ (\Sigma^T \Sigma + \lambda I)^{-1}\Sigma^T$ is given explicitly as the $n \times m$ matrix


$$(\Sigma^T \Sigma + \lambda I)^{-1}\Sigma^T = \left(\begin{array}{ccc|cc} \frac{\sigma\_1}{\sigma\_1^2 + \lambda} & \\ 
 & \ddots &&&0 \\
&& \frac{\sigma\_r}{\sigma\_r^2 + \lambda} \\
\hline
&&& \\
&0&&& 0 \\
&&&&& \end{array}\right). $$
with top-left block a digonal $r \times r$ matrix. In the limit as $\lambda \to 0$, the top-left block is just
$$\begin{pmatrix} \frac{1}{\sigma\_1} \\
&\ddots\\
&& \frac{1}{\sigma\_r} \end{pmatrix} $$ and hence the entire matrix is just $\Sigma^+$ as desired.


