Let $X,Y$ be real separable Hilbert space, and let $HS(X,Y)$ be the space of Hilbert-Schmidt operators from $X$ to $Y$, endowed with the [Hilbert-Schmidt norm](https://en.wikipedia.org/wiki/Hilbert%E2%80%93Schmidt_operator). Let $x\in X$ and $y\in Y$. I am interested in finding the derivative of the map
$$
HS(X,Y)\ni A\mapsto \|y-Ax\|\_Y^2\in \mathbb{R}.
$$


As $HS(X,Y)$ is a Hilbert space, the derivative (as a bounded linear operator from $HS(X,Y)$ to $\mathbb{R}$) can be identified as an element in $HS(X,Y)$. Is there an easy expression of it?



Derivative with respect to a Hilbert-Schmidt operator
Let me denote by $F : HS(X,Y) \to \mathbb{R}$ the map which you defined as $F(A) := \|y-Ax\|\_Y^2$. By standard arguments, you indeed have that $F$ is $C^1$ and, for any $A \in HS(X,Y)$,
$$
\mathcal{L}(HS(X,Y),\mathbb{R}) \ni DF\_{\rvert A} = \begin{cases}
HS(X,Y) \to \mathbb{R}, \\
B \mapsto 2 \langle Bx , Ax - y \rangle\_Y.
\end{cases}
$$
As you recall, $HS(X,Y)$ is a Hilbert space for the scalar product
$$
\langle A, B \rangle\_{HS} := \sum\_{i \in I} \langle A e\_i, B e\_i \rangle\_Y
$$
for any orthonormal basis $(e\_i)\_{i \in I}$ of $X$.


Given $A \in HS(X,Y)$, by the Riesz representation theorem, we look for some $C \in HS(X,Y)$ such that $DF\_{\rvert A} = \langle C, \cdot \rangle\_{HS}$. Thus, we must have, for all $B \in HS(X,Y)$,
$$
\begin{split}
\sum\_{i \in I} \langle C e\_i, B e\_i \rangle\_Y
& = 2 \langle Bx, Ax-y \rangle\_Y \\
& = 2 \left\langle B \left(\sum\_{i\in I} \langle x, e\_i \rangle\_X e\_i \right), Ax-y \right\rangle\_Y \\
& = \sum\_{i\in I} \left \langle Be\_i, 2 \langle x, e\_i \rangle\_X (Ax-y) \right \rangle\_Y.
\end{split}
$$
Hence, one can define $C$ by $C e\_i := 2 \langle x, e\_i \rangle\_X (Ax-y)$.
Then $C \in HS(X,Y)$ and $\|C\|\_{HS} = 2 \|x\|\_X \| Ax-y\|\_Y$.
Eventually, one can thus identify the derivative as
$$
HS(X,Y) \ni DF\_{\rvert A} : \begin{cases}
X \to Y, \\
z \mapsto 2 \langle x, z \rangle\_X (Ax-y).
\end{cases}
$$


