Given three matrices $A$ (broad), $B$ and $C$, I'd like to find the derivative of 


\begin{align}
f = \textrm{tr} \{BA^+\} + \textrm{tr} \{B(A^+)^TCA^+B^T\}
\end{align}


with respect to $A$, where $A^+ = A^T(AA^T)^{-1}$ is the Moore–Penrose pseudo inverse. I know how to compute the derivative of $\frac{\partial \ \textrm{tr} \{ BA^+\}}{\partial A^+} = B^T$ , but cannot figure out a) how to compute the derivative of $A^+$ with respect to $A$ and b) how to use the matrix derivative chain rule to combine these two results. 



Derivative of trace of pseudo inverse
You should think of $A$ as a time dependent matrix $A=A\_t$, denote by $\dot{A}$ the $t$-derivative of $A$, and then think of $f$ as a function of $t$. 


To compute the derivative of the inverse of $t$-dependent matrix $B\_t$ proceed as follows


$$ 1=B\_t B\_t^{-1}\Rightarrow 0= \dot{B}\_t B\_t+B\_t\frac{d}{dt}B\_t^{-1} $$


so that


$$ \frac{d}{dt}B\_t^{-1}=-B\_t^{-1}\dot{B}\_t B\_t. $$


Now let $B\_t= A\_tA\_t^\*$. Then $\dot{B}\_t=\dot{A}\_tA\_t^\*+A\_t\dot{A}\_t^\*$ and


$$A\_t^+=A\_t B\_t^{-1},$$


$$\dot{A}\_t^+= \dot{A}^\*\_t B\_t^{-1}+A\_t^\*\frac{d}{dt} {B\_t}^{-1}=\dot{A}^\*\_t B\_t^{-1}-A\_t^\* B\_t^{-1}\dot{B\_t} B\_t^{-1}. $$


The Frechet derivative of $A\mapsto A^+$ at the point $A\_0$ is the linear map $H\to L(H)$ obtained from the above equality by setting $t=0$ and $\dot{A}\_0= H$.


$$ H\mapsto L(H)=H^\*B\_0^{-1}-A\_0^\*B\_0^{-1}\dot{B}\_0 B\_0^{-1} =H^\*B\_0^{-1}-A\_0^\*B\_0^{-1}(HA\_0^\*+A\_0H^\*)B\_0^{-1}, $$


$$ B\_0= A\_0A\_0^\*. $$


This follows from the fact that the Frechet derivative of $A\to A^+$ in the direction $H$ is the derivative at $t=0$ of the path $A\_t^+$, $A\_t=A\_0+tH$.


**Update.** The $\DeclareMathOperator{\tr}{tr}$ trace defines an inner product onn the space of $n\times n$ matrice via the equality


$$(A,B):=\tr(A^\* B). $$


The differential of the function $\tr(BA^+)$ is a linear functional on the space of $n\times n$ matrices given by


$$ H\mapsto L(H) =\tr B \bigl(H^\*(A\_0A\_0^\*)^{-1}-A\_0^\*(HA\_0^\*+A\_0H^\*)(A\_0A\_0^\*)^{-1}\bigr). $$


To simplify the presentation let me set $S=(A\_0A\_0)^{-1}$ so that


$$ L(H)= \tr B(HS-A\_0^\*HA\_0^\*S-A\_0^\*A\_0H S) $$


$$= \tr BHS-\tr B A\_0^\*HA\_0^\*S-\tr BA\_0^\*A\_0 H S $$


$$ =\tr SB H-\tr A\_0^\*S B A\_0^\*H-\tr SBA\_0^\*A\_0 H $$


$$= \tr \underbrace{(SB - A\_0^\*S B A\_0^\*- SBA\_0^\*A\_0)}\_{=: G} H =(G^\*, H). $$


We deduce that the gradient at $A\_0$ of the function $A\mapsto \tr(B A^+)$ is the matrix $G^\*$.


