I'd like to compute the derivative of an expected value w.r.t one of the parameters that define the mean of a Gaussian:
$ Z=\int \mathcal{N}(x;\mu,\Sigma)f(x) \, dx $, then $ \frac{dZ}{dK}=\text{??}$ when $\mu=g(K)\in\mathbb{R}^{n\times 1}$.
I have three problems with this: 1. The dimensions are not adding up for me. 2. How can I compute this also numerically if $g(K)$ gets too complicated? 3. How do I do this if $\Sigma=h(K)\in\mathbb{R}^{n\times n}$, too?


1. This is what I have so far, if I ignore the dependency of $\mu$ in $K$ for a minute:
$$ \frac{dZ}{d\mu} = \int \frac{d\mathcal{N}(x;\mu,\Sigma)}{d\mu}f(x)\,dx $$
and since $\mathcal{N}(x;\mu,\Sigma)=(2\pi)^{-n/2} \det(\Sigma)^{-1} \exp(-0.5(x-\mu)^T\Sigma^{-1}(x-\mu))$, we have that $\frac{d\mathcal{N}(x;\mu,\Sigma)}{d\mu}=\mathcal{N}(x;\mu,\Sigma)\frac{d\ln\mathcal{N}(x;\mu,\Sigma)}{d\mu}$
so now we can get:
$$ \frac{dZ}{d\mu} = \int \mathcal{N}(x;\mu,\Sigma)\frac{d\ln\mathcal{N}(x;\mu,\Sigma)}{d\mu} f(x) \, dx = \mathbb{E}\left[\frac{d\ln\mathcal{N}(x;\mu,\Sigma)}{d\mu}\right]$$
This means I can compute $$ \frac{dZ}{d\mu} = \mathbb{E}\left[\frac{d\ln\mathcal{N}(x;\mu,\Sigma)}{d\mu}\right] = \mathbb{E}\left[\Sigma^{-1}(x-\mu)\right]$$
Now, comes the problem - this expected value is going to be a vector $\in \mathbb{R}^{n\times 1}$. Now if I want to derive the gradient w.r.t $K$ then:
$$ \frac{dZ}{dK} = \frac{dZ}{d\mu}\frac{d\mu}{dK}$$ and lets say that $\mu$ is the result of calculations of $K\in\mathbb{R}^{p\times q}$, what are the dimensions of $\frac{d\mu}{dK}$? Even if I got the ordering or transpose wrong in the matrix derivations, I still don't see how that dimensions should work because I expect that $ \frac{dZ}{dK} \in\mathbb{R}^{p\times q}$.
An example for $\mu$ can be:
$$ \mu = \left[ AKv ; (AK)^2v; (AK)^3v ; \ldots \right] \in \mathbb{R}^{n\times 1} $$ where the constant $A\in\mathbb{R}^{q\times p}, v\in\mathbb{R}^{q\times 1}$.
2. Is there any software that can compute this symbolically? Or numerically? Matlab will not derive a vector by a matrix. I'm trying to do this with PyTorch in Python with the [autograd](https://pytorch.org/tutorials/beginner/basics/autogradqs_tutorial.html#optional-reading-tensor-gradients-and-jacobian-products), but the problem is again the dimensions, I'm not sure what to put in the vector $v$ that multiplies the gradient.
3. Now let's suppose that $K$ also affects $\Sigma$ too, how do I compute the total influence (gradient) of $K$ on $Z$? With the technique above I can also compute:
$$ \frac{dZ}{d\Sigma} = \mathbb{E}\left[\frac{d\ln\mathcal{N}(x;\mu,\Sigma)}{d\Sigma}\right] = \mathbb{E}\left[\Sigma^{-1}(x-\mu)(x-\mu)^T\Sigma^{-1}-\Sigma^{-1}\right] $$



Derivative of an integral of a Gaussian
**Q1:** $dZ/dK$ is a $p\times q$ matrix with elements
$$\bigl[dZ/dK\bigr]\_{ij}=\sum\_{k=1}^n\frac{\partial \mu\_k}{\partial K\_{ij}}\,\mathbb{E}[f(x)\,\Sigma^{-1}\cdot(x-\mu)]\_k.$$
**Q2:** To evaluate this you will have to specify the function $f(x)$. For some $f$ you will be able to evaluate the expectation value symbolically, but in general a numerical evaluation should be easy and accurate because of the Gaussian weight factor.  

**Q3:** If $\Sigma$ also depends on $K$ you add the terms $\sum\_{kl}(\partial\Sigma\_{kl}/\partial K\_{ij})(\partial Z/\partial\Sigma\_{kl})$.


