Take the $\alpha$-divergence...

\begin{align}
D_\alpha(p||q) = \frac{1}{\alpha(1-\alpha)} \int_x \alpha p(x) +(1-\alpha)q(x)-p(x)^{\alpha}q(x)^{1-\alpha}dx
\end{align}

We are going to examine a modified form of this equation. Notice the $p(x)^{\alpha}$ in the final term. We will replace this with a linear term...

\begin{align}
J_\alpha(p||q) = \frac{1}{\alpha(1-\alpha)} \int_x \alpha p(x) +(1-\alpha)q(x)-p(x)q(x)^{1-\alpha}dx
\end{align}

Let $q(x)$ be the pdf for a Gaussian, $\mathcal{N}(\mu, V)$, and $p(x)$ be unknown. We would like to show that minimizing $\frac{\partial J}{\partial \mu_q}$ results in $q(x)$ fitting to one of the modes of $p(x)$. Let us begin by evaluating the partial derivative.

\begin{align}
\frac{\partial J_\alpha}{\partial \mu_q} &= \frac{1}{\alpha} \int_x q_{\mu}(x)\bigg(1-\frac{p(x)}{q(x)^{\alpha}}\bigg)dx \\
 \frac{\partial J_\alpha}{\partial \mu_q} &= \frac{1}{\alpha} \int_x q(x)\Sigma^{-1}(x-\mu_q)\bigg(1-\frac{p(x)}{q(x)^{\alpha}}\bigg)dx
\end{align}


After observing that $\int_x q_{\mu}(x) dx = 0$, via...


\begin{align}
\int_x q_{\mu}(x) dx &= \int_x q(x) \Sigma^{-1} (x-\mu_q)dx \\
&= \Sigma^{-1} \bigg(\int_x q(x) x dx - \mu_q \int_x q(x) dx \bigg)\\
&= \Sigma^{-1} \big(\mu_q - \mu_q \big) \\
&= 0\\
\end{align}


The equation reduces to...


\begin{align}
\frac{\partial J_\alpha}{\partial \mu_q} = -\frac{\Sigma^{-1}}{\alpha} \int_x p(x)q(x)^{1-\alpha}(x-\mu_q) dx
\end{align}

Our objective is to identify the values of $\mu_q$ that make $\frac{\partial J_\alpha}{\partial \mu}$ equal zero. In other words...

\begin{align}
F(\mu_q) = \int_x p(x)q(x)^{1-\alpha}(x-\mu_q) dx = 0
\end{align}


Recall that $\mu_q$ is also used in the definition of $q(x)$, therefore we cannot isolate it with simple algebra. To help us, let us represent $p(x)$ exactly as an infinite Gaussian mixture model

\begin{align}
p(x) = \sum_{i=0}^\infty \pi_i p_i(x)
\end{align}

where each $p_i(x)$ is $\mathcal{N}(\mu_i, V_i)$. Then our objective function becomes

\begin{align}
F(\mu_q) & = \int_x  \sum_{i=0}^\infty \pi_i p_i(x) q(x)^{1-\alpha}(x-\mu_q) dx \\
& = \sum_{i=0}^\infty \pi_i \int_x p_i(x) q(x)^{1-\alpha}(x-\mu_q) dx \\
& = \sum_{i=0}^\infty \pi_i \int_x g_i(x)(x-\mu_q) dx \\
& = \sum_{i=0}^\infty \pi_i \bigg ( \int_x g_i(x)x dx - \mu_q\int_x g_i(x) dx \bigg ) \\
& = \sum_{i=0}^\infty \pi_i \bigg ( \mathbb{E}_{g_i} \big[ x \big ] - \mu_q\int_x g_i(x) dx \bigg ) 
\end{align}

where $g_i(x) = p_i(x)q(x)^{1-\alpha}$, a product of two gaussians. We see, that if we can determine the functional form of $g_i(x)$ along with its integral and expectation, we will have removed the $\mu_q$ term from the definition of $q(x)$.

If we look at equations (10), (11), and (12) from [Products and Convolutions of Gaussian Probability Density Functions](http://www.tina-vision.net/docs/memos/2003-003.pdf) we see that $g_i(x)$ is a scaled gaussian pdf with mean and covariance...

\begin{align}
\mu_{g_i} & = (1 + (1-\alpha)V_q V_{p_i}^{-1})\mu_{p_i} +  (1-\alpha)((1 - \alpha + V_{p_i} V_q^{-1})\mu_q \\
V_{g_i}^{-1} & = V_{p_i}^{-1} + (1-\alpha)V_q^{-1}
\end{align}

$\mathbb{E}_{p} \bigg[ \bigg ]$


```python

```


```python

```

95ae8e13529c804661909dfe16e1b29a9481c2e9
ipynb
Jupyter Notebook
dissertation/Minimizing J finds the peaks of P.ipynb
mathnathan/notebooks
63ae2f17fd8e1cd8d80fef8ee3b0d3d11d45cd28
MIT
2019-12-04T11:04:45.000Z
2019-12-04T11:04:45.000Z
dissertation/Minimizing J finds the peaks of P.ipynb
mathnathan/notebooks
63ae2f17fd8e1cd8d80fef8ee3b0d3d11d45cd28
MIT
dissertation/Minimizing J finds the peaks of P.ipynb
mathnathan/notebooks
63ae2f17fd8e1cd8d80fef8ee3b0d3d11d45cd28
MIT
Qwen/Qwen-72B
1. YES
2. YES