Suppose $x_1, ... x_n \sim IG(\mu, \lambda)$ with density:

$$
p(x) = \bigg(\frac{\lambda}{2 \pi x^3} \bigg)^{1/2} \exp \bigg[- \frac{\lambda}{2 \mu^2} \frac{(x - \mu)^2}{x} \bigg]
$$

then, ignoring irrelevant terms the log-likelihood, is given by:

$$
log L(\mu, \lambda) = \frac{n}{2} \log \lambda - \frac{\lambda}{2 \mu^2} \sum_{i=1}^{n} \frac{(x_i - \mu)^2}{x_i}
$$

(a) The score equations for the MLE are given by:

$$
\begin{align}
\frac{\partial}{\partial \mu} \log L(\mu, \lambda) & = \frac{\lambda}{\mu^3} \sum_{i=1}^{n} \frac{(x_i - \mu)^2}{x_i} + \frac{\lambda} {\mu^2} \sum_{i=1}^{n} \frac{(x_i - \mu)}{x_i} \\
& = \frac{\lambda}{\mu^3} \sum_{i=1}^{n} \frac{(x_i - \mu)^2}{x_i} \bigg[1 + \frac{\mu}{(x_i - \mu)} \bigg] \\
& = \frac{\lambda}{\mu^3} \bigg[ \sum_{i=1}^{n} x_i - n \mu \bigg] \\
& = \frac{n \lambda}{\mu^3}[\bar{x} - \mu] = 0
\end{align}
$$
and
$$
\frac{\partial}{\partial \lambda} \log L(\mu, \lambda) = \frac{n}{2 \lambda} - \frac{1}{2 \mu^2} \sum_{i=1}^{n} \frac{(x_i - \mu)^2}{x_i} = 0
$$

The MLEs are the solutions to the above equations and are given by:

$$
\hat{\mu} = \bar{x}
$$

and 

$$
\begin{align}
0 & = \frac{n}{2 \hat{\lambda}} - \frac{1}{2 \bar{x}^2} \sum_{i=1}^n \frac{(x_i - \bar{x})^2}{x_i} \\
& =  \frac{n}{2 \hat{\lambda}} - \frac{1}{2 \bar{x}^2} \sum_{i=1}^n \bigg( x_i - 2 \bar{x} + \frac{\bar{x}^2}{x_i} \bigg) \\
& =  \frac{n}{2 \hat{\lambda}} - \frac{1}{2 \bar{x}^2} \bigg( n \bar{x} - 2n \bar{x} + \bar{x}^2 \sum_{i=1}^n \frac{1}{x_i} \bigg) \\
& = \frac{n}{2 \hat{\lambda}} - \frac{n}{2} \bigg( \frac{1}{n} \sum_{i=1}^n \frac{1}{x_i} - \frac{1}{\bar{x}} \bigg) \\
& = \frac{n}{2} \bigg[ \frac{1}{\hat{\lambda}} - \bigg( \frac{1}{n} \sum_{i=1}^n \frac{1}{x_i} - \frac{1}{\bar{x}} \bigg) \bigg]
\end{align}
$$

So:
$$
\hat{\lambda} = \bigg( \frac{1}{\tilde{x}} - \frac{1}{\bar{x}} \bigg)^{-1}
$$

where the harmonic sample mean is given by:

$$
\tilde{x} = \frac{n}{\sum_{i=1}^n \frac{1}{x_i}}
$$

(b) The off-diagonal elements of the Fisher information matrix are given by:

$$
- \frac{\partial^2}{\partial \lambda \partial \mu} \log L(\mu, \lambda) \bigg|_{\mu = \hat{\mu}, \lambda = \hat{\lambda}} = \frac{\partial^2}{ \partial \mu \partial \lambda} \log L(\mu, \lambda) \bigg|_{\mu = \hat{\mu}, \lambda = \hat{\lambda}} = 0
$$

The diagonal elements are given by:

$$
- \frac{\partial^2}{\partial \mu^2} \log L(\mu, \lambda) \bigg|_{\mu = \hat{\mu}, \lambda = \hat{\lambda}} = \frac{n \hat{\lambda}}{\bar{x}^3}
$$

and

$$
- \frac{\partial^2}{\partial \lambda^2} \log L(\mu, \lambda) \bigg|_{\mu = \hat{\mu}, \lambda = \hat{\lambda}} = \frac{n}{\hat{\lambda}^2}
$$

59a0e43b943555b954ff2e82b359fd5fb003e190
ipynb
Jupyter Notebook
python/chapter-3/exercises/EX3-11.ipynb
covuworie/in-all-likelihood
6638bec8bb4dde7271adb5941d1c66e7fbe12526
MIT
python/chapter-3/exercises/EX3-11.ipynb
covuworie/in-all-likelihood
6638bec8bb4dde7271adb5941d1c66e7fbe12526
MIT
2020-03-24T17:53:04.000Z
2021-08-23T20:16:17.000Z
python/chapter-3/exercises/EX3-11.ipynb
covuworie/in-all-likelihood
6638bec8bb4dde7271adb5941d1c66e7fbe12526
MIT
2021-11-21T10:24:59.000Z
2021-11-21T10:24:59.000Z
Qwen/Qwen-72B
1. YES
2. YES