I posted this over on MSE without much luck. Not sure if posting here is considered cross-posting but I can remove it if it is.


Let $X\sim\mathcal N(\sqrt 2,1/x^2)$. The expected value $\mathsf EX^{-1}$ is undefined; however, we can [assign it a value](https://math.stackexchange.com/questions/2368344/cauchy-principal-value-of-mathbbe1-x-for-x-sim-mathcaln-mu-sigma2) via the Cauchy principal value interpretation
$$
\mathsf{E}X^{-1}\overset{\text{p.v.}}{=}\lim\_{\epsilon\nearrow 0}\left(\int\_{-\infty}^{-\epsilon}+\int\_\epsilon^\infty\right)\frac{1}{t}f\_X(t)\,\mathrm dt=\sqrt 2 x\mathcal D(x),
$$
where $\mathcal D(x):=e^{-x^2}\int\_0^xe^{t^2}\,\mathrm dt$ is [Dawson's integral](https://mathworld.wolfram.com/DawsonsIntegral.html). Notice that as $x\to\infty$, $\mathsf{Var}X\to 0$ and so I wondered what would happen if I were to apply the $\delta$-method to estimating $\mathsf EX^{-1}$. Let $g(X)=1/X$. Expanding $g$ in a Taylor series about $\mathsf EX=\sqrt 2$ gives
$$
g(X)=\frac{1}{\sqrt 2}\sum\_{k=0}^\infty\left(\frac{\sqrt 2-X}{\sqrt 2}\right)^k
$$
and so evaluating the expected value simply requires knowing central moments of the normal distribution. We find
$$
\mathsf Eg(X)
%=\frac{1}{\sqrt 2}\sum\_{k=0}^\infty\mathsf E\left(\frac{\sqrt 2-X}{\sqrt 2}\right)^k
%=\frac{1}{\sqrt 2}\sum\_{k=0}^\infty\mathsf E\left(\frac{X-\sqrt 2}{\sqrt 2}\right)^{2k}
=\frac{1}{\sqrt 2}\sum\_{k=0}^\infty\frac{(2k-1)!!}{(2x^2)^k}.
$$
Looking back at our expression for $\mathsf EX^{-1}$ it stands to reason that if we then divide $\mathsf Eg(X)$ by $\sqrt 2 x$ that we obtain an estimate for $\mathcal D(x)$ at large $x$, namely,
$$
\mathcal D(x)\sim\frac{1}{2x}\sum\_{k=0}^\infty\frac{(2k-1)!!}{(2x^2)^k}=\frac{1}{2x}+\frac{1}{4x^3}+\mathcal O(x^{-5}).
$$
Comparing these first two terms with [eqn. (9) here](https://mathworld.wolfram.com/DawsonsIntegral.html) seems to indicate the above expression is indeed an asymptotic expansion of $\mathcal D(x)$ for large $x$ and thus $\mathsf E g(X)$ provides us with an asymptotic expansion for the Cauchy principal value of $\mathsf EX^{-1}$ as $x\to\infty$.


There is no reason to stop here as we could go further and apply the same approach to "estimate" $\mathsf{Var}X^{-1}$ yielding
$$
\mathsf{Var}X^{-1}\sim
%\mathsf Eg^2(X)-(\mathsf Eg(X))^2
\frac{1}{2}\sum\_{k=0}^\infty\frac{(2k+1)(2k-1)!!}{(2x^2)^k}-\frac{1}{2}\left(\sum\_{k=0}^\infty\frac{(2k-1)!!}{(2x^2)^k}\right)^2
=\frac{1}{4x^2}+\frac{1}{x^4}+\mathcal O(x^{-6}).
$$



> 
> My question fundamentally has to do with **what exactly the $\delta$-method is estimating** when we apply it to estimating moments of random variables for which the moments to not exist? In this specific case, we could claim (at least I think) that the $\delta$-method gave us an asymptotic expansion for the Cauchy principal value of $\mathsf EX^{-1}$. However, for the second moment, the Cauchy principal value interpretation of the integral would give us $\mathsf EX^{-2}\overset{\text{p.v.}}{=}\infty$ whereas the $\delta$-method gave us a finite expression. So what the heck did the $\delta$-method estimate?
> 
> 
> 



Using $\delta$-method to "estimate" undefined moments of a random variable?
$\newcommand\vp\varepsilon$


1. Of course, your divergent series for $EX^{-1}$ and $EX^{-2}$ should be understood as [asymptotic expansions](https://en.wikipedia.org/wiki/Asymptotic_expansion#:%7E:text=In%20mathematics%2C%20an%20asymptotic%20expansion,the%20function%20tends%20towards%20a). The delta method practically never involves series; it involves asymptotic expansions instead, with an appropriately controlled remainder.
2. Take any natural $k$. With the modifications discussed above, your delta method estimates a lot of things. In particular, for $X\sim N(a,1/x^2)$ with a real $a>0$ and $x\to\infty$ and for any $\vp\in(0,a/2)$, it estimates
$$EX^{-k}1(|X|>\vp)=I\_{k,a}+J\_{k,a}(\vp),$$
where
$$I\_{k,a}:=EX^{-k}1(|X-a|<a/2),$$
$$J\_{k,a}(\vp):=EX^{-k}1(|X|>\vp,|X-a|>a/2).$$
We have
$$|J\_{k,a}(\vp)|\le\vp^{-k}P(|X-a|>a/2)=2\vp^{-k}P(Z>ax/2)=o(x^{-m})\tag{1}$$
for any natural $m$, where $Z\sim N(0,1)$.


Using the series
$$x^{-k}=\sum\_{n=0}^\infty \binom{-k}{n}a^{-k-n}(x-a)^n$$
for $x$ with $|x-a|<a/2$, we get the asymptotic expansion
$$
\begin{aligned}I\_{k,a}&\sim\sum\_{n=0}^\infty \binom{-k}{n}a^{-k-n} E(X-a)^n1(|X-a|<a/2) \\ 
&\sim\sum\_{n=0}^\infty \binom{-k}{n}a^{-k-n} E(X-a)^n+o(x^{-m})
\end{aligned}$$
for any natural $m$ (cf. (1)). So,
$$EX^{-k}1(|X|>\vp)\sim\sum\_{n=0}^\infty \binom{-k}{n}a^{-k-n} E(X-a)^n.$$
In particular,
$$EX^{-k}1(|X|>\vp)=\frac1{a^k}\Big(1+\frac{(k+1) k}{2 a^2 x^2}+\frac{(k+3) (k+2) (k+1) k}{8 a^4 x^4}\Big)+O(x^{-6}).$$


These results will hold if, instead of fixing $\vp$, we allow $\vp$ to go to $0$, but not overly too fast: in particular,
the following very mild requirement is enough: $\vp>e^{-ba^2x^2/k}$ for some real $b\in(0,1/2)$ and all large enough $x>0$.




---


(For $k=1$, $EX^{-k}$ exists in $\mathbb R$ in the principal value sense -- that is, as $\lim\_{\vp\downarrow0}EX^{-1}1(|X|>\vp)$, but this does not hold for any other natural $k$. This distinction has little, if anything, to do with the delta method.)


