In time series analysis, a common assumption made is that the series is wide-sense stationary, ex. that it has time invariant mean and covariance. However, as this is often not the case in real life, a common approach is to take the difference of the time series: 


D\_(i) = X\_(i) - X\_(i-1). 


If it doesn't work, then you can do it again. 


In fact, the AutoRegressive Integrated Moving Average (ARIMA) Model for time series forecasting has a parameter for the number of times to take the difference. 


My question: why does differencing work in reducing non-stationary time series to time series? Does it always work? What is the theory here? 


Thanks!



Why does differencing create wide-sense stationary time series?
I don't know why that is a common practice, but it makes sense in the case of a Brownian motion with drift $X\_t=\sigma W\_t+\nu t$. Then we have
$$
\mathbb E X\_{t+1}=\nu(t+1)
$$
which is not time invariant, but
$$
\mathbb E (X\_{t+1}-X\_t)=\nu
$$
which is time invariant. If the drift was of higher order, $\nu t^k$, then you could achieve the same effect by taking the difference $k$ times.


As for the variance, it is $\sigma^2 t$ in this case, and after taking the difference once,
$$
\text{Var}(X\_{t+1}-X\_t)=\sigma^2\text{Var}(W\_{t+1}-W\_t)=\sigma^2
$$
which is time invariant.


