I am working on a Markov-modulated Poisson process $\{N\_{t}, t \geq 0\}$, which is itself a Poisson but the rates of which are governed by a CTMC. In my case, the CTMC is a one-class, aperiodic and positive recurrent MC.


My questions are the following


1. Are the interarrival times of $\{N\_t, t \geq 0\}$ i.i.d? My guess is NO, and it is because the rate of the second interarrival is dependent on the state of the previous one.
2. If I want to get the distributions of interarrival times, say $X\_1$, can I do
\begin{equation}
\mathbb{P}\{X\_1 > t\} = \sum\_{s \in \mathcal{S}}e^{-\mu\_{s}t}\pi\_{s}
\end{equation}
where $\pi\_{s}$ is a stationary initial distribution for state $s$ in the CTMC. Is this correct?
3. If I want further to compute the joint distribution of interarrival times, I do the following
\begin{equation}
\mathbb{P}\{X\_1 > t,...,X\_n > t\} = \sum\_{s\_1, s\_2,...,s\_n}\prod\_{k=1}^{n}e^{-\mu\_{s\_k}t}\prod\_{l=1}^{n-1}R\_{s\_l,s\_{l+1}}\pi\_{s\_1}
\end{equation}
For instance, 
\begin{equation}
\mathbb{P}\{X\_1 > t, X\_2 > t, X\_3 > t\} = \sum\_{s\_1 \in \mathcal{S}, s\_2 \in S\_1, s\_3 \in S\_2}e^{-\mu\_{s\_1}t}e^{-\mu\_{s\_2}t}e^{-\mu\_{s\_3}t}R\_{s\_1,s\_2}R\_{s\_2,s\_3}\pi\_{s\_1}
\end{equation}
where $S\_1$ is the possible next states connected to $s\_1$; $R\_{s\_1,s\_2}$ is the transition probability from $s\_1$ to $s\_2$; and $\pi\_{s\_1}$ is a stationary initial distribution for the CTMC. What I do here is simply conditioning on all possible states for interarrival times and use the independence after conditioning. But I am not quite sure about this.


**Motivation** 


Since there has been no reply to the above questions, which may be because they looked like homework questions. But it is not. As a matter of fact, I am trying to solve the following random sum
\begin{equation}
Q = \sum\_{i=1}^{N}X\_i
\end{equation}
wherein $X\_i$ is the i-th interarrival time of an MMPP and $N$ is also a random variable defined as
\begin{equation}
N = \inf\{n : X\_1 < P,...,X\_n < P, X\_{n+1} > P\}
\end{equation}
namely the first time we have an interarrival time greater than some constant $P$. My goal is to get distributional properties of the random sum (distribution, expectation and so like).



Are interarrival times of doubly-stochastic Poisson I.I.D.?
If I understand correctly, there is a hidden Markov chain, say $(S\_t)$, whose state $s = S\_t$ describes the rate $\mu\_s$ at which signals of the observed counting process $N\_t$ arrive. If this is correct, then the distribution of the waiting time for the first signal is *much* more complicated. The best way to understand it is to see how $N\_t$ is constructed.




---


To avoid nested indices, let us use the notation $N(t)$, $\mu(s)$ etc. First, define the *additive functional*
$$ A(t) = \int\_0^t \mu(S(\tau)) d\tau .$$
This functional measures the average number of signals up to time $t$, conditionally on a fixed path of $(S(t))$. Therefore, if $(\tilde{N}(t))$ is a Poisson process with intensity $1$, then $(N(t))$ can be defined as
$$ N(t) = \tilde{N}(A(t)). $$
Therefore, the waiting time for the first signal $X\_1$ satisfies
$$
\mathbb{P}(X\_1>t) = \mathbb{P}(\tilde{N}(A(t)) = 0) .
$$
Using the independence of $(\tilde{N}(t))$ and $(A(t))$, we get
$$
\mathbb{P}(X\_1>t) = \mathbb{E} e^{-A(t)} .
$$
The quantity $e^{-A(t)}$ is a *multiplicative functional*. Evaluation of its expectation $$u(t, s) = \mathbb{E}(e^{-A(t)} | S(0) = s)$$ for an arbitrary Markov process $(S(t))$ is a non-trivial problem: $u$ solves the Schrödinger evolution equation $$\partial\_t u(t, s) = L u(t, s) - \mu(s) u(t, s),$$ where $L$ is the generator of $(S(t))$. For a continuous time Markov chain on a finite state space $\{s\_1, s\_2, \ldots, s\_n\}$ this reduces to a system of ODEs: the vector $u(t) = (u(t, s\_1), u(t, s\_2), \ldots, u(t, s\_n))$ satisfies $$u'(t) = \mathbb{Q} u(t) - \mu u(t),$$ where $\mathbb{Q}$ is the generator matrix and $\mu$ is a diagonal matrix with entries $\mu(s\_1), \mu(s\_2), \ldots, \mu(s\_n)$ on the diagonal; thus $u(t) = \exp(t (L - \mu))$.


Starting from a stationary distribution of $(S(t))$ does not simplify the above equation. Furthermore, after the first signal arrives, $(S(t))$ need not be in a stationary distribution any more! This makes the evaluation of the waiting time for the second signal $X\_2$ apparently complicated.




---


Let me finish with two simple examples. First, suppose that $(S(t))$ has two states $s\_1$ and $s\_2$, with $\mu(s\_1) = 0$ and $\mu(s\_2) = 1$, and that the transition rate between $s\_1$ and $s\_2$ is $1$ in both directions. The stationary distribution is $\mathbb{P}(S\_0 = s\_1) = \mathbb{P}(S\_0 = s\_2) = \tfrac{1}{2}$. After the first signal, $(S(t))$ is necessarily in state $s\_2$. In particular, this means that $X\_1$ and $X\_2$ have different distributions!


Now suppose that $(S(t))$ again has two states $s\_1$ and $s\_2$, but this time $\mu(s\_1) = 1$ and $\mu(s\_2) = 1000$, and the transition rate between $s\_1$ and $s\_2$ is zero (or just very small if you want $(S(t))$ to be ergodic) in both directions. Again $\mathbb{P}(S\_0 = s\_1) = \mathbb{P}(S\_0 = s\_2) = \tfrac{1}{2}$ is a stationary distribution. If $X\_1$ is small, it is very likely that $(S(t))$ started in state $s\_2$, and thus $X\_2$ is likely to be small as well. On the other hand, if $X\_1$ is of the order of $1$, then we can expect that $(S(t))$ is in state $s\_1$, and so $X\_2$ is highly unlikely to be small. (Remember that $(S(t))$ does not change state at all, or at least it does change it with very low intensity). Therefore, $X\_1$ and $X\_2$ are not independent.


