Consider a finite state, irreducible Markov chain with a rate matrix $Q$ and a stationary distribution $\pi$. Suppose the chain starts with the initial distribution $p$ at time $0$, then at time $t$ the distribution is given by $$p\_t = p\*e^{Qt}$$
The relative entropy of $p\_t$ with respect to the $\pi$ (also known as the Kullback-Leibler divergence $D(p\_t || \pi)$) is given by
$$D(p\_t || \pi) = \sum\_i p\_t(i) \log \left(\frac{p\_t(i)}{\pi(i)}\right)$$
where $i$ ranges over all the states.


It seems well known that $D(p\_t || \pi)$ monotonically decreases as $t$ increases. (For instance, section $2.9$ of Cover and Thomas's book here <https://web.cse.msu.edu/~cse842/Papers/CoverThomas-Ch2.pdf>).


My question is:



> 
> Is $D(p\_t || \pi)$ a convex function of $t$?
> 
> 
> 


I simulated for many random matrices using Matlab and did not find a counter example. Any references would be appreciated!



Convergence of Markov chains in terms of relative entropy
Turns out $D(p\_t||\pi)$ need not always be convex. The following paper demonstrates a counter-example in section $4.2$- <http://arxiv.org/abs/0712.2578>


