I am new to methods for simulating Markov chains in order to sample from the target, unknown distribution. After a couple days of reading, I found out that even though people have realized that non-reversible (or, irreversible) Markov chains usually converge faster to their invariant distributions. There are not many papers on irreversible Markov chains for the purpose of generating the unknown distribution; instead, reversible Markov chains dominate the literature.


Could you please please explain why the irreversible Markov chains are so less frequently used to induce the unknown distribution one wants to sample from? Thanks in advance.


Regards,


Chee



Markov chain Monte Carlo: why is non-reversible MC MC not as popular?
I will take a stab at this, even though I no longer have the full eloquence of a working mathematician. First of all reversible Markov chains are really self-adjoint operators in disguise, whose spectral properties are therefore well-understood at least in theory. But even in this nice setting, many seemingly simple questions cannot be answered. For instance, take a card shuffling example, where you swap a randomly chosen pair of adjacent cards in a deck with 1/2 probability, and ask about the rate of convergence in say the most natural total variation distance. Some of its eigenvalues that correspond to irreducible characters of the symmetric group are rational, but the majority of them are irrational. Of course this does not prevent us from getting good rates of convergence in theory. The current best record of upper and lower bounds have a factor of two in between, and nobody has any clue how to close the gap. Another more serious example is the random walk on proper coloring, for which the right order of magnitude is not known in many cases of number of colors and maximum graph degree. It was conjectured the a uniform lower bound of $n log n$ should hold for all bounded degree graph, which is reasonable in light of coupon collector's phenomenon, but it is still open. A similar assertion for Glauber dynamics on Ising model has been [resolved](http://arxiv.org/abs/0909.5162) a few years ago; you can see how hard it can be to prove such a simple statement.


Even if we know all the eigenvalues of the chain, we are still not guaranteed an easy path to its rate of convergence. One such example is the Glauber dynamics on Ising lattice, which has only recently been [resolved](http://arxiv.org/abs/0909.4320) in the high temperature case, and still open for the [critical](http://research.microsoft.com/en-us/um/people/eyal/papers/criticalising.pdf) and low temperature regime. I can't think of other good examples at the moment, but there should be plenty in [this book](http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCUQFjAA&url=http%3A%2F%2Fpages.uoregon.edu%2Fdlevin%2FMARKOV%2Fmarkovmixing.pdf&ei=vB78U-OZNoGnigKWy4HgCQ&usg=AFQjCNHGd_vISAFRCxyEkft4fdDBzrA3tg&sig2=B-0pwD72xXgk-EubPJkzzg&bvm=bv.73612305,d.cGE) and [this](http://www.stat.berkeley.edu/~aldous/RWG/book.html).


Finally there are definitely some non-reversible chains that have been extensively and successfully studied, such as the [riffle shuffle](http://projecteuclid.org/euclid.aoap/1177005705) and [Thorpe shuffle](http://link.springer.com/chapter/10.1007/978-3-662-43948-7_29). The fact that one can get exact answer for the first one and the right order of magnitude (only recently) for the second one is nothing short of miracles. But from a mathematical point of view, it is more natural to exhaust the study of "easier" reversible cases before tackling the wild west. 


