In modern treatments of statistical mechanics, the natural base is conventionally used for the Gibbs and Boltzmann entropy without careful justification. While I am aware that the properties of the Shannon entropy are invariant to the choice of base of the logarithm, I suspect that physicists might have careful theoretical justifications for this particular choice.



> 
> Question: Might there be a sound theory behind this convention?
> 
> 
> 


So far a couple reasons occurred to me:


1. Stirling's log-factorial approximation:


\begin{equation}
\ln N! \sim N \cdot \ln N - N \tag{1}
\end{equation}


which is an essential tool in statistical mechanics.


2. The exponential function diagonalises the derivative operator:


\begin{equation}
\forall \lambda \in (0,1), \frac{\partial}{\partial H} \exp(\lambda \cdot H) = \lambda \cdot \exp(\lambda \cdot H) \tag{2}
\end{equation}


which may be useful whenever one wants to analyse variations in the exponential of entropy. The advantage of the exponential of entropy is that it is parameterisation invariant [as pointed out by Tom Leinster on a related question](https://mathoverflow.net/questions/398217/geometric-interpretations-of-the-exponential-of-entropy).


As there might be subtle reasons I have ignored, any useful references on this question are more than welcome.


References:
-----------


1. von Neumann, John (1932). Mathematische Grundlagen der Quantenmechanik (Mathematical foundations of quantum mechanics) Princeton University Press., . ISBN 978-0-691-02893-4.
2. Landau and Lifshitz. Statistical Physics. Butterworth-Heinemann. 1980.
3. David J.C. MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press 2003.



The origin of the natural base in statistical mechanics
As Matt F. points out, we could just absorb a change of base of the logarithm into the coefficient. The reason that is not convenient in physics is that we would like the same coefficient $k$ to appear in the [ideal gas law](https://en.wikipedia.org/wiki/Boltzmann_constant#Roles_of_the_Boltzmann_constant) $pV=NkT$.


Note that in physics entropy changes can be defined in thermodynamic rather than statistical terms, as the absorbed heat in a reversible transition ($\Delta S=\int dQ/T$). So any change in the base of the logarithm has to be compensated by a change in the coefficient $k\mapsto k'$. And if we would define the entropy $S=k'\log\_2 W$ using the base-2 logarithm, the ideal gas law would read $pV\ln 2=Nk'T$.


