I was reading a recent theory paper in machine learning by Kenji Kawaguchi and Leslie Pack Kaelbling


<https://arxiv.org/pdf/1901.00279.pdf>


and the authors seem to suggest in section 2.2 that cross-entropy loss for classification is not twice differentiable. This seems wrong, I thought it was $C^\infty$.


What am I missing?



Cross entropy loss is not twice differentiable?
Kawaguchi and Kaelbling are saying that the results by [Liang et al.](https://arxiv.org/pdf/1805.08671.pdf) are not applicable to binary classification with cross-entropy loss: in the linked article by Liang, the loss criterion $L:\mathbb{R}^2 \rightarrow \mathbb{R}$ is of the form $L(p,q) = \tilde{L}(-pq)$, where $\tilde{L}:\mathbb{R} \rightarrow \mathbb{R}$ is a suitable function. This implies that the loss criterion $L$ is symmetric in $(p,q)$ and the cross-entropy loss, while $\mathcal{C}^2$, is not symmetric.


