  Certain patterns of symmetry fractionalization in topologically ordered phases of matter are anomalous, in the sense that they can only occur at the surface of a higher dimensional symmetry-protected topological (SPT) state. An important question is to determine how to compute this anomaly, which means determining which SPT hosts a given symmetry-enriched topological order at its surface. While special cases are known, a general method to compute the anomaly has so far been lacking. In this paper we propose a general method to compute relative anomalies between different symmetry fractionalization classes of a given (2+1)D topological order. This method applies to all types of symmetry actions, including anyon-permuting symmetries and general space-time reflection symmetries. We demonstrate compatibility of the relative anomaly formula with previous results for diagnosing anomalies for $\mathbb{Z}_2^{\bf T}$ space-time reflection symmetry (e.g. where time-reversal squares to the identity) and mixed anomalies for $U(1) \times \mathbb{Z}_2^{\bf T}$ and $U(1) \rtimes \mathbb{Z}_2^{\bf T}$ symmetries. We also study a number of additional examples, including cases where space-time reflection symmetries are intertwined in non-trivial ways with unitary symmetries, such as $\mathbb{Z}_4^{\bf T}$ and mixed anomalies for $\mathbb{Z}_2 \times \mathbb{Z}_2^{\bf T}$ symmetry, and unitary $\mathbb{Z}_2 \times \mathbb{Z}_2$ symmetry with non-trivial anyon permutations. 
Relative Anomalies in (2+1)D Symmetry Enriched Topological States