One often hears in popular explanations of the failure to find a "Grand Unified Theory" that "Gravity goes off to infinity, but cutting off the edges gives us wrong answers", and other similar mathematically vague statements. Clearly, this issue has some kind of mathematical explanation, but I'm not really qualified to read the corresponding physics work, so I'm wondering what actually fails mathematically. 



Mathematical explanation of the failure to quantize gravity naively
Other people have said that the problem is that GR isn't renormalizable. I want to explain what that means in measure-theoretic terms. What I say won't be 100% rigorous, but it should get the general story across.


Quantum field theories are generally defined using a Feynman path integral measure. This measn that you compute correlation functions of observables by summing over all histories of your system, weighting each history by e^{-S}, where S is an functional on the space of fields, called the action. In a field theory, these histories are functions on some spacetime manifold.


Just as you define the ordinary integral as a limit of Riemann sums, you define the Feynman path integral as a limit of "regularized" path integral measures. There are a lot of ways to do this; one of the most popular is the lattice regularization. We choose a finite set of points in spacetime, living on a lattice whose nearest neighbors are a distance a apart. Then we choose a "microscopic" action on our space of fields and discretize it, replacing derivatives with finite difference quotients, and approximate the path integral's weighted sum over histories with a sum over functions defined on the lattice.


Let's grant for a moment that, for any fixed lattice spacing a, this procedure defines a quantum theory, meaning that you can use the moments of the measure to compute correlation functions for your observables. Let's also grant that the expectation values of observables satisfy classical equations of motion (so that when we approach the classical limits, and the probability distributions become concentrated at their expected value, we get deterministic evolution of these values).


We usually want to take a continuum limit, making the lattice spacing smaller and smaller. If we do this, while keeping fixed the coupling constants in the microscopic action we used to define our measure, in most cases, we run into a problem: the coupling constants in the classical equations of motion depend on the lattice spacing a, and become infinite as a goes to zero. So that's probably not the limit we want. 


And indeed, in most circumstances, we know the classical physics, and are trying to find a quantum theory that reproduces it. So we'll try something else: make the coupling constants in the microscopic action depend on the lattice spacing, and hope that we can tune them in a way that keeps the classical physics fixed. 


Sometimes this works; sometimes it doesn't. It can happen that there isn't any way of tuning the coupling constants so as to reproduce a nice classical limit; when this happens, the theory is said to be non-renormalizable. This appears to be what happens in General Relativity, if you use the discretized Einstein-Hilbert action as your microscopic action.


