This is a pun on "scale". Building something "to scale" is when you build a larger structure based on a model or plan using the same proportions. Scale means to climb over something like a wall. Prison walls are made to not be climbed over so as to keep the prisoners inside.  
Prison walls are never built to scale .