Are space telescopes completely out of the earth's atmosphere? Is the Hubble Space Telescope, and every other space telescope for that matter, completely outside the Earths atmosphere?
A handful of space telescopes are located in Langrange point L2, 1.5 million km from Earth. This is much farther away than the Moon, and far outside Earth's atmosphere.

WMAP and Planck, which measure the cosmic microwave background (CMB), are located here because Earth is a hundred times brighter than the CMB in this wavelength region. Herschel observes in the infrared, which is heat radiation, so it was also put in L2 to avoid Earth's heat. Others can be found in the list provided in Keith's answer.

Another advantage of putting a telescope in L2 is that the Earth and Sun are always in the same direction, making it easier to observe most of the sky. The cost of putting it here is probably more than an order of magnitude higher, though.

Whether or not low Earth orbit telescopes like Hubble at 550 km are outside Earth's atmosphere is a bit subjective. The atmosphere doesn't stop at some particular point, instead gradually thinning. These space telescope are definitely far enough away that they are not heated up. Neither does the atmosphere affect their observations (by seeing). They are, however, slowly decelerated and thus need a boost once in a while not to fall down.