What is the evidence that galaxies rotate? Do all galaxies rotate?  If so, do they rotate in the same direction?

What observations have been made of galaxy rotation?
Rotation:

One of the easiest way to show that galaxies rotates are Doppler observations of atomic or molecular lines. For example, if the observed galaxy is seen edge-on and rotates clockwise, the left side of the galaxy is going away from you, and is then Doppler-shifted in the red. The right side, going toward you, is Doppler-shifted in the blue.

This kind of observations was realized already in the '60, in radio.

The spiral patern observed in spiral galaxies is also a clear signature of their rotation. Note that spiral are linked to density waves in the galactic disk (see for example Lin &amp; Shu 1964).

Direction:

As for their directions, you can convinced yourself that there is no tendency, just looking at this Hubble telescope gallery. You can also think that galaxies probably rotate by angular momentum conservation during their formation history, and that it highly dependent on initial conditions, that are clearly random.