Do we have some evidence of overshooting? I was going through overshooting of exchange rates (which I probably haven't seen since undergrad) and it seemed to me that the story is a bit of a stretch. I wonder if there is some empirical test that would convince me otherwise.
It depends on what you mean by overshooting. 

The original Dornbusch (1976) story is that following a surprise permanent increase in the money supply, the nominal exchange rate "overshoots" its long-run level (the long-run level increases proportionately with the money supply). This is not a directly relevant story today (and I'm not sure that it ever was), because few changes in monetary policy take the form of surprise permanent changes in the money supply.

The mechanism for Dornbusch's "overshooting" is that since the price level takes time to adjust, in the short term there is an increase in the real money supply, leading to a decrease in nominal interest rates. This decrease in nominal interest rates leads the currency to weaken. 

If we cut out the first part about the shock to the money supply and just think about the mechanism itself, where (all else equal) lower nominal interest rates lead a currency to weaken relative to its long-term level, then we have a much more relevant and plausible story. Whenever central banks are looser or tighter than expected in policy, you usually see the exchange rate weaken or strengthen, respectively, as a result. This is related to the notion of uncovered interest rate parity, but it's a weaker version: it doesn't say that UIP always holds (which it definitely doesn't), just that it conditionally holds in response to an interest rate shock.

Surprisingly, the international finance literature has been mixed on whether this is true or not, but Michael Kiley has recently shown that conditional UIP seems to hold in response to monetary policy surprises, as identified by monetary policy statement days. I suspect that the previous literature, which was based mainly on VARs without a clear identification strategy, struggled to find a consistent effect due to bad research design.