Why do interest rates move together? Why do interest rates (mortgage, LIBOR, Treasury yields, etc.) move together? What is the fundamental reason behind that?

From my understanding, what drives it is the Federal Funds Rate, which is the rate at which banks borrow from the Federal Reserve. If that rates decrease, then they can borrow at a lower rate, which in turn impact their loan business because they can issue lower rate loans. 

Are there other reasons why that is the case? And what are the cases where rates do not move together?
The assumption is based on intuition but it a bit more complicated:
Basically each and every country has short term interest rates and long term interest rates, based on the debt the country is managing.

Short term interest is usually being set by central banks (the fed in the US), when the long term debt is managed by treasury. long term affect mortgages while short terms affect, short term loans and deposits
Long and short term rates affect each other - the long term interest, being riskier, in the same country, will yield more

In today's open global capital markets, money seeks the best returns. with free money flows, so the best interest - inflation ( - risk) will pull more capital
And on the other hand, it will increase the country's currency value (higher demands for the currency). This in turn, will affect exporting, for each country.
Since this game is a global game between countries, they will pretty much react to each other's actions, effectively matching rates and policies

Note that the description I gave may sound weird, and is a bit contradictory of economic theory, but that is the reality for the past decade...

Monetary policy in the world today, is in a theoretical no mans land