Why regulate the financial world rather than take over some of the services? The government regulates some of the behavior of financial markets. Understandably, especially considering the crisis.

But would it not be easier if the government overtook some of the aspects of the financial markets? In that case, regulation would not be necessary, since the government was in charge of that particular aspect. 

For example, say the government overtook the business of giving out mortgages to house buyers. In this case, the government would not have to worry about banks giving out subprime mortgages, securitizing them, and selling them off, the government would be in charge of the entire process and since the government does not seek profit-maximization, it would make sure to do it right.
Governments do offer financial services. It varies from country to country, and changes over time. The political drift has been to reduce the role of government since the 1980s, so this role has generally shrunken.

For example, some countries had postal banking - citizens had basic banking functions offered at post offices.

I will just use some Canadian examples; other countries have similar programmes.

The Canadian government has a Business Development Bank, which helps new firms develop (website). It is taking on risks that commercial banks would be reluctant to take. 

The Canada Mortgage and Housing Corporation (CMHC - website) has a programme of mortgage insurance that is compulsory for all high loan-to-value mortgages in Canada (“subprime”). The loans have to meet minimum risk standards, but the loan guarantee means that the private banks are not exposed to the risk of the mortgages. The Canadian housing market slowed down during the Financial Crisis, but fared better than the United States. The fact that the Canadian banks were insulated from the credit risk of risky mortgages was certainly a factor.

Returning to your question - why not expand the programme, and have the government completely take over the mortgage lending role? One obvious advantage for the government is that it is not forced to justify lending decisions to citizens - that is being done in the private sector. People could easily be concerned about favouritism in lending decisions if the government controlled all mortgage lending. For example, if the child of a politician gets a big mortgage when other people are being refused, it could easily turn into a political scandal. On top of this, there is the ongoing political argument between government control versus free markets. The objective of this website is to avoid opinion-based answers, so I will not dwell on that political debate.