The 57-41 vote failed to get the 60 supporters required to proceed on the regulatory overhaul; one Democrat, Ben Nelson of Nebraska, voted with the Republicans.
The vote will send lawmakers back to the negotiating table to hammer out a bipartisan compromise with amended legislation expected to return to the Senate floor soon.
President Barack Obama and the Democrats want to tighten the rules on banks and capital markets to prevent a repeat of the 2008 financial crisis, which led to the great recession.
Many Republicans agree with a need for new bank regulations but oppose the Democrats' Restoring American Financial Stability Act.
Among numerous stipulations, the proposed bill would provide shareholders with a say on pay and corporate affairs with a non-binding vote on executive compensation and director nominations.
And hedge funds with values in excess of $100 million US would be required to register and disclose financial data. Currently there are no such stipulations on the shadowy institutions.