The remark was directed at a woman in the audience who had criticised Mr Martelly and his administration.
According to Haitian media, Mr Martelly told her to "go get a man and go into the bushes" at the rally in Miragoane.
A presidential adviser said Mr Martelly's words had been distorted.
Mr Martelly, who is in the final year of his five-year-term, is a former singer of Haitian dance music known for his flamboyant style and sometimes colourful language.
The row was triggered by President Martelly's reaction to a woman at a campaign rally in the port city of Miragoane on 29 July.
The woman complained about the failure of the government to provide electricity to her area, prompting Mr Martelly's hostile response, which in turn was greeted by cheers and clapping from the audience.
In response to the incident, Women's Affairs Minister Yves Rose Morquette, Social Affairs Minister Victor Benoit and Secretary of State for Alphabetisation Mozart Clerisson announced on Wednesday that they would step down from their posts.
They all belong to the Fusion of Haitian Social Democrats party, which is part of the governing coalition.
Party leader Edmonde Supplice Beauzile said Mr Martelly's behaviour was "unbecoming" to a head of state.
Ms Supplice Beauzile is running for president in the upcoming presidential election in October, in which Mr Martelly will not be able to stand again.
She said that "by insulting this woman, he has insulted all Haitian women".
"Such behaviour is a shame for the country," she added.
She also urged voters to "punish" candidates for Mr Martelly's party standing in Sunday's legislative election.

Ronaldsway Airport has been closed due to a plane not being able to taxi off the runway following its unsuccessful attempt to fly to Belfast.
The Isle of Man Steam Packet Company cancelled the 08:45 GMT Douglas to Heysham ferry and the 14:15 return.
The Manx Met Office said strong winds were expected to hit the island throughout the day.
Director of Ports Ann Reynolds said the plane had been unable to land in Belfast and had returned to the island, and that all passengers on board had been "returned to the airport by bus".
She added that the pilot landed safely but decided the winds were too strong to taxi back to the airport.
Citywing tweeted that all flights to and from the island were "cancelled for today" and the airport said it would remain closed until 13:00.
A spokesman for the ferry company said the 19:45 service was expected to depart as scheduled.
The strong winds have also brought trees and power cables down across the Isle of Man affecting motorists.
Isle of Man Constabulary have shared pictures on social media of fallen trees and damage to the roof of the power station in Douglas.
A police spokesman said the south of the island was "worst affected".
"Trees are coming down all over the island, so please do not travel - if you have to travel, keep journeys to a minimum," he said.
The Isle of Man Met Office has issued an amber weather warning and also warned of heavy rainfall at times throughout the day.
Three officials in Haiti's governing coalition announced their resignation on Wednesday in protest at a remark President Michel Martelly made at a campaign rally last week.
[NEXT_CONCEPT]
Gale force winds have led to the cancellation of ferries and flights between the Isle of Man and the UK.