Homs Governor Talal Barrazi said the evacuation of al-Wair was part of an existing deal with community leaders and would take six to eight weeks.
Opposition activists said the rebels would be allowed to depart with their families for rebel-held northern Syria.
Al-Wair, home to an estimated 75,000 people, has been under siege by government forces since late 2013.
Homs was dubbed the "capital of the revolution" after residents embraced the call to overthrow President Bashar al-Assad in 2011 and drove security forces out of much of the city the following year.
That prompted the government to begin a brutal two-year siege that left whole areas destroyed and eventually forced rebels to withdraw from the Old City in 2014.
Al-Wair, on the western outskirts, is the last rebel-held district.
In December 2015, community leaders agreed a five-stage truce with the government.
The first phase led to 300 rebels leaving and the government allowing in a limited amount of food and humanitarian aid. But the deal subsequently stalled.
The second phase did not take place until September, when several hundred more rebels were evacuated and the government freed 200 imprisoned al-Wair residents.
The next two phases will see about 10,000 to 15,000 people leaving in several groups over the coming weeks, opposition activists from the Homs Media Centre told Reuters news agency.
The pro-opposition Orient News website reported that the rebels planned to head to Jarablus, a town near the border with Turkey that was captured from so-called Islamic State last August.
In a separate development on Monday, several rebel factions said they were not yet ready to send a delegation to attend talks with the government that are scheduled to begin in Kazakhstan on Tuesday.
Ahmed Othman of the Sultan Murad brigade told Reuters that one of the reasons for the boycott was unfulfilled promises relating to a cessation of hostilities agreement brokered by Russia and Turkey at the end of December.
Mr Othman complained that Russia, an ally of President Bashar al-Assad, had failed to halt the government's bombardment of rebel-held areas.

This includes stadium safety expert Paul Scott.
The Cabinet Office report found relations behind the scenes were "broken", and it would be at least another year before another planning application could be submitted.
However, the authors said that a new stadium was still "achievable".
They said the overall business case "remains sound".
The report was commissioned after Mr Scott told a Stormont committee in April that the emergency-exiting arrangements in the proposed 38,000-capacity stadium were flawed.
However, the report has recommended that he be replaced as chair of the Safety Technical Group overseeing the Casement project.
The report has called for other personnel changes:
Sports Minister Carál Ní Chuilín has indicated she will implement all of the recommended changes.
She said a key finding that the stadium is still achievable was welcome and provided clarity.
The report stated: "Relationships between the key stakeholders are broken.
"We believe the use of an independent mediator will be needed to reset working relationships."
Work on the stadium was supposed to start this year but planning permission was overturned in December 2014.
The report stated: "The Casement Park project is behind schedule and has to address two key issues before it can make progress.
"It needs to find a way to resolve issues raised with the proposed emergency-exiting arrangements for the stadium.
"It also needs to reapply for and achieve planning permission."
The report also included a "delivery confidence assessment" that is based on traffic lights, with green being the highest rating.
Casement is rated "amber/red".
Although the report recommends that Mr Scott be replaced as chair of the safety technical group, that does not mean he cannot continue to sit on the group.
The report says his replacement will need mediation and facilitation skills "rather than technical knowledge".
The chairman of the Stormont committee which is investigating Mr Scott's allegations said the Cabinet Office report "is part of the resolution but only part".
Democratic Unionist Party MLA Nelson McCausland said the minister and her officials had more questions to answer.
"This coming week we'll be convening a special meeting of the committee at Stormont and we will be requesting that the minister and the permanent secretary appear before the committee to answer the questions that we will be putting to them," he said.
Syrian rebel fighters have agreed to leave their last enclave in the city of Homs, government officials say.
[NEXT_CONCEPT]
A report into the planned new stadium at Casement Park in Belfast has recommended the replacement of senior figures from key posts in the project.