Mr Gustafsson, 42, is being flown back to Sweden from Africa, Sweden's Foreign Minister Margot Wallström said in a statement on Monday.
He was seized by al-Qaeda in the Islamic Maghreb (AQIM) along with two other men, one of whom was freed in a dawn raid in 2015.
Ms Wallström said Mr Gustafsson was "in good spirits", local media report.
"It is with great pleasure that I can announce that Johan Gustafsson has been released," Ms Wallström added.
She said that the Swede's release was thanks to "extensive efforts" and co-operation between the Swedish foreign ministry, police and "foreign authorities".
Ms Wallström said she had spoken with Mr Gustafsson, who she described as being "happy" and "overwhelmed" by Monday's events.
"I cannot say more at the current time," she added.
Sweden's former Foreign Minister Carl Bildt said that Mr Gustafsson's kidnapping weighed heavily on his mind during his time in the role, which ended in 2014.
He tweeted on Monday: "Extremely gratifying that Johan Gustafsson is free. No single case concerned me more as foreign minister."
Mr Gustafsson was kidnapped along with South African Stephen McGown and Dutchman Sjaak Rijke.
Mr Rijke was freed by French special forces in April 2015 after he was discovered by chance in a dawn raid in northern Mali.
AQIM took a number of Western hostages before the French military deployed its forces in January 2013.
In a separate incident in December 2014, French hostage Serge Lazarevic was freed after a prisoner swap.

The report warns that a failure to reintegrate returnees may lead to further radicalisation.
Somalia's Islamist al-Shabab militants are believed to be recruiting heavily in neighbouring north-eastern Kenya.
Kenya has seen a series of militant attacks with one at a university earlier this year killed 148 people.
Although the report does not mention where the returnees came from, Deputy Secretary General of the Supreme Council of Kenya Muslims (Supkem) Hassan Ole Nadu has confirmed to the BBC that they were fighting for al-Shabab.
The 36-page report was compiled by the IOM, Supkem and the Kenya's interior ministry.
Researchers found that although most adults joined the group voluntarily, nearly a third of children interviewed said they were forced to enlist.
The return of the youth, the report says, provides both "opportunities and threats for and to Kenyans".
"On the one hand, they present an opportunity to counter the radicalising ideologies and recruitment strategies of armed groups," the report says.
"On the other, the response of the security forces is widely perceived to be promoting further radicalisation as the returnees themselves believe that they are unsafe and have limited reintegration options."
Security in the country was identified by participants as the main obstacle to their successful reintegration in the country.
A total of 185 respondents were interviewed, representing nearly 30% of the estimated total of 684 returnees in the country. The majority were between 18 and 34 years old.
Hostage Johan Gustafsson, held by al-Qaeda in Mali since 2011, has been freed, the Swedish government says.
[NEXT_CONCEPT]
Nearly 700 recruits have returned to Kenya after quitting militant groups, a report by the International Organization for Migration (IOM) says.