They have been taken to UN centres in Dohuk in Kurdish northern Iraq.
It is unclear whether they escaped in Iraq or were freed, as the UN declined to give more information to avoid jeopardising future releases.
IS killed and enslaved thousands of Yazidis after seizing the northern town of Sinjar in 2014.
Kurdish Peshmerga forces regained control in 2015 but many Yazidis were held captive by IS elsewhere as the group took over large swathes of northern Iraq.
The 36 survivors include men, women and children who were enslaved, the UN Office for the Co-ordination of Humanitarian Affairs said.
They reached Dohuk two nights ago where they are being housed in centres established by the UN Population Fund.
There, they are being reunited with family members and offered care including clothing and medical and psychological aid, the UN said.
The women and children are being cared for at dedicated service points and will be referred for more specialised treatment.
The UN's humanitarian co-ordinator for Iraq, Lise Grande, said the UN was going to do "everything possible" for the rescued Yazidi.
"What these women and girls have endured is unimaginable," she said.
Who, What, Why: Who are the Yazidis?
The UN - which has accused IS of committing genocide against the Yazidi - estimates that 1,500 women and girls are still being held and could be suffering protracted sexual abuse.
A UN report released last year said the extremists had subjected captured Yazidis to the "most horrific of atrocities", killing or enslaving thousands, warning IS aimed to completely erase their way of life.
In March. lawyer Amal Clooney urged the UN to back a UK-led investigation into the crimes committed by IS.
IS has been under increasing pressure in Iraq, losing much of the territory it seized in 2014.
Iraqi forces have recaptured most the city of Mosul from IS, but they are still trying to push the group out of the Old City district in the west.

Hundreds of animal lovers arrived at the scene of the blaze, which broke out at Manchester Dogs' Home in Moss Brook Road in Harpurhey on Thursday evening.
They brought blankets, dog food and other supplies to help some 150 dogs that were pulled from the fire alive.
Others visited a JustGiving page, which raised £500,000 in under 17 hours.
A 15-year-old boy has been arrested on suspicion of arson and an investigation into the cause of the fire continues.
Among those helping with the rescue effort is the Rev Mike McGurk, of nearby Christ Church in Harpurhey, who said: "When any disaster or anything happens in this community, you actually see the true character of the people.
"Hundreds of people have turned out bringing everything from dog leads to food, to water, to finances and it's been really overwhelming.
"We're opening up the church - that's become the centre where people can drop off donations. There'll be staff there throughout the day and night."
Robert Smith travelled from Oldham to help.
He said: "I've got dogs myself and I just want to do what I can. It's tragic. It's seemed like thousands of people have come along - you couldn't move around here."
Tracy Cassidy, from Bury, said: "A friend of mine sent me a message telling me about it and I just instantly burst out crying. I had my daughter on my knee and I just couldn't control it.
"I've got a big fold-up cage and I thought that could be good if they have bigger dogs. I just said to my boyfriend I'm just going to go and see what I can do."
Meanwhile, a call went out on Facebook for people to post "dog selfies" to help raise funds.
People also left messages of sympathy on the JustGiving website.
Jay McMullen said: "It's heart-warming to know there are so many nice people coming together to help."
Thirty-six members of the Yazidi religious minority are free after nearly three years in the hands of so-called Islamic State (IS), the UN says.
[NEXT_CONCEPT]
Volunteers and well-wishers have pulled together in a huge effort to raise more than £500,000 in the wake of a fire that killed 60 animals at a dogs' home.