The Lord Advocate said he had informed First Minister Nicola Sturgeon of his intention to quit the post after five years in the job.
He said it had been a "real privilege" to lead the prosecution service in Scotland and to provide legal advice to the Scottish government.
The law officer added that it was time to "step down and do other things".
In a statement, Mr Mulholland said: "In recent years the Crown has embedded specialisms in the way it does its job.
"Our expertise in handling offences including rape, domestic abuse, serious organised crime, counter terrorism and cold cases has helped us become one of the most effective prosecution services in the world and given victims greater confidence to report crimes.
"It's been an honour to do this job working with so many dedicated and talented people to deliver justice in some of the most demanding and challenging of cases."
Nicola Sturgeon described Mr Mulholland as "an outstanding Lord Advocate".
In a statement, she said he had carried out his role with "dedication, energy, integrity and intellect".
She highlighted his work in leading Scotland's first successful "double jeopardy" murder prosecution, against the Worlds End murderer Angus Sinclair.
The first minister also praised his role in establishing a National Sexual Crimes Unit and appointing Scotland's first specialist prosecutor to deal with domestic abuse.
"It is clear that he has worked to bring about change to ensure that the system makes a real difference to people's lives, and his dedication to the law and his compassion for others has been behind that drive," she added.
"Frank has made a substantial contribution to the law and Scottish society."
Mr Mulholland served as Solicitor General until 2011, when he succeeded Dame Elish Angiolini as Lord Advocate.
He has faced some criticism during his tenure, most notably over the Crown's decision not to press charges over the Glasgow bin lorry crash.
The families of some of the victims of the 2014 tragedy are now pursuing a private prosecution against the driver.
It is understood that he will formally step down when his successor is appointed by the Queen.

Fire crews were called to the property in Lon Tanyrallt, Alltwen, near Pontardawe, at 01:40 BST on Wednesday.
Jac Davies was rescued from an upstairs bedroom but died at the scene.
His three-year-old brother was also rescued and taken to Swansea's Morriston Hospital along with his sister, six, an 11-month-old brother and their mother who had all managed to escape the blaze.
They were all suffering from suspected smoke inhalation which is not thought to be life-threatening.
In a tribute, his mother, Jennifer Davies, said in a statement: "My big, handsome blue-eyed boy - you're my life, my world, my everything and will never, ever forget you.
"Go ride your train up to heaven and wait for me.
"Kelsey, Riley and Andrew will always remember you."
Mid and West Wales Fire and Rescue Service said Ms Davies, called for help and neighbours tried to get in to help the stranded children.
The cause of the blaze is being investigated by the police and fire service.
Chris Margetts, senior operations manager for the fire service, described the blaze as "very severe" but said smoke detectors may have raised the alarm.
He added: "It is a very close community, and specialist teams from the fire and rescue service will be working with the community in the coming days."
Outside the family home, visibly shocked neighbours stood in the street with their hands over their mouths, trying to make sense of the tragedy.
One woman was consoled by a police officer as she laid a floral tribute at the scene along with two teddy bears.
A card left with some flowers said: "To Jack. So sad to hear you are sadly gone. Harry will miss you, seeing you smile and playing with you at school. Gone too young."
South Wales Police Det Ch Insp Esyr Jones, of Swansea and Neath Port Talbot CID, said: "I would like any witnesses or anyone involved at the scene who have not yet spoken to police to come forward with any information they may have."
Scotland's top law officer, Frank Mulholland, will step down after the Scottish Parliament elections in May.
[NEXT_CONCEPT]
A four-year-old boy has died following a fire at a house in Neath Port Talbot.