Stabbing victim John Culshaw's remains were retained by Greater Manchester Police without his family's knowledge.
Dozens of other families have also been told their loved ones' body parts were kept by the force.
GMP said whether or not to contact the affected families was a decision it "agonised over for a number of months".
Mr Culshaw, 26, was stabbed to death in Wigan in 1993.
His family buried him believing his body was intact, and were unaware his stomach, liver and other tissues had been retained after two post-mortem examinations until police officers visited them in September last year.
His mother, Jennifer Shaw, said she wanted to know why it had taken so long to come to light.
"He's my son. And he's been left - half of him," she said.
"If he'd have died and they'd asked me if they could use the parts to help somebody, then yes. But just to be sat in a lab for 23 years doing nothing, that's just horrendous."
The body parts were discovered following an audit of the former Forensic Science Service in 2014, the Manchester Evening News revealed.
It found more than 180 samples - including major organs - from victims of crime in Greater Manchester were being held for the police force, which has admitted there could be more body parts unaccounted for.
It became illegal in 2006 to remove or store human tissue without families' consent following an investigation at Liverpool's Alder Hey hospital in 1999.
Organs from hundreds of babies who died at the hospital were secretly taken.
The body parts kept for Greater Manchester Police were removed before that date.
Assistant Chief Constable Debbie Ford said: "Once we knew about the samples we felt that such a sensitive issue was never going to be solely about following the regulations or the letter of the law.
"For this reason we consulted with a number of relevant people, including community and faith groups, as well as our own Ethics Committee. Following this extensive consultation it was agreed we had an ethical and moral duty to inform the next-of-kin in the vast majority of cases."
She said officers had offered a range of options for "sensitive disposal" of the samples, which the force would pay for.
However, families have been told some samples could remain unaccounted for "for various reasons", she said.
John Culshaw's family will bury his organs in a private second ceremony on Thursday.
"Somebody has made a big mistake. Not just me but a lot of other families are suffering as well," Ms Shaw said.
"I don't want anyone else to go through this. It's devastating."

LudoSport has opened its first academy teaching seven forms of combat from the Star Wars world using flexible blades mounted on weighted hilts.
The sport began eight years ago in Italy but has only just come to England with the first classes in Cheltenham.
Instructor Jordan Court said people were already "hooked".
The classes in Cheltenham began last month. So far there are six pupils, but this number is expected to increase.
Mr Court attended an international boot camp to learn the different stages of the sport which range in characteristics from defensive in stage one to "aggressive and flamboyant" in stage four and "unpredictable" in stage seven.
The 34-year-old said fighting with "an elegant weapon for a more civilized age" was a childhood dream come true.
"My brother and I used to steal our Mum's bamboo garden canes to fight with," he said.
The lightsabers used in the sport are all hand-made and are provided for use during the classes, Mr Court said.
The sport is so new to England that there have only been a handful of classes so far but Mr Court said there were plans to begin teaching in Bristol by the end of October... if "the Force" was with the academy.
Lightsaber Combat Academy
There are several ranks for those wishing to become a fully-fledged Jedi Knight:
A mother is preparing to bury her son for a second time after being told his organs were removed and stored in secret for more than 20 years.
[NEXT_CONCEPT]
Star Wars fans are being given the opportunity to become Jedi Knights and learn how to wield lightsabers in combat.