Nineteen sailors were killed and a further 30 injured when the Type 42 destroyer took a direct hit during the Falklands War on 25 May 1982.
As the ship sank in just 20 minutes, it took with it many documents and equipment which were of "importance to the security of the United Kingdom", said Clive Gale.
He was part of a 39-strong Royal Navy team which carried out a number of dives on sunken ships, including HMS Coventry, over a four-month period beginning in the autumn of 1982.
Recalling the HMS Coventry search, he said: "We were very conscious that we were diving on a war grave and the casualties that went down with the ship.
"We actually visited their grave and that stays with you."
Carrying out Operation Blackleg was a physically demanding and dangerous mission for those involved, with some dives lasting for up to eight hours.
"We were 8,000 miles from any kind of rescue facility," Mr Gale added.
"There was still hostilities with Argentina and to cap it all there was a 500lb (227kg) unexploded bomb in the wreck somewhere.
"The whole mission was fraught with near misses.
"Cutting into a steel structure underwater traps oxygen and if you hit it with a spark it explodes quite violently.
"One chap lost the front plate out of his helmet and nearly drowned.
"I was knocked unconscious and had to be pulled back into the bell by a colleague of mine."
Mr Gale admits the mission, 300ft (91m) under the ocean's surface, was to "recover or destroy certain items".
But some things brought back from HMS Coventry, like the cross of nails, were not on the official list.
"I took off the big brass plaque while somebody else recovered the bell and the cross of nails was found," he continued.
"All these things that were really personal to the ship's company were then presented to Coventry Cathedral after we'd finished the job."

The Bundesliga club handed the 28-year-old a 2,250 euros (£1,768) fine - 50 euros for every minute.
"I didn't hear my alarm," the Denmark international told Germany's Bild newspaper. "It's my mistake."
The striker was also punished last month after posting a photo of himself alongside his Mercedes car, when Wolfsburg's policy only allows employees to turn up to training in a Volkswagen.
Nicknamed 'Lord Bendtner' by the German press, the player was once fined £80,000 and banned for a game by Uefa for revealing sponsored underwear after scoring against Portugal in Euro 2012.
And he was also fined by former club Arsenal in 2014 for an unauthorised trip to Copenhagen.

Jin Ok scored the game-winning goal after GB had come back from 2-0 down.
Third-period goals from Angela Taylor and Leanne Ganney levelled the game after Jong Su-hyo and Won Chol-sun had put North Korea in charge.
The game gives Britain their first point of the tournament but they are bottom of Group A after three games.
"Nobody likes losing and it's disappointing," said Britain's head coach Cheryl Smith.
"I am more disappointed because we took some risks. We were 2-0 down, took a risk and got the goal back.
"We got our heads together and the girls listened and we got back into the game.
"The game was going our way when we got back into it at 2-2, but we just didn't finish it.
"We had the chances in overtime but we just did not get it done."
Britain, who have also lost to the Netherlands and South Korea in this event, return to action tomorrow when they face Slovenia at 16:30 (08:30 BST).
A naval diver who searched the wreck of HMS Coventry after it was sunk during the Falklands has described how he was told to ensure secret information from the ship did not get into the wrong hands.
[NEXT_CONCEPT]
Wolfsburg forward Nicklas Bendtner has been fined for turning up 45 minutes late for training after sleeping through his alarm.
[NEXT_CONCEPT]
Great Britain's women fell to an overtime defeat by North Korea in their third Group A game in World Championship Division Two in Gangneung.