Arthur Cory, 49, said the flag had been kept in a cupboard at Penllyn Castle near Cowbridge, since he was a boy.
The flag, one of only three surviving from the battle, is expected to fetch up to Â£50,000 at auction.
It is said to have flown above Nelson's warship HMS Leviathan.
The flag's "most probable" route to Penllyn Castle was through Mr Cory's ancestor Nicholas Cory, a rear admiral on William IV's Royal George yacht, the auction house said.
Roland Elworthy, senior valuer at Holt's Auctioneers, said he had "no doubt as to the flag's provenance".

Ashton Carter, US defence secretary, said about 90 people were being trained in a secure location, and they would be paid by the United States.
He said more fighters would begin training soon at sites in Jordan, Turkey, Qatar and Saudi Arabia.
Supporting the troops militarily on the battlefield has not been ruled out.
Nearly 4,000 fighters volunteered for the programme but, after an extensive screening process, only 400 were given places.
"We're starting with the people that we have that we've vetted very carefully," Mr Carter said, "We expect that to be successful and therefore to grow. But you have to start somewhere, and this is where we're starting."
Mr Carter was at pains to point out that the role of the force would be to take on the Islamic State and not the regime of President Bashar al-Assad. However, he did not say what would happen if the rebels engaged the Syrian army.
"We have not determined yet all of the rules of engagement... but we have acknowledged that we have some responsibility to support them," the defence secretary said.
The fighters will be paid by the United States, and, when deployed, will continue to receive back up and support from coalition forces.
There are about 450 coalition forces involved in the training at the four sites, including about 350 Americans.
The Pentagon intends to train and arm more than 15,000 rebels over the next three years.
Mr Carter made it clear that the support did come with strict conditions.
"An explicit part of their training is how to conduct themselves in a way that is consistent with international law," adding: "Any continued support for them would be strongly conditioned upon their continued good conduct."
The Pentagon would not disclose when the troops would be ready to enter the battlefield.
Since last year, the US has been part of an international coalition conducting an air strike campaign against Islamic State, which took over large swathes of Syria and Iraq in a lightning offensive.

Trained by Colin Tizzard, the 2-1 favourite took control at the penultimate hurdle and pulled away to win by eight lengths.
Jockey Tom Scuadmore said: "He gave me a lot of pleasure. It's a special day."
Reve De Sivola held on for second and Deputy Dan trailed in third, another 12 lengths further back.

A statement called on Haitians to remain calm until a solution is found.
Mr Privert was appointed in February for a six-month term.
He filled the vacancy left by former President Michel Martelly, who stepped down with no successor after the suspension of elections last year.
Haiti has been mired in political crisis since the disputed first round on 25 October.
Representatives of Haiti's main donors, which include the UN and the Organisation of American States (OAS), have called for fresh elections without delay.
Has the international community failed Haiti?
"This... marks the end of the 120-day mandate of the republic's provisional president," read a statement from the Senate deputy president and the president of the Chamber of Deputies.
The statement urged people to "remain calm until the National Assembly meets to take the necessary measures".
Last month, a special commission recommended throwing out the disputed results of last year's first-round presidential election and holding a new vote.
The commission said various irregularities meant last October's ballot could not be considered legitimate.
Haiti has been in political turmoil since the first round, in which Jovenel Moise came first but fell short of an overall majority.
Opposition challenger Jude Celestin accused the electoral authorities of favouring Mr Moise and threatened to pull out of the run-off.
Deadlines have since been missed for four run-off votes, on some occasions leading to violent unrest.
A man from the Vale of Glamorgan is selling a rare union jack flag, said to have been flown at the Battle of Trafalgar.
[NEXT_CONCEPT]
The US military has begun training a small group of Syrian rebels in an effort to build a force capable of defeating Islamic State militants.
[NEXT_CONCEPT]
Thistlecrack ended the three-year reign of Reve De Sivola to cruise to an impressive victory in the Grade One Long Walk Hurdle at Ascot.
[NEXT_CONCEPT]
Haiti faces a power vacuum after parliament allowed the mandate of interim President Jocelerme Privert to lapse with still no solution to the country's political crisis.