General Jonathan Vance, Canada's top soldier, has opened an investigation into the weekend incident and has called it "deplorable".
Video from the day shows a tense exchange between the servicemen and people attending the ceremony.
One claims they are members of an alt-right group called "Proud Boys".
That group describes itself on Facebook as "a fraternal organisation of Western Chauvinists who will no longer apologise for creating the modern world".
Among the values they cite are minimal government, anti-political correctness, closed borders, anti-racial guilt, anti-racism, glorifying the entrepreneur and venerating the housewife.
In video from Saturday's confrontation, the servicemen are seen carrying Canada's former "Red Ensign" flag, which includes the Union flag and went out of official use in 1965 after being replaced by the maple leaf design.
One woman is heard asking: "What caused you to feel the need to bring a British flag?"
One man responds: "Because it's a British colony."
"You're recognising your heritage and so are we," another says.
After about 10 minutes, the group leaves the park.
Gen Vance, Canada's Chief of the Defence Staff, said in a statement that he "detests any action by a Canadian Armed Forces member that is intended to show disrespect towards the very people and cultures we value in Canada".
He adds: that any military member "who is not prepared to be the defender we need them to be will face severe consequences, including release from the forces".
Gen Vance said the servicemen involved have been removed from active duty and that their future in the military is in doubt.
A sixth armed forces member was also involved but is not under review, according to a National defence spokesperson.
The First Nations ceremony held by a small group of protesters in Halifax on Canada Day was meant to mark the country's troubled history with indigenous peoples. It was held next to the statue of the city's founder, British military officer Edward Cornwallis.
Halifax is in the midst of a long debate over how the city commemorates Cornwallis, who in 1749 placed a bounty on the scalps of Mi'kmaq people after they rebelled against the British.

The Acting Up report suggests the high audition costs for drama school is one of several factors deterring people from working class backgrounds from entering the arts.
It also recommends more school trips to the theatre to encourage young pupils.
The government said it was a "priority" to ensure everyone can take part in the arts and culture, including in schools.
Former Coronation Street and EastEnders actress Tracy Brabin, who was first elected as an MP in by election last year, led the inquiry along with fellow Labour MP Gloria De Piero, who worked as a presenter on GMTV before becoming a politician.
Ms Brabin, whose best-known role was as single mother Tricia Armstrong in Coronation Street from 1994 to 1997, said there was a "diversity crisis on our stages and screens".
The report, commissioned by Labour's deputy leader Tom Watson, looked at what more could be done at school level to encourage young people to consider a career in the arts.
It said that drama, and other related subjects, are often treated as inferior at some schools - and that students are sometimes discouraged from studying them.
It also suggested there are poor drama facilities, few arts excursions and a lack of awareness about arts career opportunities in many schools.
Ms Brabin, who also worked as a TV writer in her pre-political life, said: "Our performing arts are some of the best in the world and they should represent our whole nation, not just a privileged section of it.
"But the systematic eradication of arts education in schools, sky-high drama school audition fees, chronic low pay and a lack of diversity behind the scenes are all contributing to a diversity crisis on our stages and screens."
The publication of the report follows comments made by presenter Anita Rani last week, who said the pay gap among the BBC's top talent was as much about race and class as gender.
A government spokesman said it was investing Â£300m between 2016 and 2020 to encourage young people to participate in music and arts.
He said: "We are completely committed to ensuring that everyone has the opportunity to take part in arts and culture, including in schools."
Follow us on Facebook, on Twitter @BBCNewsEnts, or on Instagram at bbcnewsents. If you have a story suggestion email entertainment.news@bbc.co.uk.
Five members of the Canadian Armed Forces are facing possible expulsion from the military after crashing an indigenous event on Canada Day.
[NEXT_CONCEPT]
The arts industry is suffering from a "class-shaped hole", a Labour Party inquiry says.