The video, filmed by Smith and obtained by a newspaper, appears to show him laughing while retired gymnast Luke Carson mimics Islamic prayer practices.
Smith, who won pommel horse silver at Rio 2016, later said he was "deeply sorry" for his "thoughtless actions".
British Gymnastics said: "We will be investigating the behaviours reported."
In a statement, the governing body added: "Members who break our code of conduct can face suspension or expulsion from our organisation."
Smith, who has won medals at the past three Olympic Games, has already been censured twice this year by British Gymnastics.
In April, the 27-year-old apologised for questioning the judging at the British Championships, where he was beaten to pommel gold by Max Whitlock.
He was also reprimanded in June for posting an image on social media of an American gymnast, who was 16 when the photograph was taken, accompanied by a comment British Gymnastics said was "unbefitting to a participant".
That reprimand was to remain on his record for two years with the added warning "any further misconduct may lead to even greater consequences".
Smith is one of Britain's best-known gymnasts and won the BBC show Strictly Come Dancing in 2012.
He is taking a break from the sport and is instead touring as a guest celebrity on the Keep Dancing stage production.

A witness described seeing a "flash right across the sky" as the aircraft came down in a field off Burrows Lane, in the Middle Stoke area.
The condition of the pilot is not known.
The Air Accidents Investigation Branch (AAIB) said it had deployed to an accident involving a light aircraft near Stoke Medway Airfield.
Representatives from UK Power Networks and fire crews also attended the scene.
Louise Holloway, who lives nearby, said: "I was outside in the garden and heard this huge bang.
"There was a massive orange flash right across the sky.
"One man ran across the field but there was no way the pilot could have got out. It's really sad."

Four were treated for smoke inhalation and one was taken to hospital with burns, Devon and Somerset Fire and Rescue Service said.
The gorse fire, which began at about 13:45 BST, has engulfed a large area of dry heath-land on Woodbury Common.
Twenty fire engine crews were sent to tackle the flames that were fanned by high winds.
The fire is being fought on a number of fronts and a police helicopter has been deployed to help survey the scene.
Chief Fire Officer Lee Howell said it was a "very difficult incident" and his teams were struggling with wind direction and a lack of water supplies.
He said: "It's a very, very large area - at the moment it's difficult to quantify and there is a fire front on a number of different parts of the common - and so our resources are being targeted to bring that under control in a number of different areas."
He added that firefighters will be at the scene overnight and into the early hours of Monday morning.
The nearby Woodbury Park Golf and Country Club has been evacuated as a precaution.
Smoke can be seen for miles around and the public have been advised to avoid the area.

The Reds, who are fifth in National Two South, beat third-placed Chinnor 29-12 for a second win in a row.
"Tommy Phillips, Ben Priddey and Craig Williams really led the team from the front," Churcher told BBC Cornwall.
"If your front row and front five are making 70-plus tackles in a game you're going to be doing well and that's what happened against Chinnor."
The Reds face seven teams who are below them in the table in their next eight games, as well as league leaders Old Elthamians.
Churcher believes his side still has a lot of room for improvement, adding: "Chinnor wasn't the complete performance, but emotionally it was a really dogged performance.
"We've got to build on what we did last weekend, because if we don't, last weekend will have been a waste of time."
Four-time Olympic medallist Louis Smith may face "suspension or expulsion" by British Gymnastics over a video in which he apparently mocks Islam.
[NEXT_CONCEPT]
A microlight has hit an electricity pylon and burst into flames near Rochester in Kent.
[NEXT_CONCEPT]
Five firefighters have been injured while tackling a blaze on common land near Exeter.
[NEXT_CONCEPT]
Redruth boss Marek Churcher says his side's front row have been key to the club's good form.