The officer was carrying a handgun in the railway station in Inverness on Tuesday as he walked through the main concourse.
Police Scotland said the officer had breached new force policy.
A spokesman said he had been reminded of the strict criteria about the issue.
It was a sight more common in 2013 when a policy was introduced allowing firearms officers to carry handguns with them at all times while on duty.
However, after concerns from politicians, the chief constable of Police Scotland, Sir Stephen House, stated last October that specialist armed police officers in Scotland would only be deployed to firearms incidents or where there was a threat to life.

Ryan Thomas McManus, 31, from Ava Street in Belfast and Ryan Milligan, 29, from Manse Road in Dundrum were given seven-year terms for over 40 drugs offences.
They will serve half their terms in prison and half on licence.
A third man on charges died on Tuesday.
Connor Devenney, 36, from Dunwellan Park in Newcastle, County Down, appeared in court last week, when the Crown set out its case against the three men.
However, a judge was told that Devenney, who faced two charges linked to the operation, died on Tuesday night.
Passing sentence on McManus and Milligan, the judge said the pair were involved in a "widespread supply of a quantity of illicit drugs" that they distributed "both in the UK and elsewhere".
He said that on one level there was a degree of sophistication, due to the use of the internet and Bitcoin currency, but on another there was a lack of sophistication as they left a "forensic trail".
He said this trail allowed both the police and other authorities to "become aware of the criminal activity" that ultimately led to their arrest.
Last week, the court heard that Milligan and McManus were arrested on 5 August, 2013, when several properties they were linked to were searched and drugs were seized.
A crown prosecutor said that during the period of offending - which spanned from January 2012 to August 2013 - the pair received consignments of drugs that were delivered to addresses either occupied or controlled by them.
They also used a virtual mailbox facility in the centre of Belfast.
The court heard some of the drugs were purchased online using Bitcoin currency. Drugs seized included more than 4,000 Class A tablets, and over 2,700 doses of DPP, as well as cocaine and over 100 grams of herbals.

Olive Cooke, 92, of Bristol, joined the Royal British Legion (RBL) aged 16 following her father who was an active member after serving in World War One.
Mrs Cooke said selling poppies took on a whole new meaning after her first husband was killed in action in 1943.
The RBL said it could not be exact but that Mrs Cooke was "one of our longest-standing poppy collectors".
For the past 12 years Mrs Cooke has sold poppies from the entrance to Bristol Cathedral.
She said she wished everyone would wear one with "great pride" to remember those who died in both world wars.
She said although she had only been married two-and-a-half years when her first husband Leslie Hussey-Yeo was killed, "he is in my memory every day".
"You never forget the lost people and he was such a good man you wouldn't want to," said Mrs Cooke.
"His words to me were that if anything were to happen to him, to always wear a smile and I've never got bitter. I never thought bitterness helped.
"I think of my husband, cousins and many of my school friends who were killed aged 18, all young men that we were at school with, so all of those faces come back to me."
Col Brian John, president of the City of Bristol group of RBL branches, said Mrs Cooke was their "most-valued poppy collector".
Sue Potepa, from the RBL in Bristol, said she was "well-loved" by the team.
"This year 350,000 volunteers like Olive will help us to ensure that the poppy is available to all those who wish to wear one and we are extremely grateful to her for the support she has given the Royal British Legion and our annual Poppy Appeal for more than 75 years," she said.
A police officer in Inverness has been photographed carrying a firearm despite assurances that sidearms would only be deployed under special circumstances.
[NEXT_CONCEPT]
Two men involved in an international drugs operation that included buying illegal substances on the 'dark web' using Bitcoin currency have been jailed.
[NEXT_CONCEPT]
A charity volunteer who has clocked up 76 years of service is one of the UK's longest-serving poppy sellers.