After James Tavernier was penalised for a last-minute challenge on Jonny Hayes, James Maddison curled home a fantastic set-piece from 25 yards.
"The referee handled a difficult game very well," said Warburton.
"He did a really good job up to that point, but to give that one there is frustrating beyond belief."
Warburton is convinced Tavernier made an "excellent tackle" and confronted referee John Beaton on the pitch at full-time.
"I will be careful what I say about the free-kick," he added. "Everyone saw the ball move. The fourth official on the sidelines saw the ball move and made a comment. And yet a free-kick is given.
"You have got to be, I think, 100% sure in these type of games, in those areas of the park.
"Late in the game, when your players are fatigued, I thought Tav made an excellent tackle there. I don't think their player moaned about it and it is a huge decision to give."
Rangers dominated the first half as the teams met for the first time since January 2012.
However, the visitors were caught cold in the opening minute of the second period when Hayes broke clear to score.
A surging run from Rangers captain Lee Wallace was halted by Hayes, leading to a penalty converted by Andy Halliday.
Maddison's stunning strike left Rangers in the bottom half of the table, with just two wins from their seven league outings, but Warburton insists his team are moving in the right direction.
He said: "We dominated the ball first half completely but we went in at half-time frustrated it finished 0-0.
"We gave away a schoolboy goal and for three or four minutes we looked a bit rocky. We lost our composure then got the ball down, then played. We got our rewards via the penalty and I thought there was only one team in it.
"I am never happy to drop a point, never mind three. But we moved to the next level as a team and we are getting better.
"The boys are settling in and we were delighted with long periods of that game where we completely dominated."

Gwent Police said it was "extremely concerned" after five incidents potentially linked to heroin were reported between 23 October and Friday.
They happened in Caerphilly, Newport and Blaenau Gwent.
Det Insp Leigh Mears said he "cannot rule out the possibility that the deaths and the medical emergencies are linked to the same batch of heroin".
He added: "The use of heroin is of course illegal, but I would urge anyone using the drug to be especially cautious and vigilant.
"If they experience any unusual symptoms after taking drugs, they should seek medical attention immediately."

Margaret Fleming, 36, was last seen at her home in Inverkip by her two carers at about 17:40 on Friday 28 October.
She is described as about 5ft 5in tall, with collar-length black hair, brown eyes and has a heavy build, although her weight can fluctuate.
Police had said they were trying to build a picture of Ms Fleming's life, part of which she kept "quite private".
When last seen, she was wearing a green tartan fleece or jumper, dark jeans or trousers, and dark Karrimor-type boots. She also had a satchel-type handbag.
A police spokesman said: "There is a specialist search team combing the area around where the missing person was last seen, this includes in the garden of her last known address."
Speaking earlier this month, Ch Insp Elliot Brown, area commander for Greenock, said: "We are currently working with limited information and whilst it does seem like this is quite an unusual missing person case, it is challenging.
"As we continue to try to piece together Margaret's life, we understand she kept a side of her life quite private so we're trying to look into this in a bit more detail to see if this will help with our inquiries.
"We are working backwards to try to find out more about Margaret, so we've been speaking to friends and family members to establish a clearer picture of her life over the years."
The detective said that Ms Fleming was a student at James Watt College in Greenock between 1996 and 1997.
He said he was keen to speak to anyone who remembered her from then, and who might have been in touch with her over the years.
Rangers boss Mark Warburton was furious with the award of a free-kick from which Aberdeen scored their winning goal in a 2-1 victory at Pittodrie.
[NEXT_CONCEPT]
Police have issued a warning about heroin after the deaths of three people in five days.
[NEXT_CONCEPT]
Police have searched the home and garden of a vulnerable woman who has been missing for a month in Inverclyde.