Midfielder Doughty stabbed home in the 94th minute after Millwall failed to clear a free-kick.
Lee Gregory burst through the defence to score his 22nd goal of the season and poor defending allowed Steve Morison to make it 2-0 to Millwall.
But Nicky Ajose's penalty, after Raphael Rossi Branco was fouled, set up Doughty's dramatic finish.
Millwall remain fifth and have gone five games without defeat, while Swindon - whose unbeaten run now stands at six league games - drop to 14th.
Williams signed a five-year contract in midweek after a successful spell as caretaker boss.
Millwall manager Neil Harris told BBC Radio London:
Media playback is not supported on this device
"Huge frustration, I'm certainly disappointed with the outcome. I thought we were so good for so long with and without the ball.
"I thought we were outstanding in the first half and I think in the second half even though Swindon had all the football. On the counterattack we were really good, we just lacked that little bit of quality and ruthlessness in the final third.
"If you don't see the game off - and it's a very dubious penalty against you, about the softest penalty I've seen for a long time - you're backs-to-the-wall a little bit."

A two-vehicle crash on the A470, between Church Street in Rhayader and the Llangurig roundabout, happened shortly after 11:45 GMT on Friday.
It affected motorists travelling between Builth Wells and Llanidloes before the road reopened by 17:30.
The A483, north of Welshpool, was later shut after a crash involving five vehicles just before 15:00 and the road remained closed into the evening.
Check if this is affecting your journey

Daniel Begg, 21, of Dundee, admitted culpable and reckless behaviour.
The fan who was struck avoided serious injury in the incident at Pittodrie Stadium on Sunday 25 September.
Begg was fined Â£260 at Aberdeen Sheriff Court. The court refused a Crown motion for a football banning order stopping Begg from attending future games.
Rangers fan Begg was in the away end at Pittodrie.
Police had earlier issued a CCTV appeal to trace him.
Michael Doughty earned a draw against Millwall in Swindon's first game since Luke Williams was named manager.
[NEXT_CONCEPT]
Two Powys roads were forced to close after separate serious collisions.
[NEXT_CONCEPT]
A football fan who threw stadium seats at another supporter during an Aberdeen v Rangers game has avoided a ban from matches.