But the former Scotland midfielder insists that he can turn the Cumbernauld outfit around despite only one win in 12 games.
"Of course I can," the 39-year-old manager told BBC Scotland.
"But, if we have any aspirations about getting into the play-offs, we need to win on Saturday."
With Forfar Athletic eight points clear of Arbroath at the top, Clyde slipped 15 points behind third-placed Elgin City when they were trounced 4-1 away to the Moray outfit on Saturday.
The Bully Wee are also 11 points behind this weekend's hosts, who sit in the final play-off spot.
Ferguson's side do have three games in hand over Annan, but Montrose and Berwick Rangers have also overtaken Clyde during their run of nine league games without a win.
"It is tough," admitted Ferguson, who previously had a spell as caretaker manager with Blackpool. "Listen, it's easy to walk away, but I am there fighting and we'll get into the play-offs."
Clyde have played fewer games as a result of a run to the fifth round of the Scottish Cup that ended last midweek after they lost 2-1 in extra-time at home to Championship outfit Ayr United.
"We had a long game on Tuesday night - went into 120 minutes and into extra-time, but I'm not going to use it as an excuse," said the Bully Wee boss, whose side were praised for their display against the full-time visitors.
"We try to do everything properly, the recovery - we went and got protein shakes, recovery shakes, we took them up the night before, which we normally don't do because it is a bit of a cost to the club.
"But it just wasn't good enough. I'm not going to mix my words. I told them after the game."
Ferguson, who had a long playing career in the top flight in Scotland and England, has often expressed his frustration at managing part-time players.
"On Tuesday night, Ayr United scored the winner in extra-time," he said. "I had four guys who were going straight from that game to night shift for eight or nine-hour shifts.
"I take my hat off to them as a realise now how difficult it is.
"But that's why we do a pre-season and we should should be fit enough to play three games in a week."

Chase signed for Tigers until the end of the season on Saturday, having left Championship side Leigh in May.
The 30-year-old played for Tigers between 2009 and 2013, before leaving to join Salford Red Devils.
"I'm not match fit because I've had some time off, but mentally I'm in the best shape I've ever been," he said.
"Physically I've got a bit of work to do but you can get through anything if you're mentally right. There's some hard work to do, I'm aware of that, but I'm looking forward to it."
Chase also praised the Rugby Football League (RFL) for the support they offer players who are struggling with mental health-related issues.
"It's massive and it's a credit to the RFL that they touch on it because a lot of the boys struggle with it," Chase said.
"People should be aware of it because a lot of us boys have to go through a lot of things and we put on this bravado that we're OK. It's not something that people like talking openly about because you feel weak and vulnerable.
"But really we go home and we feel alone. It's good to get the message out there that it does happen in our game, a lot of people do suffer from depression.
"When you are in that dark place, you feel weak and vulnerable, you just have a lot of down days and it's not nice. But when you come through it, it's an unbelievable feeling. You feel strong."
Clyde's Barry Ferguson admits it is make or break for his side's promotion play-off chances when they travel to face Annan Athletic on Saturday.
[NEXT_CONCEPT]
Former England half-back Rangi Chase says he is "back in love" with rugby league after rejoining Castleford following a fight with depression.