The raiders targeted the Central England Co-operative property on Chesterfield Road in Duckmanton at about 04:00 BST.
The owner of nearby business WB Motors, Ian Womack, said the raiders had torn down the front of the shop.
Mr Womack said: "It looks like they devastated the property to be fair - the whole front has fallen down."
He said: "There's a large piece of machinery - a JCB - was taken from a building site around the corner - and half the shop front has been pulled down by the JCB to take out the cash machine.
"It will be weeks before it is sorted out. It was not just the cash machine that was stolen - the whole front wall was pulled out."
Mr Womack said the Co-op was cordoned off by police when he arrived at work, adding that CCTV images from his forecourt "would probably identify the culprits".
He said an abandoned vehicle found nearby would also provide "some interesting evidence".
A Co-op spokesman said: "While the building is structurally sound, at present we cannot say when we will be able to open the store again for trading."
The culprits fled the scene in a white Vauxhall van and police are appealing for witnesses to contact them.
The cash machine and van were found later by officers near Buttermilk Lane, police said.
A police spokesman said several vehicles were involved, but officers were "still trying to piece together exactly what happened".

Four steel columns need to be installed at Scarthin Books, in Cromford, Derbyshire, to support the weight of about 100,000 books over four floors.
A crowd funding appeal began on Sunday and more than £5,000 has been raised.
Manager David Booker said building work was put off in favour of restocking during the shop's 41 years.
The shop, which has four floors and a vegetarian cafe, was dubbed one of the best in the world by The Guardian, in 2008.
It was also used as a set for the film When Did You Last See Your Father, in 2007.
However, a number of cracks have appeared on the walls and the staircases between floors are bowing.
Mr Booker said customers have often questioned how the bookshop "has stayed upright" and after a survey by a structural engineer, fears were realised.
"He said we needed to do some fairly sharpish structural work," said Mr Booker.
"It's a bill we could do without as an independent bookshop with tight financial constraints."
He said despite the costs of its upkeep moving out of the building had never been a consideration.
"The building itself is more than half of the overall character of Scarthin itself... people travel from far and wide.
"It's our priority to make sure this place stays standing for another 40 years."
The appeal has raised £5,510 in three days by the shop's customers who will receive a number of rewards in return.

The Boeing 767 bound for Miami experienced an "uncontained engine failure", officials involved in the investigation said.
The pilot aborted the take-off and evacuated everyone on board via emergency chutes as black smoke billowed from the plane.
Twenty people suffered minor injuries, the fire department said.
Nine crew members and 161 passengers were on board.
A federal official quoted by the Associated Press news agency said the plane appeared to have suffered a rare and serious type of engine failure in which parts break off and are spewed outside the engine.
Passengers reported an explosion followed by flames and black smoke as the plane was speeding down the runway.
Sarah Ahmed said everyone on the right side of the plane rushed from their seats and moved to the left side.
"People are yelling, 'Open the door! Open the door!' Everyone's screaming and jumping on top of each other to open the door," she told WLS-TV.
"Within that time, I think it was seven seconds, there was smoke in the plane and the fire is right up against the windows, and it's melting the windows."
The incident took place at 14:35 local time (19:35 GMT).
An initial statement by Federal Aviation Administration (FAA) said the plane had burst a tyre, but that information was later deleted from the statement.
Why do people grab their bags after a plane crash?
American Airlines said passengers who still wanted to travel had been put on another flight to Miami.
Elsewhere, a FedEx plane caught fire at Fort Lauderdale, Florida, airport after its landing gear collapsed on landing, the FAA said.
Thieves used a JCB to smash into the front of a filling station store during a cash machine raid in Derbyshire.
[NEXT_CONCEPT]
A bookshop once named as one of the best in the world could fall down unless £10,000 can be raised for structural repairs, say its owners.
[NEXT_CONCEPT]
An American Airlines plane has caught fire on the runway of Chicago's O'Hare airport while taking off.