Bomb disposal officers were sent to Birmingham's Lee Bank area on Friday and nearby roads were closed and cordoned off.
The men, two aged 32 and 37 were arrested in Stoke-on-Trent and three others, aged 18, 24 and 28, were arrested in Birmingham.
Magistrates granted police a further seven days to question the men.
Detectives from the West Midlands Counter Terrorism Unit appeared before London's Westminster Magistrates' Court via video link on Saturday to submit their request for a warrant of further detention, a spokesman said.
The men were arrested on suspicion of being concerned in the commission, preparation or instigation of acts of terrorism.
Ch Supt Sue Southern, from West Midlands Police, said: "The arrests of the five men were intelligence led and our investigation continues at full pace."
A number of properties in the Stoke and Birmingham areas have been searched as part of the investigation.
The BBC understands the force was dealing with two suspect devices that were found at a business in Lee Bank on Friday.

NHS prescriptions have risen steadily in recent years, despite awareness of the risks.
Nearly 49,000 items were dispensed in England in 2016.
In 2014 the figure was about 37,000, and in 2012 it was 27,000, NHS Digital data shows.
Experts say the drug, which can also be given as a gel, is extremely effective and appropriate to use, as long as patients are carefully monitored.
Millie Kieve, founder of the campaign group the Adverse Psychiatric Reactions Information Link, has told The Times newspaper that Roaccutane is being overused.
The British Association of Dermatologists said as long as safety recommendations were being adhered to then the increasing number of prescriptions was not a cause for concern.
The drug carries a warning of side-effects including depression and mood changes.
Medicines watchdog the MHRA issued a warning to doctors about the potential for psychiatric disorders such as depression, anxiety, and in rare cases suicidal thoughts, at the end of 2014.
It followed some high-profile reports of young people who had killed themselves.
The parents of a man who took his own life in 2012 have called for further assessments of its risks.
Heather and David Roberts say their son Olly, 32, suffered from anxiety and depression for 11 years after he took a course of Roaccutane.
Manufacturer Roche said: "Information provided with isotretinoin [Roaccutane] carries a warning that some patients may experience mood changes, including an increase in depression."
Experts advise:
About 80% of adolescents will have acne for some period, and it can occur in adults too.
A spokesman from the British Association of Dermatologists said: "Isotretinoin [Roaccutane] is a very effective treatment for acne and has hugely improved many peoples' quality of life, particularly as the psychological burden of acne can be enormous for some people.
"Because it is a very potent drug it is vital that it is prescribed by experts, that patients are informed about the potential risks and closely monitored, and that the guidelines for its use are carefully followed. As long as these recommendations are adhered to then the increasing number of prescriptions for isotretinoin is not, in itself, a cause for concern."
Police have been given more time to question five men arrested on suspicion of terrorism offences.
[NEXT_CONCEPT]
Campaigners fear too many young people are being put on the powerful anti-acne drug Roaccutane, which has been linked with suicidal feelings.