The League One side are hoping to construct a new stadium at Mill Hill in Medway, opposite their training ground.
The Gills have been based at their current 11,582-seater Priestfield ground in Medway since 1893.
"When I came here in 1995 I realised that there was no long-term future for this club on this site," Scally said.
"In 1999, I put out a document, 'The Path to Premier League Football', which some may have smiled at, but those that smiled then are probably not smiling now if you look at examples of other clubs that have succeeded with lesser grounds or lesser conurbations."
Gillingham, who currently have an average home attendance of 5,000-6,000, are 18th in League One, four points above the relegation zone.
They have not been in the second tier of English football since being relegated in 2005.
"Bournemouth comes to mind as the most recent example. But Brighton's also a very good case study," Scally told BBC Radio Kent.
"If you build a new stadium that is fit for purpose as far as sustainability is concerned, and you make it of a size capable of hosting Premier League football, and you have the right investment into the club and into the project, then the two in parallel can actually take you to the heady heights of the Premier League and beyond.
"I'm not saying that if you build a stadium that guarantees you Premier League status, of course, but what I am saying is that if you don't build a stadium, you really can't sustain top Championship or Premier League football in a stadium such as we have now."

A spokesman for the Judicial Office said Nick Harrington had resigned with immediate effect.
The anti-Irish tweet appeared on the Warwick district councillor's now deleted account during the Eurovision Song Contest but was widely retweeted.
Other tweets on the feed which "may have caused concern" are also being looked at, Warwickshire Police said.
Police confirmed on Sunday they were investigating the tweet, which contained offensive and racist language, and are treating it as a suspected hate crime.
More from Coventry and Warwickshire
On Monday, a spokesman said officers were aware of other tweets, posted on the same account, which may have caused concern.
"These will be considered during the investigation," the spokesman added.
The leader of Warwick District Council, Andrew Mobbs, said he was also aware of other comments and it was up to a formal standards committee to purse all avenues before coming to any conclusions.
Mr Mobbs had already suspended Mr Harrington, who represents the Stoneleigh and Cubbington ward, from the Conservatives on the district council for six months saying his behaviour was unacceptable.
The Judicial Office said magistrates are required by the terms of their declaration and undertaking to inform their bench chairman and justices' clerk if they are subject to an investigation by the police.
An advisory committee will then consider whether a conduct investigation is required.
The national Conservative Party has suspended Mr Harrington, pending an investigation.
The district council has said it will wait for the police investigation to conclude before deciding what, if any, action to take.
Gillingham chairman Paul Scally says a new stadium could allow the club to sustain a place in the top two divisions of English football.
[NEXT_CONCEPT]
A Tory councillor being investigated over a racist tweet which appeared on his account has quit as a magistrate.