The man, who was in his 20s, was rescued by a volunteer crew from Bangor RNLI.
The lifeboat's operations manager said medical personnel told him the man could have died if he had remained in the water for another five minutes.
The alarm was raised by Mark Pollock, who was working in Banks Car Park.
Hearing faint shouts, he initially thought it was someone calling for their dog, but kept looking until he realised there was someone in the sea.
He then called the Coastguard.
Bangor RNLI's helmsman James Gillespie said when they arrived "the early morning light made it difficult to see, but fortunately the water was flat calm, and on scanning the area I saw a slight movement as the casualty raised his hand".
Another crew member, Johnny Gedge, entered and helped the man, who was only just conscious, until he could be lifted on board.
Mr Gillespie said: "Our extensive training in casualty care is invaluable at a time like this.
"Because of this, we know the importance of not trying to warm the patient too quickly, as this can cause cardiac arrest.
"Instead, we made the patient safe, and prevented further cooling, and returned as quickly and safely as we could to the lifeboat station where an ambulance and paramedics were waiting to take over."
The rescued man was wearing only tracksuit bottoms, a t-shirt and socks. It is not clear why he was in the water.
"I am just delighted that I heard his calls, and hope he makes a full recovery," Mr Pollock said.

The region's chief operating officer says the claims of head coach Sean Holley and forwards coach Jonathan Humphreys will also be assessed.
New Zealander Hore, a former Wales fitness coach, said: "We definitely will be looking at both scenarios."
Johnson will become Scotland boss Andy Robinson's senior assistant coach.
Hore said: "We've been talking for a long period of time about where to go from here, that kind of thing.
"What we want to do now is sit down and look at the blueprint.
"We started this exercise four years ago to bring him [Johnson] in to help develop those coaches and that coaching staff and put those systems in place.
Are we happy with where we're at?
"And we've just got to review that now and make a decision on whether we bring somebody else in, that maybe the process needs to be finished.
"Or, are we happy with where we're at and go with from within, as we said?
"And then it's who, from within, is going to head that up?"
Hore admits the festive period has made it difficult for the Osprey's power brokers, such as joint managing directors Roger Blyth and Mike Cuddy, to meet to discuss the issue.
"With some directors being away, obviously with the festive period and that, we just haven't had a chance to sit down because it [the news about Johnson leaving] came out a little bit earlier than we expected," Hore added.
"So yeah, we've got a bit of work to do yet."
The Ospreys stopped a run of three defeats with a 17-12 win over Welsh rivals Cardiff Blues on New Year's Day at the Liberty Stadium.

Reports on Tuesday suggested the 28-year-old's representatives are set for talks with the English side.
"There are no direct offers," the source told BBC Sport.
"Only by meeting the release clause of 94,376,000 euros will the player leave Napoli. Otherwise Higuain will stay for the remaining two years of his deal."
Gareth Bale's £85m move from Tottenham to Real Madrid in 2013 is the most expensive transfer in football history, surpassing the £80m Real paid Manchester United for Cristiano Ronaldo in 2009.
In 2013, then Real Madrid forward Higuain agreed personal terms with Arsenal, but joined Napoli after the Gunners failed to make an offer.
Last season, Higuain won the Capocannoniere - Serie A's golden boot - after scoring a league record 36 goals as Napoli finished runners-up to Juventus.
Meanwhile, Napoli have not received any offers for 25-year-old Senegal centre-back Kalidou Koulibaly, who has been linked with Everton and Chelsea.
A man has been rescued from the sea 200 metres off the coast at Ballyholme Bay, near Bangor, after a man working nearby heard shouts and called the Coastguard.
[NEXT_CONCEPT]
Andrew Hore says the Ospreys will consider finding a successor for director of coaching Scott Johnson, who leaves for Scotland in June.
[NEXT_CONCEPT]
Arsenal must pay Napoli 94m euros (£80m) to sign Argentina striker Gonzalo Higuain, according to a source close to the Italian club.