Many aristocratic residents never returned, preferring to take new houses in the West End, where fashionable new districts such as St. James's were built close to the main royal residence, which was Whitehall Palace until it was destroyed by fire in the 1690s, and thereafter St. James's Palace.
Mange beboere fra aristokratiet vendte aldrig tilbage, da de foretrak nye huse i West End, hvor fashionable nye distrikter som St. James's blev opført tæt på den primære kongelige residens, der var Whitehall Palace, indtil det blev ødelagt af en brand i 1690'erne, og derefter St. James's Palace.