Elizabeth Seymour's son and heir was Charles Bruce, 3rd Earl of Ailesbury (died 1747), of Houghton House in the parish of Maulden, in Bedfordshire, who in 1721 rebuilt Totnam Lodge to the design of his brother-in-law the pioneering Palladian architect Lord Burlington.