Lady Charlotte was born on 19 May 1812 at Uffington House in Uffington, Lincolnshire, the daughter of Albemarle Bertie, 9th Earl of Lindsey (1744–1818), and his second wife Charlotte Susanna Elizabeth Layard.