Born into the Villiers family as Barbara Villiers, in the parish of St. Margaret's, Westminster, Middlesex, she was the only child of William Villiers, 2nd Viscount Grandison, a half-nephew of the 1st Duke of Buckingham, and of his wife Mary Bayning, co-heiress of Paul Bayning, 1st Viscount Bayning.