Charles Juste was born at the Hôtel de Craon, Lunéville in the capital of the Duchy of Lorraine. His family were the most powerful in Lorraine after the ruling Duke of Lorraine. His mother, Anne Marguerite de Ligneville, was the mistress of Leopold, Duke of Lorraine, husband of Élisabeth Charlotte d'Orléans (niece of Louis XIV). He was the twelfth of twenty children.