Pyddoke was the elder of two sons (he also had a younger sister) of Henry Whately Pyddoke, of Bonnyrigg, Tonbridge, Kent, formerly of Oxhill, Loughton, Essex, and Edith, daughter of Major John Wilson, of the Scots Greys.