However, his increasing womanising and reckless behaviour in the 1920s and the 1930s worried Prime Minister Stanley Baldwin and his hardworking, strict father, King George V. Edward was also uninterested with royal duties and disliked the British establishment. George V was disappointed by Edward's failure to settle down in life and was furious with his many affairs with married women, and he was reluctant to see him inherit the Crown. George V preferred his son Albert (later King George VI), a family man who was loyal to his beloved wife.