From 1953 to 1973, Forsyth was married to Penny Calvert. They had three daughters: Debbie, Julie and Laura. According to his autobiography, he dated Miss World 1964 Ann Sidney during her reign. In a 2009 interview, Kathy Kirby said she was Forsyth's lover and that he proposed marriage. From 1973 to 1979, he was married to Anthea Redfern, who was the hostess on "The Generation Game". They had two daughters: Charlotte and Louisa. He fell in love with his fellow judge, Wilnelia Merced, at the 1980 Miss World competition. They married in 1983, and had one son together, Jonathan Joseph Forsyth Johnson (born 1987). By his six children, he had eight grandchildren and three great-grandchildren.