Kerr was born in Cape Town, South Africa to Australian parents. He moved with his family to Australia and grew up in Wagga Wagga, New South Wales. He served in the army during World War II. He moved to the United Kingdom in 1947. He was first married to Virginia from 1955 until their divorce. They had two children. He got married a second time to Sandra in 1983. They remained married until Kerr's death. They also had two children.