Bronfman was married to Ann Loeb from 1953 until they divorced in 1973. Then he was married to Lady Carolyn Townshend from 1973 until they divorced in 1974. Then he was married to Rita "Georgiana" Webb until they divorced. Then he was married to Jan Aronson from 1994 until his death in 2013. Bronfman has seven children.