Carson, a long-time smoker, died in 2005 of emphysema. He was married to Linda Carson, an anchorwoman at Sarasota television station WWSB. His daughter Cathi Carson is the sports reporter at two Jacksonville stations in Jacksonville WTEV and WAWS and was formerly a reporter at WWSB. He also had a son, Cliff, and a daughter, Dana, as well as a stepson, Gary Ford.