Goff was born in Los Angeles, California, on June 20, 1931, to Samuel J. Goff and Elizabeth A. Wilhelm Goff and graduated from Inglewood High School in Inglewood, California, in June 1949, after which he attended Pepperdine College on its campus in Los Angeles.