George Henry Hoyt was born in Athol, Massachusetts, on November 25, 1837. He was the only surviving son of Athol physician and abolitionist George Hoyt and his wife Avelina Witt Hoyt. In 1851, the Hoyts moved to Boston, where George studied law. Lysander Spooner, abolitionist anarchist and good friend of Dr. Hoyt, strongly influenced young George's views of abolition, as did radical orator Wendell Phillips.