Runge spent the first few years of his life in Havana, where his father Julius Runge was the Danish consul. His mother was Fanny Schwartz Tolmé. The family later moved to Bremen, where his father died early (in 1864).
In 1880, he received his Ph.D. in mathematics at Berlin, where he studied under Karl Weierstrass.  In 1886, he became a professor at the Technische Hochschule Hannover in Hanover, Germany.
His interests included mathematics, spectroscopy, geodesy, and astrophysics.  In addition to pure mathematics, he did  experimental work studying spectral lines of various elements (together with Heinrich Kayser), and was very interested in the application of this work to astronomical spectroscopy.
In 1904, on the initiative of Felix Klein he received a call to the University of Göttingen, which he accepted. There he remained until his retirement in 1925.