The Washburnes had seven children including sons Gratiot, Hempstead, William P., and Elihu B. Jr., and daughters Susan and Marie L. The Washburnes were married for 42 years, which ended with Washburne's death.