Lengkepan kalimah di handap ieu: Dina 1813, Ruffin nikah Susan Hutchings Travis of Williamsburg. Pasangan éta pindah ka perkebunan anu diwariskeun ku Ruffin ti aki na, di Coggin's Point, sapanjang Walungan James di Prince George County, anu dicatet salaku titik pamitan Benedict Arnold saatos anjeunna ngalihkeun kasatiaan sareng nyerang sapanjang Walungan James (kalebet Richmond) di 1781. Maranéhanana boga sabelas anak saméméh Susan Ruffin maot dina 1846, sanajan lolobana bakal maot saméméh Perang Sipil Amérika réngsé jeung bapana bunuh diri: Edmund Ruffin Jr. (1814-1875), Agnes Ruffin Beckwith (1816<unk>1865), Julian Calx Ruffin (1821<unk>1864, anu maot di
Perang Drewry's Bluff), Elizabeth Ruffin Sayre (1821-1860), Rebecca Ruffin Bland (1823-1855), Jane Ruffin Dupuy (1829-1855), Charles Lorraine Ruffin (1832-1870; lulusan VMI anu salamet perang), Ella Ruffin (1832-1855), Ann Ruffin (1844-1863) jeung George Champion Ruffin (1845-1913).