Smith married Martha "née" Hunt (1826–1898) in London in 1848 and with her had seven children: Thomas Smith (1849–1928); Henry John Smith (1850–1889); John Smith (1852–1853); Walter Smith (1854–1923); twins Emanuel Smith (1857–1857) and Martha Smith (1857–1939); Priscella Smith (1858–1929), and Francis (Frank) Smith (1860–1878).