Metcalfe was born on 30 January 1785 in Lecture House, Calcutta, Bengal Presidency, India, the second son of Thomas Metcalfe, then a major in the Bengal Army, who first went to India in 1767 as a cadet in the King's Army, afterwards became a Baronet, MP, and Director of the British East India Company (1789–1812), and was created a baronet on 21 December 1802.