Bannerman resigned as governor in 1864 and returned to England where he caught a cold and, in his enfeebled state, fell down a flight of stairs causing his death on 30 December in Mayfair, London aged 76.