He was born Edward Albert Schäfer, in Hornsey in London, the third son of Jessie Brown and James William Henry Schäfer, a merchant born in Hamburg, who had come to England as a young man, and was a naturalised citizen.