Born in New Haven, Connecticut, Ingersoll was the son of Jared Ingersoll (1722–1781), a prominent British official whose strong Loyalist sentiments would lead to his being tarred and feathered by radical Patriots.