The Earl's own house, then known as Southampton House and later as Bedford House after the square and the rest of the Bloomsbury Estate passed by marriage from the Earls of Southampton to the Dukes of Bedford, occupied the whole of the north side of the square, where Bedford Place is now located.