In 1221 Cantilupe built a castle at Eaton, which became the "caput" of his feudal barony and was described by the monks of nearby Dunstable Priory in the "Annals of Dunstable" as being "a serious danger to Dunstable and the neighbourhood".