After the Norman Conquest Robert de Montgomery ordered the building of a motte-and-bailey castle on what is now called St Ann's Hill, a strategic bluff on a curve of the River Rother, overlooking the cross-roads and a long stretch of Rother Valley to the north, east and the west, protecting the River Rother crossing.