This is a pun on the phrase 'manner of speaking', which is used to say that a statement is true or accurate in a certain way even if it is not literally or completely true. In this case 'manor' is used in place of 'manner' because it sounds similar and 'manor' is another term for a large country house 
A man ' s home is his castle in a manor of speaking .