Pepys never remarried .
Pepys did have a long - term housekeeper named Mary Skinner who was assumed by many of Pepys ' contemporaries to Pepys ' his mistress and sometimes referred to as Mrs. Pepys .
Pepys never remarried , but he did have a long - term housekeeper named Mary Skinner who was assumed by many of his contemporaries to be his mistress and sometimes referred to as Mrs. Pepys .