I think the joke is a pun.  'Hardest' can mean the most difficult, but it could also mean physically hard because it is a large rock that is being talked about.
My friend asked me to carve a sculpture out of a large rock . It was the hardest thing I ' ve ever had to make !