He made a whip of cords, and threw all out of the temple, both the sheep and the oxen; and he poured out the changers' money, and overthrew their tables.

 There is no mention of Joseph. The explanation most probably is that by this time Joseph was dead. It would seem that Joseph died quite soon, and that the reason why Jesus spent eighteen long years in Nazareth was that he had to take upon himself the support of his mother and his family. It was only when his younger brothers and sisters were able to look after themselves that he left home. 