The joke is a pun.  The word 'summarily' means to say something in summary of something.  The word has 'sum' in it, and a sum is the result of an addition equation in mathematics.
I ' m a mathematician , Tom added summarily .