The joke is a play on words. The phrase 'look into it' typically means to 'investigate or gather info' on something.  One can literally look into a mirror to see their reflection however.
An old lady once asked the dispatcher of a local trucking company if they could ship an antique mirror to her sister in Toronto . The dispatcher says , ' I don ' t know madam , I ' d have to look into it first ' .