Representative Jim Saxton declined to be on the show .
Colbert interviewed his challenger instead ( defeated ) .
Representative Jim Saxton declined to be on the show , so Colbert interviewed his challenger instead ( defeated ) .