Although he did not agree with all the party's principles, he would not associate with the Democrats, the party he had spent much of his career denouncing, nor would he associate with the new Republican Party because of their stance against slavery.