Logistic regression is a statistical model widely used for solving classification problems. Maximum likelihood is used train the model parameters. When data from two classes is linearly separable, maximum likelihood is ill-posed. To address this problem as well as to handle over-fitting issues, regularization is commonly considered. A regularization coefficient is used to control the tradeoff between model complexity and data fit and cross-validation is applied to determine the coefficient. In this paper, we develop a regularization framework for logistic regression using Jeffreys prior, which is free of any tuning parameters. Our experiments show that the proposed regularization outperforms other well-known regularization approaches.
['Tam Nguyen', 'Raviv Raich', 'Phung Lai']
Jeffreys prior regularization for logistic regression