Hi -- is there any analogue or adjustment of, say, Schwartz Bayesian (or other) information criterion that would be applicable to model selection in ridge regression with a given ridge parameter $\eta$, i.e. $\hat{\beta}\_\eta = (X'X+\eta I)^{-1}X'Y$?


Thanks!!



Information criteria for ridge regression
The ridge estimator corresponds to the posterior mean under a Normal linear regression model with a conjugate Normal-inverse-gamma prior on the regression coefficients: $\beta \mid \sigma^2, \lambda \sim \mbox{N}(0, \lambda^{-1}\sigma^2 \mbox{I})$ and $\sigma^2 \sim \mbox{IG}(a,b)$ for known hyperparameters $a$ and $b$. One may additionally put a prior distribution over $\lambda$. If you consider a discrete number of possible values for $\lambda$ then one may compute posterior probabilities for each of these values or compute Bayes factors to compare different values.


As BIC and AIC and other such "information criterions" can be viewed as approximations to Bayes factors, this may answer your question. Usually, as you probably know, one simply checks prediction error for the different values via cross-validation (at least in prediction contexts) and selects lambda that way. 


