A fast algorithm for optimizing ridge parameters in a generalized ridge regression by minimizing a model selection criterion

Abstract In this paper, we deal with the optimization method of ridge parameters in a generalized ridge regression by minimizing a model selection criterion (MSC). The optimization methods based on minimizations of generalized C p criterion and GCV criterion are fast because the minimizers of the two criteria can be derived as closed forms. Although the methods are fast, they do not work well when the number of explanatory variables is larger than the sample size. Methods based on minimizations of other MSCs may work well even if the number of explanatory variables is larger than the sample size and are not fast because the minimizers of the MSCs have not been derived as closed forms. Even though a minimization problem of MSC cannot be solved analytically, we propose a fast optimization algorithm to minimize MSC by specifying the small number of minimizer candidates. By conducting numerical examinations, we compare performances of the optimization methods based on the minimizations of MSCs.

[1]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[2]  Bhramar Mukherjee,et al.  A Small-Sample Choice of the Tuning Parameter in Ridge Regression. , 2015, Statistica Sinica.

[3]  R. Nishii Asymptotic Properties of Criteria for Selection of Variables in Multiple Regression , 1984 .

[4]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[5]  宏和 柳原,et al.  多変量一般化リッジ回帰におけるリッジパラメータ最適化のためのバイアス補正 Cp 規準 , 2009 .

[6]  Yuanyuan Shen,et al.  Multilocus association mapping using generalized ridge logistic regression , 2011, BMC Bioinformatics.

[7]  Moudud Alam,et al.  A Novel Generalized Ridge Regression Method for Quantitative Genetics , 2013, Genetics.

[8]  A. Atkinson A note on the generalized information criterion for choice of a model , 1980 .

[9]  D. Harville Matrix Algebra From a Statistician's Perspective , 1998 .

[10]  B. G. Quinn,et al.  The determination of the order of an autoregression , 1979 .

[11]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[12]  J. F. Lawless,et al.  Mean Squared Error Properties of Generalized Ridge Estimators , 1981 .

[13]  C. L. Mallows Some comments on C_p , 1973 .

[14]  Colin L. Mallows,et al.  Some Comments on Cp , 2000, Technometrics.

[15]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[16]  Stephen G. Walker,et al.  Generalized ridge regression and a generalization of the CP statistic , 2001 .

[17]  Clifford M. Hurvich,et al.  Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion , 1998 .

[18]  K. Satoh,et al.  Optimization of ridge parameters in multivariate generalized ridge regression by plug-in methods , 2012 .

[19]  Jianming Ye On Measuring and Correcting the Effects of Data Mining and Model Selection , 1998 .

[20]  Y. Fujikoshi,et al.  Modified AIC and Cp in multivariate linear regression , 1997 .

[21]  Clifford M. Hurvich,et al.  Regression and time series model selection in small samples , 1989 .

[22]  H. Yanagihara Explicit solution to the minimization problem of generalized cross-validation criterion for selecting ridge parameters in generalized ridge regression , 2018, Hiroshima Mathematical Journal.