论文信息 - ON THE METHOD OF PENALIZATION

ON THE METHOD OF PENALIZATION

In this article, we study convergence properties of the method of penal- ization and related estimates. A penalized estimate is defined as an optimizer of a scaled criterion with a penalty that penalizes undesirable properties of the parame- ters. We develop some exponential probability bounds for the penalized likelihood ratios with a general penalty. Based on these inequalities, rates of convergence of the penalized estimates can be quantified. When convergence is measured by the Hellinger distance, the rate of convergence of the penalized maximum likelihood estimate depends only on the size of the parameter space and the penalization co- efficient. We also explore the role of penalty in the penalization process, especially its relationship with the convergence properties and its connection with Bayesian analysis. We illustrate the theory by several examples.

Xiaotong Shen

[1] A. Kolmogorov,et al. Entropy and "-capacity of sets in func-tional spaces , 1961 .

[2] I. J. Schoenberg,et al. SPLINE FUNCTIONS AND THE PROBLEM OF GRADUATION , 1964 .

[3] M. Birman,et al. PIECEWISE-POLYNOMIAL APPROXIMATIONS OF FUNCTIONS OF THE CLASSES $ W_{p}^{\alpha}$ , 1967 .

[4] C. J. Stone,et al. Optimal Global Rates of Convergence for Nonparametric Regression , 1982 .

[5] B. Silverman,et al. On the Estimation of a Probability Density Function by the Maximum Penalized Likelihood Method , 1982 .

[6] D. Pollard. Convergence of stochastic processes , 1984 .

[7] S. Geer. Estimating a Regression Function , 1990 .

[8] G. Wahba. Spline models for observational data , 1990 .

[9] D. Cox,et al. Asymptotic Analysis of Penalized Likelihood and Related Estimators , 1990 .

[10] Zehua Chen. Interaction Spline Models and Their Convergence Rates , 1991 .

[11] Chong Gu,et al. Smoothing spline density estimation: theory , 1993 .

[12] George G. Lorentz,et al. Constructive Approximation , 1993, Grundlehren der mathematischen Wissenschaften.

[13] D. Cox. An Analysis of Bayesian Inference for Nonparametric Regression , 1993 .

[14] Pin T. Ng,et al. Quantile smoothing splines , 1994 .

[15] W. Wong,et al. Convergence Rate of Sieve Estimates , 1994 .

[16] W. Wong,et al. Probability inequalities for likelihood ratios and convergence rates of sieve MLEs , 1995 .

[17] W. Silverman. BY THE MAXIMUM PENALIZED LIKELIHOOD METHOD , .