Information complexity-based regularization parameter selection for solution of ill conditioned inverse problems

We propose an information complexity-based regularization parameter selection method for solution of ill conditioned inverse problems. The regularization parameter is selected to be the minimizer of the Kullback–Leibler (KL) distance between the unknown data-generating distribution and the fitted distribution. The KL distance is approximated by an information complexity criterion developed by Bozdogan. The method is not limited to the white Gaussian noise case. It can be extended to correlated and non-Gaussian noise. It can also account for possible model misspecification. We demonstrate the performance of the proposed method on a test problem from Hansen’s regularization tools. (Some figures in this article are in colour only in the electronic version)

[1]  C. L. Mallows.  Some comments on C_p , 1973 .

[2]  G. Kitagawa,et al.  Generalised information criteria in model selection , 1996 .

[3]  Hamparsum Bozdogan,et al.  Mixture-Model Cluster Analysis Using Model Selection Criteria and a New Informational Measure of Complexity , 1994 .

[4]  David S. Gilliam,et al.  Quantifying information content for ill-posed problems , 1990 .

[5]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[6]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[7]  H. Bozdogan.  On the information-based measure of covariance complexity and its application to the evaluation of multivariate linear models , 1990 .

[8]  A. G. Yagola,et al.  The L-curve method always introduces a nonremovable systematic error , 1997 .

[9]  G. Wahba.  Spline models for observational data , 1990 .

[10]  C. Vogel.  Non-convergence of the L-curve regularization parameter selection method , 1996 .

[11]  D. Cox,et al.  Asymptotic Analysis of Penalized Likelihood and Related Estimators , 1990 .

[12]  H. Bozdogan,et al.  Akaike's Information Criterion and Recent Developments in Information Complexity. , 2000, Journal of mathematical psychology.

[13]  David L. Phillips,et al.  A Technique for the Numerical Solution of Certain Integral Equations of the First Kind , 1962, JACM.

[14]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[15]  A Tikhonov,et al.  Solution of Incorrectly Formulated Problems and the Regularization Method , 1963 .

[16]  C. H. Oh,et al.  Some comments on , 1998 .

[17]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[18]  H. Bozdogan.  Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions , 1987 .

[19]  V. A. Morozov,et al.  Methods for Solving Incorrectly Posed Problems , 1984 .