Indirect measurements: combining parameter selection with ridge regression

Indirect measurements often require the solution of inverse problems, which can be ill-posed or ill-conditioned. There are several methods for dealing with ill-conditioned problems. Two of them, parameter selection and ridge regression, are described in the paper. The idea of parameter selection consists of estimation of the subset of model parameters selected on the ground of sensitivity analysis and ascribing `typical' values to the others. Ridge regression is a regularization technique reducing collinearities in the matrix being inverted. The methods have been combined to maximize their advantages and improve the accuracy of measurement. Statistical analysis of parameter estimators, including their bias, variance and total uncertainty, is presented together with a simple artificial as well as a complex realistic example. The regularized estimation of selected parameters proposed in the paper is characterized by an improvement in the numerical conditioning of model identification and error reduction in indirect measurement.

[1]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[2]  C. W. Groetsch,et al.  The theory of Tikhonov regularization for Fredholm equations of the first kind , 1984 .

[3]  G. C. McDonald,et al.  A Monte Carlo Evaluation of Some Ridge-Type Estimators , 1975 .

[4]  A. E. Hoerl,et al.  Ridge Regression: Applications to Nonorthogonal Problems , 1970 .

[5]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[6]  John Skilling,et al.  Data analysis : a Bayesian tutorial , 1996 .

[7]  Tor Arne Johansen,et al.  Identification of non-linear systems using empirical data and prior knowledge - an optimization approach , 1996, Autom..

[8]  Donald W. Marquaridt.  Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation , 1970 .

[9]  A. E. Hoerl,et al.  Ridge regression:some simulations , 1975 .

[10]  David M. Allen,et al.  The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction , 1974 .

[11]  Graham C. Goodwin,et al.  Estimation of model quality , 1994, Autom..

[12]  Keith A. Woodbury,et al.  Inverse problems and parameter estimation: integration of measurements and analysis , 1998 .

[13]  Lennart Ljung,et al.  Nonlinear black-box modeling in system identification: a unified overview , 1995, Autom..

[14]  R. Farebrother.  The Minimum Mean Square Error Linear Estimator and Ridge Regression , 1975 .

[15]  Diego A. Murio,et al.  The Mollification Method and the Numerical Solution of Ill-Posed Problems , 1993 .

[16]  Aleksey V. Nenarokomov,et al.  Uncertainties in parameter estimation: the inverse problem , 1995 .

[17]  R. R. Hocking.  The analysis and selection of variables in linear regression , 1976 .

[18]  William H. Press,et al.  Numerical recipes in C (2nd ed.): the art of scientific computing , 1992 .

[19]  Tor Arne Johansen,et al.  On Tikhonov regularization, bias and variance in nonlinear system identification , 1997, Autom..

[20]  S. Wold,et al.  The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses , 1984 .

[21]  George E. P. Box,et al.  Empirical Model‐Building and Response Surfaces , 1988 .

[22]  A. G. Polak,et al.  A forward model for maximum expiration , 1998, Comput. Biol. Medicine.

[23]  Roman Z. Morawski,et al.  Deconvolution algorithms for instrumental applications: A comparative study , 1995 .

[24]  L. Joseph,et al.  Bayesian Statistics: An Introduction , 1989 .

[25]  A. Emery,et al.  Optimal experiment design , 1998 .

[26]  Eric Walter,et al.  Qualitative and quantitative experiment design for phenomenological models - A survey , 1990, Autom..

[27]  I. Helland.  Partial least squares regression and statistical models , 1990 .

[28]  A. G. Polak,et al.  A metrological model for maximum expiration , 1998 .

[29]  Raman K. Mehra,et al.  Optimal input signals for parameter estimation in dynamic systems--Survey and new results , 1974 .

[30]  C. Cobelli,et al.  Sensitivity analysis in parameter estimation of physiological systems , 1996, Proceedings of 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.