Regularization aspects in continuous-time model identification

This paper presents an analysis of some regularization aspects in continuous-time model identification. The study particulary focuses on linear filter methods and shows that filtering the data before estimating their derivatives corresponds to a regularized signal derivative estimation by minimizing a compound criterion whose expression is given explicitly. A new structure based on a null phase filter corresponding to a true regularization filter is proposed and allows to discuss the filter phase effects on parameter estimation by comparing its performances with those of the Poisson filter-based methods. Based on this analysis, a formulation of continuous-time model identification as a joint system input-output signal and model parameter estimation is suggested. In this framework, two linear filter methods are interpreted and a compound criterion is proposed in which the regularization is ensured by a model fitting measure, resulting in a new regularization filter structure for signal estimation.

[1]  P. Young,et al.  Dynamic harmonic regression. , 1999 .

[2]  J. E. Valstar.  In-Flight Dynamic Checkout , 1963, IEEE Transactions on Aerospace.

[3]  P. Young.  Optimal IV identification and estimation of continuous-time TF models , 2002 .

[4]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[5]  Ganti Prasada Rao,et al.  Identification of Continuous Dynamical Systems , 1983 .

[6]  Peter C. Young,et al.  The identification of continuous-time linear and nonlinear models: a tutorial with environmental applications , 2003 .

[7]  Peter C. Young.  Process Parameter Estimation and Self Adaptive Control , 1966 .

[8]  Peter C. Young,et al.  Data-Based Mechanistic Modeling of Engineering Systems , 1998 .

[9]  B. R. Hunt,et al.  The Application of Constrained Least Squares Estimation to Image Restoration by Digital Computer , 1973, IEEE Transactions on Computers.

[10]  Tor Arne Johansen,et al.  On Tikhonov regularization, bias and variance in nonlinear system identification , 1997, Autom..

[11]  Jan Nygaard Nielsen,et al.  Parameter estimation in stochastic differential equations: An overview , 2000 .

[12]  Torsten Söderström,et al.  Perspectives on errors-in-variables estimation for dynamic systems , 2002, Signal Process..

[13]  Lennart Ljung,et al.  Initialisation aspects for subspace and output-error identification methods , 2003, 2003 European Control Conference (ECC).

[14]  Heinz Unbehauen,et al.  A review of identification in continuous-time systems , 1998 .

[15]  Ali Mohammad-Djafari,et al.  Joint estimation of parameters and hyperparameters in a Bayesian approach of solving inverse problems , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[16]  Peter C. Young,et al.  Process Parameter Estimation and Self Adaptive Control , 1965 .

[17]  David L. Phillips,et al.  A Technique for the Numerical Solution of Certain Integral Equations of the First Kind , 1962, JACM.

[18]  P. Young,et al.  Time variable and state dependent modelling of non-stationary and nonlinear time series , 1993 .

[19]  Heinz Unbehauen,et al.  Continuous-time approaches to system identification - A survey , 1990, Autom..

[20]  Rolf Johansson,et al.  System modeling and identification , 1993 .

[21]  S. Twomey,et al.  On the Numerical Solution of Fredholm Integral Equations of the First Kind by the Inversion of the Linear System Produced by Quadrature , 1963, JACM.

[22]  Peter C. Young,et al.  Recursive and en-bloc approaches to signal extraction , 1999 .

[23]  William J. Fitzgerald,et al.  Markov chain Monte Carlo methods with applications to signal processing , 2001, Signal Process..

[24]  R.H.T. Bates,et al.  Image Restoration and Construction , 1986 .

[25]  Chun. Loo,et al.  BAYESIAN APPROACH TO SYSTEM IDENTIFICATION , 1981 .

[26]  Hugues Garnier,et al.  Numerical illustrations of the relevance of direct continuous-time model identification , 2002 .

[27]  Guy Demoment,et al.  Image reconstruction and restoration: overview of common estimation structures and problems , 1989, IEEE Trans. Acoust. Speech Signal Process..

[28]  P. Young,et al.  Refined instrumental variable methods of recursive time-series analysis Part III. Extensions , 1980 .

[29]  Hugues Garnier,et al.  Continuous-time model identification from sampled data: Implementation issues and performance evaluation , 2003 .

[30]  J. Cullum.  Numerical Differentiation and Regularization , 1971 .

[31]  Peter C. Young,et al.  A Direct Approach to the Identification and Estimation of Continuous-time Systems from Discrete-time Data Based on Fixed Interval Smoothing , 1993 .

[32]  Marion Gilson,et al.  Developments for the matlab contsid toolbox , 2003 .

[33]  Peter Young,et al.  Parameter estimation for continuous-time models - A survey , 1979, Autom..

[34]  T. McKelveyDepartment.  On the Use of Regularization in System Identification , 1992 .

[35]  Graham C. Goodwin,et al.  INVERSE PROBLEMS WITH CONSTRAINTS , 2002 .

[36]  T. Söderström,et al.  Least squares parameter estimation of continuous-time ARX models from discrete-time data , 1997, IEEE Trans. Autom. Control..

[37]  Heinz Unbehauen,et al.  Continuous-time Approaches to System Identification , 1988 .

[38]  Jerry M. Mendel,et al.  Optimal seismic deconvolution , 1983 .

[39]  J. Hadamard,et al.  Lectures on Cauchy's Problem in Linear Partial Differential Equations , 1924 .

[40]  Peter C. Young,et al.  Data-based mechanistic modelling of environmental, ecological, economic and engineering systems. , 1998 .

[41]  Peter J. Gawthrop,et al.  On the estimation of continuous time transfer functions , 2001 .

[42]  Michel Mensler.  Analyse et étude comparative de méthodes d'identification des systèmes à représentation continue : développement d'une boite à outils logicielle , 1999 .

[43]  B. A. Lewis.  On the Numerical Solution of Fredholm Integral Equations of the First Kind , 1975 .

[44]  Tor Arne Johansen,et al.  Identification of non-linear systems using empirical data and prior knowledge - an optimization approach , 1996, Autom..

[45]  Peter C. Young,et al.  Parameter Estimation for Continuous Time Models - A Survey , 1979 .

[46]  P. Young.  An instrumental variable method for real-time identification of a noisy process , 1970 .