Regularization networks for inverse problems: A state-space approach

Linear inverse problems with discrete data are equivalent to the estimation of the continuous-time input of a linear dynamical system from samples of its output. The solution obtained by means of regularization theory has the structure of a neural network similar to classical RBF networks. However, the basis functions depend in a nontrivial way on the specific linear operator to be inverted and the adopted regularization strategy. By resorting to the Bayesian interpretation of regularization, we show that such networks can be implemented rigorously and efficiently whenever the linear operator admits a state-space representation. An analytic expression is provided for the basis functions as well as for the entries of the matrix of the linear system used to compute the weights. The results are illustrated through a deconvolution problem where the spontaneous secretory rate of luteinizing hormone (LH) of the hypophisis is reconstructed from measurements of plasma LH concentrations.

[1]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[2]  Daniel Yellin,et al.  Multichannel system identification and deconvolution: performance bounds , 1999, IEEE Trans. Signal Process..

[3]  T. Stockham,et al.  An automated film reader for DNA sequencing based on homomorphic deconvolution , 1994, IEEE Transactions on Biomedical Engineering.

[4]  D Liberati,et al.  Non‐parametric deconvolution provides an objective assessment of GH responsiveness to GH‐releasing stimuli in normal subjects , 1997, Clinical endocrinology.

[5]  D. Liberati,et al.  Linear and nonlinear techniques for the deconvolution of hormone time-series , 1993, IEEE Transactions on Biomedical Engineering.

[6]  M. Bertero.  Linear Inverse and III-Posed Problems , 1989 .

[7]  D Liberati,et al.  LH and FSH secretory responses to GnRH in normal individuals: a non-parametric deconvolution approach. , 1999, European journal of endocrinology.

[8]  Giuseppe De Nicolao,et al.  Bayesian Function Learning Using MCMC Methods , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  J D Veldhuis,et al.  Metabolic clearance of biologically active luteinizing hormone in man. , 1986, The Journal of clinical investigation.

[10]  G. Wahba.  Practical Approximate Solutions to Linear Operator Equations When the Data are Noisy , 1977 .

[11]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[12]  Jaafar M. H. Elmirghani,et al.  Blind channel estimation and echo cancellation using chaotic coded signals , 1999, IEEE Communications Letters.

[13]  Petros G. Voulgaris,et al.  On optimal ℓ∞ to ℓ∞ filtering , 1995, Autom..

[14]  Giuseppe De Nicolao,et al.  Regularization networks: fast weight calculation via Kalman filtering , 2001, IEEE Trans. Neural Networks.

[15]  Chong Sze Tong.  Blur detection using a neural network , 1995, Optics & Photonics.

[16]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[17]  Mario Bertero,et al.  Linear inverse problems with discrete data. I. General formulation and singular system analysis , 1985 .

[18]  D Rodbard,et al.  Pulsatile secretion of luteinizing hormone in agonadal men before and during testosterone replacement therapy , 1990, Journal of endocrinological investigation.

[19]  G. Ferrari-Trecate,et al.  A Kalman filtering algorithm for regularization networks , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[20]  G. Wahba.  Spline models for observational data , 1990 .

[21]  Andrzej Cichocki,et al.  Self-whitening algorithms for adaptive equalization and deconvolution , 1999, IEEE Trans. Signal Process..

[22]  Giuseppe De Nicolao,et al.  Nonparametric input estimation in physiological systems: Problems, methods, and case studies , 1997, Autom..

[23]  C. Loan.  Computing integrals involving the matrix exponential , 1978 .