Best approximation of the identity mapping: The case of variable finite memory

This paper concerns the best causal operator approximation of the identity mapping subject to a specified variable finite memory constraint. The causality and memory constraints require that the approximating operator takes the form of a lower stepped matrix A. To find the best such matrix, we propose a new technique based on a block-partition into an equivalent collection of smaller blocks, {L"0,K"1,L"1,...,K"@?,L"@?} where each L"r is a lower triangular block and each K"r is a rectangular block and where @? is known. The sizes of the individual blocks are defined by the memory constraints. We show that the best approximation problem for the lower stepped matrix A can be replaced by an equivalent collection of @? independent best approximation problems in terms of the matrices [L"0],[K"1,L"1],...,[K"@?,L"@?]. The solution to each individual problem is found and a representation of the overall solution and associated error is given.

[1]  Colin J. Champion.  Empirical Bayesian estimation of normal variances and covariances , 2003 .

[2]  Phil Howlett,et al.  Best approximation of operators in the modeling of nonlinear systems , 2002 .

[3]  Vincent J. Bruno.  A Weierstrass approximation theorem for topological vector spaces , 1984 .

[4]  P. M. Prenter.  A Weierstrass theorem for real, separable Hilbert spaces , 1970 .

[5]  I. Sandberg.  Approximately-finite memory and input-output maps , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[6]  Anatoli Torokhti,et al.  Constructing fixed rank optimal estimators with method of best recurrent approximations , 2003 .

[7]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[8]  Anatoli Torokhti,et al.  A methodology for the constructive approximation of nonlinear operators defined on noncompact sets , 1997 .

[9]  I. W. Sandberg,et al.  /spl Ropf//sub +/ fading memory and extensions of input-output maps , 2002 .

[10]  Vasile I Istrăţescu.  A Weierstrass theorem for real Banach spaces , 1977 .

[11]  Sheng Chen,et al.  Representations of non-linear systems: the NARMAX model , 1989 .

[12]  Phil Howlett,et al.  On the constructive approximation of non-linear operators in the modelling of dynamical systems , 1997 .

[13]  Jan P. Allebach,et al.  Combating Speckle In SAR Images: Vector Filtering And Sequential Classification Based On A Multiplicative Noise Model , 1990 .

[14]  Anatoli Torokhti,et al.  Method of Best Successive Approximations for Nonlinear Operators , 2003 .

[15]  Vladimir N. Fomin,et al.  Abstract Optimal Linear Filtering , 2000, SIAM J. Control. Optim..

[16]  Anatoli Torokhti,et al.  The modelling and numerical simulation of causal non-linear systems , 2001 .

[17]  Irwin W. Sandberg,et al.  Z+ fading memory and extensions of input–output maps , 2001, Int. J. Circuit Theory Appl..

[18]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series , 1964 .

[19]  Anatoli Torokhti,et al.  Optimal Mathematical Models for Nonlinear Dynamical Systems , 2003 .

[20]  Thomas L. Marzetta,et al.  Estimating a covariance matrix from incomplete realizations of a random vector , 1992, IEEE Trans. Signal Process..

[21]  Vladimir N. Temlyakov,et al.  Nonlinear Methods of Approximation , 2003, Found. Comput. Math..

[22]  Jean Pierre Delmas,et al.  On eigenvalue decomposition estimators of centro-symmetric covariance matrices , 1999, Signal Process..

[23]  I. Singer.  Best Approximation in Normed Linear Spaces by Elements of Linear Subspaces , 1970 .

[24]  Anatoli Torokhti,et al.  Weak interpolation and approximation of non-linear operators on the space C([0,1]) , 1998 .

[25]  V. Gimeno,et al.  Obtaining the EEG envelope in real time: a practical method based on homomorphic filtering. , 1987, Neuropsychobiology.

[26]  H. W. Bode,et al.  A Simplified Derivation of Linear Least Square Smoothing and Prediction Theory , 1950, Proceedings of the IRE.

[27]  Gene H. Golub,et al.  Matrix computations , 1983 .

[28]  Alan S. Willsky,et al.  A Krylov Subspace Method for Covariance Approximation and Simulation of Random Processes and Fields , 2003, Multidimens. Syst. Signal Process..

[29]  R. Carroll,et al.  A Note on the Efficiency of Sandwich Covariance Matrix Estimation , 2001 .

[30]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[31]  Anatoli Torokhti,et al.  A philosophy for the modelling of realistic nonlinear systems , 2003 .

[32]  Harold W. Sorenson,et al.  Parameter estimation: Principles and problems , 1980 .

[33]  Vlad Timofte,et al.  Stone-Weierstrass theorems revisited , 2005, J. Approx. Theory.

[34]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series, with Engineering Applications , 1949 .

[35]  Tatsuya Kubokawa,et al.  Estimating the covariance matrix: a new approach , 2003 .

[36]  Anatoli Torokhti,et al.  Optimal transform formed by a combination of nonlinear operators: the case of data dimensionality reduction , 2006, IEEE Transactions on Signal Processing.

[37]  Anatoli Torokhti,et al.  Best causal mathematical models for a nonlinear system , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[38]  Anatoli Torokhti,et al.  An optimal linear filter for random signals with realisations in a separable Hilbert space , 2003, The ANZIAM Journal.

[39]  Anatoli Torokhti,et al.  Computational Methods for Modeling of Nonlinear Systems , 1961 .