Balanced least squares: Estimation in linear systems with noisy inputs and multiple outputs

This paper revisits the linear model with noisy inputs, in which the performance of the total least squares (TLS) method is far from acceptable. Under the assumption of Gaussian noises, the maximum likelihood (ML) estimation of the system response is reformulated as a general balanced least squares (BLS) problem. Unlike TLS, which minimizes the trace of the product between the empirical and inverse theoretical covariance matrices, BLS promotes solutions with similar values of both the empirical and theoretical error covariance matrices. The general BLS problem is reformulated as a semidefinite program with a rank constraint, which can be relaxed in order to obtain polynomial time algorithms. Moreover, we provide new theoretical results regarding the scenarios in which the relaxation is tight, as well as additional insights on the performance and interpretation of BLS. Finally, some simulation results illustrate the satisfactory performance of the proposed method.

[1]  Stefan Werner,et al.  Analysis of the Gradient-Descent Total Least-Squares Adaptive Filtering Algorithm , 2014, IEEE Transactions on Signal Processing.

[2]  J. Kiefer,et al.  Sequential minimax search for a maximum , 1953 .

[3]  Gordon L. Stüber,et al.  Channel Estimation for Amplify and Forward Relay Based Cooperation Diversity Systems , 2007, IEEE Transactions on Wireless Communications.

[4]  Jonathan William Gillard,et al.  AN OVERVIEW OF LINEAR STRUCTURAL MOD- ELS IN ERRORS IN VARIABLES REGRESSION , 2010 .

[5]  Charles Elkan,et al.  Expectation Maximization Algorithm , 2010, Encyclopedia of Machine Learning.

[6]  Juha Karhunen,et al.  Principal component neural networks — Theory and applications , 1998, Pattern Analysis and Applications.

[7]  R. J. Adcock Note on the Method of Least Squares , 1877 .

[8]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.

[9]  J. Vandewalle,et al.  Analysis and solution of the nongeneric total least squares problem , 1988 .

[10]  Stefan Werner,et al.  Recursive Total Least-Squares Algorithm Based on Inverse Power Method and Dichotomous Coordinate-Descent Iterations , 2014, IEEE Transactions on Signal Processing.

[11]  I. Markovsky,et al.  A Recursive Restricted Total Least-Squares Algorithm , 2014, IEEE Transactions on Signal Processing.

[12]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[13]  Chong-Yung Chi,et al.  Training Sequence Design for Discriminatory Channel Estimation in Wireless MIMO Systems , 2010, IEEE Transactions on Signal Processing.

[14]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[15]  Ignacio Santamaría,et al.  Balanced Least Squares: Linear model estimation with noisy inputs , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[16]  Ami Wiesel,et al.  Linear Regression With Gaussian Model Uncertainty: Algorithms and Bounds , 2008, IEEE Transactions on Signal Processing.

[17]  Gábor Pataki,et al.  On the Rank of Extreme Matrices in Semidefinite Programs and the Multiplicity of Optimal Eigenvalues , 1998, Math. Oper. Res..

[18]  Chunguang Li,et al.  Distributed Sparse Total Least-Squares Over Networks , 2015, IEEE Transactions on Signal Processing.

[19]  Xiangyun Zhou,et al.  Two-Way Training for Discriminatory Channel Estimation in Wireless MIMO Systems , 2011, IEEE Transactions on Signal Processing.

[20]  Ami Wiesel,et al.  Maximum likelihood estimation in linear models with a Gaussian model matrix , 2006, IEEE Signal Processing Letters.

[21]  Sabine Van Huffel,et al.  Overview of total least-squares methods , 2007, Signal Process..

[22]  Dimitri Peaucelle,et al.  SEDUMI INTERFACE 1.02: a tool for solving LMI problems with SEDUMI , 2002, Proceedings. IEEE International Symposium on Computer Aided Control System Design.

[23]  Gene H. Golub,et al.  Some modified matrix eigenvalue problems , 1973, Milestones in Matrix Computation.

[24]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[25]  Daniel Pérez Palomar,et al.  Rank-Constrained Separable Semidefinite Programming With Applications to Optimal Beamforming , 2010, IEEE Transactions on Signal Processing.

[26]  Jitendra K. Tugnait,et al.  On Superimposed Training for MIMO Channel Estimation and Symbol Detection , 2007, IEEE Transactions on Signal Processing.