Recursive least squares with linear constraints

Recursive least squares (RLS) algorithms have widespread applications in many areas, such as real-time signal processing, control and communications. This paper shows that the unique solutions to linear-equality constrained and the unconstrained LS problems, respectively, always have exactly the same recursive form. Their only difference lies in the initial values. Based on this, a recursive algorithm for the linear-inequality constrained LS problem is developed. The RLS algorithm, in a theoretically equivalent form by a simple modification, is shown to be robust in that the constraints are always guaranteed to be satisfied no matter how large the numerical errors are.

[1]  J. Stoer.  On the Numerical Solution of Constrained Least-Squares Problems , 1971 .

[2]  John M. Cioffi,et al.  The fast adaptive ROTOR's RLS algorithm , 1990, IEEE Trans. Acoust. Speech Signal Process..

[3]  Richard J. Hanson,et al.  Linear least squares with bounds and liner constraints , 1986 .

[4]  R. Plackett.  Some theorems in least squares. , 1950, Biometrika.

[5]  Å. Björck.  A General Updating Algorithm for Constrained Linear Least Squares Problems , 1984 .

[6]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[7]  O. L. Frost,et al.  An algorithm for linearly constrained adaptive array processing , 1972 .

[8]  J. Stoer,et al.  A factorization method for the solution of constrained linear least squares problems allowing subsequent data changes , 1978 .

[9]  Han-fu Chʿen.  Recursive estimation and control for stochastic systems , 1985 .

[10]  Karen H. Haskell,et al.  An algorithm for linear least squares problems with equality and nonnegativity constraints , 1981, Math. Program..

[11]  Lal C. Godara.  9 Constrained beamforming and adaptive algorithms , 1993, Signal Processing and its Applications.

[12]  John M. Cioffi.  The fast Householder filters-RLS adaptive filter , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[13]  Douglas James.  Implicit Nullspace Iterative Methods for Constrained Least Squares Problems , 1992, SIAM J. Matrix Anal. Appl..

[14]  Gene H. Golub,et al.  Matrix computations , 1983 .

[15]  David A. Wismer,et al.  Introduction to nonlinear optimization , 1978 .

[16]  Mårten Gulliksson,et al.  Modifying the QR-Decomposition to Constrained and Weighted Linear Least Squares , 1992, SIAM J. Matrix Anal. Appl..

[17]  Thomas Kailath,et al.  Constrained-input/constrained-output stability for adaptive RLS lattice filters , 1991 .

[18]  R. Streit.  Solution of systems of complex linear equations in the l ∞ 0E norm with constraints on the unknowns , 1986 .

[19]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[20]  Mårten Gulliksson.  Iterative refinement for constrained and weighted linear least squares , 1994 .

[21]  N. Nichols,et al.  Iterative Methods for Equality-Constrained Least Squares Problems , 1988 .

[22]  Felix F. Wu,et al.  Power system state estimation: a survey , 1990 .

[23]  R. Farebrother.  Linear least squares computations , 1988 .

[24]  P. Caines.  Linear Stochastic Systems , 1988 .

[25]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[26]  Haesun Park,et al.  Self-scaling fast rotations for stiff and equality-constrained linear least squares problems , 1996 .

[27]  Achiya Dax,et al.  On Row Relaxation Methods for Large Constrained Least Squares Problems , 1993, SIAM J. Sci. Comput..

[28]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[29]  Donald M. Wiberg,et al.  Dynamic system identification: Experiment design and data analysis , 1983 .

[30]  B.D. Van Veen,et al.  Beamforming: a versatile approach to spatial filtering , 1988, IEEE ASSP Magazine.

[31]  T. Kailath,et al.  Fast, recursive-least-squares transversal filters for adaptive filtering , 1984 .

[32]  Åke Björck,et al.  Numerical methods for least square problems , 1996 .

[33]  C. Lawson,et al.  Extensions and applications of the Householder algorithm for solving linear least squares problems , 1969 .

[34]  Roy L. Streit,et al.  ALGORITHM 635: an algorithm for the solution of systems of complex linear equations in the L∞ norm with constraints on the unknowns , 1985, TOMS.

[35]  L. Ljung,et al.  Fast calculation of gain matrices for recursive estimation schemes , 1978 .