A Unified View of Adaptive Variable-Metric Projection Algorithms

We present a unified analytic tool named variable-metric adaptive projected subgradient method (V-APSM) that encompasses the important family of adaptive variable-metric projection algorithms. The family includes the transform-domain adaptive filter, the Newton-method-based adaptive filters such as quasi-Newton, the proportionate adaptive filter, and the Krylov-proportionate adaptive filter. We provide a rigorous analysis of V-APSM regarding several invaluable properties including monotone approximation, which indicates stable tracking capability, and convergence to an asymptotically optimal point. Small metric-fluctuations are the key assumption for the analysis. Numerical examples show (i) the robustness of V-APSM against violation of the assumption and (ii) the remarkable advantages over its constant-metric counterpart for colored and nonstationary inputs under noisy situations.

[1]  Boris Polyak,et al.  Constrained minimization methods , 1966 .

[2]  Wolfgang Utschick,et al.  Proportionate adaptive algorithm for nonsparse systems based on Krylov subspace and constrained optimization , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  Isao Yamada,et al.  Efficient Fast Stereo Acoustic Echo Cancellation Based on Pairwise Optimal Weight Realization Technique , 2006, EURASIP J. Adv. Signal Process..

[4]  B. Farhang-Boroujeny,et al.  Adaptive Filters: Theory and Applications , 1999 .

[5]  Ali H. Sayed,et al.  Fundamentals Of Adaptive Filtering , 2003 .

[6]  J. Nagumo,et al.  A learning method for system identification , 1967, IEEE Transactions on Automatic Control.

[7]  Françoise Beaufays,et al.  Transform-domain adaptive filters: an analytical approach , 1995, IEEE Trans. Signal Process..

[8]  Isao Yamada,et al.  An Adaptive Projected Subgradient Approach to Learning in Diffusion Networks , 2009, IEEE Transactions on Signal Processing.

[9]  Sergios Theodoridis,et al.  Online Kernel-Based Classification Using Adaptive Projection Algorithms , 2008, IEEE Transactions on Signal Processing.

[10]  I. Yamada,et al.  Pairwise Optimal Weight Realization—Acceleration Technique for Set-Theoretic Adaptive Parallel Subgradient Projection Algorithm , 2006, IEEE Transactions on Signal Processing.

[11]  Isao Yamada,et al.  Multiaccess Interference Suppression in Orthogonal Space–Time Block Coded MIMO Systems by Adaptive Projected Subgradient Method , 2008, IEEE Transactions on Signal Processing.

[12]  Isao Yamada,et al.  An efficient robust adaptive filtering algorithm based on parallel subgradient projection techniques , 2002, IEEE Trans. Signal Process..

[13]  Donald L. Duttweiler,et al.  Proportionate normalized least-mean-squares adaptation in echo cancelers , 2000, IEEE Trans. Speech Audio Process..

[14]  A. Goldstein.  Convex programming in Hilbert space , 1964 .

[15]  S.L. Gay,et al.  An efficient, fast converging adaptive filter for network echo cancellation , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[16]  Steve Rogers,et al.  Adaptive Filter Theory , 1996 .

[17]  Wolfgang Utschick,et al.  A Fast Stochastic Gradient Algorithm: Maximal Use of Sparsification Benefits under Computational Constraints , 2010, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[18]  W. Kenneth Jenkins,et al.  A fast quasi-Newton adaptive filtering algorithm , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[19]  Andreas Antoniou,et al.  Analysis of LMS-Newton adaptive filtering algorithms with variable convergence factor , 1995, IEEE Trans. Signal Process..

[20]  Isao Yamada,et al.  Multi-Domain Adaptive Learning Based on Feasibility Splitting and Adaptive Projected Subgradient Method , 2010, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[21]  I. Yamada,et al.  Signal processing in dual domain by adaptive projected subgradient method , 2009, 2009 16th International Conference on Digital Signal Processing.

[22]  Kazuhiko Ozeki,et al.  An adaptive filtering algorithm using an orthogonal projection to an affine subspace and its properties , 1984 .

[23]  Masahiro Yukawa,et al.  Krylov-Proportionate Adaptive Filtering Techniques Not Limited to Sparse Systems , 2009, IEEE Transactions on Signal Processing.

[24]  José Antonio Apolinário,et al.  Constrained normalized adaptive filters for CDMA mobile communications , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[25]  Jacob Benesty,et al.  Acoustic MIMO Signal Processing (Signals and Communication Technology) , 2006 .

[26]  Isao Yamada,et al.  Adaptive Parallel Variable-Metric Projection Algorithm -An Application to Acoustic Echo Cancellation , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[27]  W. T. Federer,et al.  Stochastic Approximation and NonLinear Regression , 2003 .

[28]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[29]  Isao Yamada,et al.  Adaptive Parallel Quadratic-Metric Projection Algorithms , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[30]  Milos Doroslovacki,et al.  Proportionate adaptive algorithms for network echo cancellation , 2006, IEEE Transactions on Signal Processing.

[31]  J. Benesty,et al.  Double-talk robust fast converging algorithms for network echo cancellation , 1999, Proceedings of the 1999 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. WASPAA'99 (Cat. No.99TH8452).

[32]  John F. Doherty,et al.  Generalized projection algorithm for blind interference suppression in DS/CDMA communications , 1997 .

[33]  A. Peterson,et al.  Transform domain LMS algorithm , 1983 .

[34]  I. Yamada,et al.  The Adaptive Projected Subgradient Method over the Fixed Point Set of Strongly Attracting Nonexpansive Mappings , 2006 .

[35]  Andreas Antoniou,et al.  A new quasi-Newton adaptive filtering algorithm , 1997 .

[36]  Isao Yamada,et al.  A Flexible Peak-To-Average Power Ratio Reduction Scheme for OFDM Systems by the Adaptive Projected Subgradient Method , 2009, IEEE Transactions on Signal Processing.

[37]  I. Yamada,et al.  Adaptive Projected Subgradient Method for Asymptotic Minimization of Sequence of Nonnegative Convex Functions , 2005 .

[38]  Isao Yamada,et al.  Efficient Blind MAI Suppression in DS/CDMA Systems by Embedded Constraint Parallel Projection Techniques , 2005, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[39]  Boris Polyak.  Minimization of unsmooth functionals , 1969 .

[40]  W. K. Jenkins,et al.  The use of orthogonal transforms for improving performance of adaptive filters , 1989 .

[41]  T. Hinamoto,et al.  Extended theory of learning identification , 1975 .

[42]  Sergios Theodoridis,et al.  Adaptive Constrained Learning in Reproducing Kernel Hilbert Spaces: The Robust Beamforming Case , 2009, IEEE Transactions on Signal Processing.