Adaptive projected subgradient method and its acceleration techniques

Abstract In this paper, we introduce the adaptive projected subgradient method (APSM) that can minimize asymptotically certain sequence of nonnegative unsmooth convex functions over a closed convex set in a real Hilbert space. The main theorem of the method unifies a wide range of set theoretic adaptive filtering schemes, e.g., NLMS, Projected NLMS, Constrained NLMS, APA and Adaptive parallel outer projection algorithm etc., for possibly nonstationary random processes. We also briefly present a pair of acceleration techniques that can improve efficiently the performance of adaptive parallel outer projection algorithm. These include the ideas using optimal supporting hyperplane of the stochastic property set as well as Pairwise Optimal WEight Realization (POWER) of parallel projection. The numerical examples show that the proposed adaptive filtering scheme realizes dramatically fast and stable convergence for highly coloured excited speech like input signals in severely noisy situations.

[1]  H. Trussell,et al.  The feasible solution in signal restoration , 1984 .

[2]  Isao Yamada,et al.  Efficient Adaptive Stereo Echo Canceling Schemes Based on Simultaneous Use of Multiple State Data , 2004 .

[3]  Isao Yamada,et al.  Acceleration of adaptive parallel projection algorithms by Pairwise Optimal WEight Realization , 2004, 2004 12th European Signal Processing Conference.

[4]  T. Hinamoto,et al.  Extended theory of learning identification , 1975 .

[5]  J. Nagumo,et al.  A learning method for system identification , 1967, IEEE Transactions on Automatic Control.

[6]  Patrick L. Combettes,et al.  Convex set theoretic image recovery by extrapolated iterations of parallel subgradient projections , 1997, IEEE Trans. Image Process..

[7]  Boris Polyak.  Minimization of unsmooth functionals , 1969 .

[8]  John F. Doherty,et al.  Generalized projection algorithm for blind interference suppression in DS/CDMA communications , 1997 .

[9]  Isao Yamada,et al.  Adaptive projected subgradient method and its applications to set theoretic adaptive filtering , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[10]  Ali H. Sayed,et al.  H∞ optimality of the LMS algorithm , 1996, IEEE Trans. Signal Process..

[11]  Adaptive parallel outer projection algorithm based on supporting hyperplane approximation , 2003 .

[12]  Kazuhiko Ozeki,et al.  An adaptive filtering algorithm using an orthogonal projection to an affine subspace and its properties , 1984 .

[13]  Isao Yamada,et al.  An efficient robust adaptive filtering algorithm based on parallel subgradient projection techniques , 2002, IEEE Trans. Signal Process..