A proximal splitting approach to regularized distributed adaptive estimation in diffusion networks

We propose a proximal splitting approach to regularized distributed estimation over networks employing diffusion adaptation strategies. Playing a central role in the proposed framework is the so-called proximity operator, which is a generalization of the convex projection mapping, that enables us to handle convex regularization terms efficiently. The diffusion algorithms developed using the proximal formalism endow networks with new learning abilities and open up possibilities for enhancing performance of the networks by utilizing more general convex penalties. We present performance analysis of the proposed method and provide simulations to demonstrate its feasibility in recovering sparse signals.

[1]  J. Moreau Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[2]  Gregory B. Passty Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .

[3]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[4]  D. Gabay Applications of the method of multipliers to variational inequalities , 1983 .

[5]  P. Tseng Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[6]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[7]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[8]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[9]  Ali H. Sayed,et al.  Fundamentals Of Adaptive Filtering , 2003 .

[10]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[11]  I. Yamada,et al.  Adaptive Projected Subgradient Method for Asymptotic Minimization of Sequence of Nonnegative Convex Functions , 2005 .

[12]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[13]  Ali H. Sayed,et al.  Diffusion recursive least-squares for distributed estimation over adaptive networks , 2008, IEEE Transactions on Signal Processing.

[14]  Ali H. Sayed,et al.  Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis , 2008, IEEE Transactions on Signal Processing.

[15]  Alfred O. Hero,et al.  Sparse LMS for system identification , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[17]  Isao Yamada,et al.  An Adaptive Projected Subgradient Approach to Learning in Diffusion Networks , 2009, IEEE Transactions on Signal Processing.

[18]  Ali H. Sayed,et al.  Diffusion LMS Strategies for Distributed Estimation , 2010, IEEE Transactions on Signal Processing.

[19]  Georgios B. Giannakis,et al.  Online Adaptive Estimation of Sparse Signals: Where RLS Meets the $\ell_1$ -Norm , 2010, IEEE Transactions on Signal Processing.

[20]  Isao Yamada,et al.  Link probability control for probabilistic diffusion least-mean squares over resource-constrained networks , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  A. Hero,et al.  Regularized Least-Mean-Square Algorithms , 2010, 1012.5066.

[22]  Vahid Tarokh,et al.  SPARLS: The Sparse RLS Algorithm , 2010, IEEE Transactions on Signal Processing.

[23]  Ali H. Sayed,et al.  Foraging behavior of fish schools via diffusion adaptation , 2010, 2010 2nd International Workshop on Cognitive Information Processing.

[24]  Isao Yamada,et al.  Diffusion Least-Mean Squares With Adaptive Combiners: Formulation and Performance Analysis , 2010, IEEE Transactions on Signal Processing.

[25]  Isao Yamada,et al.  A sparse adaptive filtering using time-varying soft-thresholding techniques , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[26]  Jianshu Chen,et al.  Bacterial motility via diffusion adaptation , 2010, 2010 Conference Record of the Forty Fourth Asilomar Conference on Signals, Systems and Computers.

[27]  Isao Yamada,et al.  Acceleration of adaptive proximal forward-backward splitting method and its application to sparse system identification , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Ali H. Sayed,et al.  Modeling Bird Flight Formations Using Diffusion Adaptation , 2011, IEEE Transactions on Signal Processing.

[29]  Sergios Theodoridis,et al.  Adaptive Robust Distributed Learning in Diffusion Sensor Networks , 2011, IEEE Transactions on Signal Processing.

[30]  I. Yamada,et al.  Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize , 2011 .

[31]  Sergios Theodoridis,et al.  Online Sparse System Identification and Signal Reconstruction Using Projections Onto Weighted $\ell_{1}$ Balls , 2010, IEEE Transactions on Signal Processing.

[32]  Dimitri P. Bertsekas,et al.  Incremental proximal methods for large scale convex optimization , 2011, Math. Program..

[33]  Ali H. Sayed,et al.  Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks , 2011, IEEE Transactions on Signal Processing.

[34]  Ali H. Sayed,et al.  Diffusion Adaptation over Networks , 2012, ArXiv.

[35]  Sergios Theodoridis,et al.  A Sparsity Promoting Adaptive Algorithm for Distributed Learning , 2012, IEEE Transactions on Signal Processing.

[36]  Ali H. Sayed,et al.  Modeling bee swarming behavior through diffusion adaptation with asymmetric information sharing , 2012, EURASIP J. Adv. Signal Process..

[37]  Zhaoyang Zhang,et al.  Diffusion Sparse Least-Mean Squares Over Networks , 2012, IEEE Transactions on Signal Processing.

[38]  Ali H. Sayed,et al.  Diffusion Strategies Outperform Consensus Strategies for Distributed Estimation Over Adaptive Networks , 2012, IEEE Transactions on Signal Processing.

[39]  Ali H. Sayed,et al.  Diffusion strategies for adaptation and learning over networks: an examination of distributed strategies and network behavior , 2013, IEEE Signal Processing Magazine.

[40]  Ali H. Sayed,et al.  Sparse Distributed Learning Based on Diffusion Adaptation , 2012, IEEE Transactions on Signal Processing.