Performance analysis of diffusion LMS in multitask networks

Diffusion LMS algorithm has been extensively studied during the last few years. This efficient approach allows to address distributed optimization problems over sensor networks in the case where the nodes have to collaboratively estimate a single parameter vector. Nevertheless, real-life problems are often multitask-oriented in the sense that the optimum parameter vector may not be the same for every node. In this paper, we conduct a theoretical analysis on the stochastic behavior of diffusion LMS when, either intentionally or unintentionally, applied to multitask problems, that is, in a situation where the founding hypothesis of this algorithm is violated. Simulation results validate our theoretical model. Theoretical analysis and simulation show that collaboration can be still beneficial, and depends on antagonistic effects of the estimation bias-variance trade-off. This work provides a theoretical justification for the need to derive new cooperative algorithms specifically dedicated to multitask problems.

[1]  Ali H. Sayed,et al.  Incremental Adaptive Strategies Over Distributed Networks , 2007, IEEE Transactions on Signal Processing.

[2]  H. Vincent Poor,et al.  Distributed learning in wireless sensor networks , 2005, IEEE Signal Processing Magazine.

[3]  Ali H. Sayed,et al.  Diffusion Adaptation Over Networks Under Imperfect Information Exchange and Non-Stationary Data , 2011, IEEE Transactions on Signal Processing.

[4]  Ali H. Sayed,et al.  Diffusion Bias-Compensated RLS Estimation Over Adaptive Networks , 2011, IEEE Transactions on Signal Processing.

[5]  Ali H. Sayed,et al.  Distributed Pareto Optimization via Diffusion Strategies , 2012, IEEE Journal of Selected Topics in Signal Processing.

[6]  Hichem Snoussi,et al.  Functional estimation in Hilbert space for distributed learning in wireless sensor networks , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Hichem Snoussi,et al.  Distributed prediction of time series data with kernels and adaptive filtering techniques in sensor networks , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[8]  Cédric Richard,et al.  Distributed dictionary learning over a sensor network , 2013, ArXiv.

[9]  Jie Chen,et al.  A Decentralized Approach for Nonlinear Prediction of Time Series Data in Sensor Networks , 2010, EURASIP J. Wirel. Commun. Netw..

[10]  Ali H. Sayed,et al.  Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis , 2008, IEEE Transactions on Signal Processing.

[11]  Ali H. Sayed,et al.  Sparse Distributed Learning Based on Diffusion Adaptation , 2012, IEEE Transactions on Signal Processing.

[12]  Ali H. Sayed,et al.  Diffusion Adaptation over Networks , 2012, ArXiv.

[13]  Ali H. Sayed,et al.  Diffusion LMS Strategies for Distributed Estimation , 2010, IEEE Transactions on Signal Processing.

[14]  Paul Honeine,et al.  Non-negative distributed regression for data inference in wireless sensor networks , 2010, 2010 Conference Record of the Forty Fourth Asilomar Conference on Signals, Systems and Computers.

[15]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.