Robust Distributed Estimation by Networked Agents

Diffusion adaptive networks tasked with solving estimation problems have attracted attention in recent years due to their reliability, scalability, resource efficiency, and resilience to node and link failure. Diffusion adaptation strategies that are based on the least-mean-squares algorithm can be nonrobust against impulsive noise corrupting the measurements. Impulsive noise can degrade stability and steady-state performance, leading to unreliable estimates. In previous work [“Robust adaptation in impulsive noise,” IEEE Trans. Signal Process., vol. 64, no. 11, pp. 2851–2865, Jun. 2016], a robust adaptive algorithm for stand-alone agents was developed, one that semi-parametrically estimates the optimal error nonlinearity jointly with the parameter of interest. Prior knowledge of the impulsive noise distribution was not assumed. In this paper, we extend the framework to solve the problem of robust distributed estimation by a network of agents. Challenges arise due to the coupling among the agents and the distributed nature of the problem. The resulting diffusion strategy is analyzed and its performance illustrated by numerical simulations.

[1]  D. Duttweiler.  Adaptive filter performance with nonlinearities in the correlation multiplier , 1982 .

[2]  Bernard Widrow,et al.  The least mean fourth (LMF) adaptive algorithm and its family , 1984, IEEE Trans. Inf. Theory.

[3]  W. Y. Chen,et al.  A variable step size LMS algorithm , 1990, Proceedings of the 33rd Midwest Symposium on Circuits and Systems.

[4]  H. Neudecker,et al.  Block Kronecker products and the vecb operator , 1991 .

[5]  William A. Sethares,et al.  Adaptive algorithms with nonlinear data and error functions , 1992, IEEE Trans. Signal Process..

[6]  Teresa H. Y. Meng,et al.  Stochastic gradient adaptation under general error criteria , 1994, IEEE Trans. Signal Process..

[7]  J. Chambers,et al.  Convergence and steady-state properties of the least-mean mixed-norm (LMMN) adaptive algorithm , 1996 .

[8]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[9]  Theodore S. Rappaport,et al.  Measurements and simulation of radio frequency impulsive noise in hospitals and clinics , 1997, 1997 IEEE 47th Vehicular Technology Conference. Technology in Motion.

[10]  David Middleton,et al.  Non-Gaussian Noise Models in Signal Processing for Telecommunications: New Methods and Results for Class A and Class B Noise Models , 1999, IEEE Trans. Inf. Theory.

[11]  Tareq Y. Al-Naffouri,et al.  On the selection of optimal nonlinearities for stochastic gradient adaptive algorithms , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[12]  John N. Tsitsiklis,et al.  Gradient Convergence in Gradient methods with Errors , 1999, SIAM J. Optim..

[13]  Tareq Y. Al-Naffouri,et al.  Adaptive Filters with Error Nonlinearities: Mean-Square Analysis and Optimum Design , 2001, EURASIP J. Adv. Signal Process..

[14]  Abdelhak M. Zoubir,et al.  Multiuser Detection in Heavy Tailed Noise , 2002, Digit. Signal Process..

[15]  Deborah Estrin,et al.  Guest Editors' Introduction: Overview of Sensor Networks , 2004, Computer.

[16]  Mani Srivastava,et al.  Overview of sensor networks , 2004 .

[17]  I. Yamada,et al.  Adaptive Projected Subgradient Method for Asymptotic Minimization of Sequence of Nonnegative Convex Functions , 2005 .

[18]  Neil J. Bershad.  On Error Saturation Nonlinearities for LMS Adaptation in Impulsive Noise , 2008, IEEE Transactions on Signal Processing.

[19]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[20]  Ali H. Sayed,et al.  Diffusion LMS Strategies for Distributed Estimation , 2010, IEEE Transactions on Signal Processing.

[21]  Angelia Nedic,et al.  Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization , 2008, J. Optim. Theory Appl..

[22]  Angelia Nedic,et al.  Asynchronous Broadcast-Based Convex Optimization Over a Network , 2011, IEEE Transactions on Automatic Control.

[23]  Sergios Theodoridis,et al.  Adaptive Robust Distributed Learning in Diffusion Sensor Networks , 2011, IEEE Transactions on Signal Processing.

[24]  Sergios Theodoridis,et al.  Adaptive Learning in a World of Projections , 2011, IEEE Signal Processing Magazine.

[25]  Ali H. Sayed,et al.  Distributed Detection Over Adaptive Networks Using Diffusion Adaptation , 2011, IEEE Transactions on Signal Processing.

[26]  Angelia Nedic,et al.  Distributed Asynchronous Constrained Stochastic Optimization , 2011, IEEE Journal of Selected Topics in Signal Processing.

[27]  Ali H. Sayed,et al.  Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks , 2011, IEEE Transactions on Signal Processing.

[28]  Ali H. Sayed,et al.  Diffusion Adaptation over Networks , 2012, ArXiv.

[29]  Michael Muma,et al.  Robust Estimation in Signal Processing: A Tutorial-Style Treatment of Fundamental Concepts , 2012, IEEE Signal Processing Magazine.

[30]  Ali H. Sayed,et al.  Performance Limits for Distributed Estimation Over LMS Adaptive Networks , 2012, IEEE Transactions on Signal Processing.

[31]  Ali H. Sayed,et al.  Diffusion Strategies Outperform Consensus Strategies for Distributed Estimation Over Adaptive Networks , 2012, IEEE Transactions on Signal Processing.

[32]  Ali H. Sayed,et al.  A strategy for adjusting combination weights over adaptive networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[33]  Ali H. Sayed,et al.  Diffusion strategies for adaptation and learning over networks: an examination of distributed strategies and network behavior , 2013, IEEE Signal Processing Magazine.

[34]  Ali H. Sayed,et al.  An optimal error nonlinearity for robust adaptation against impulsive noise , 2013, 2013 IEEE 14th Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[35]  Ali Sayed,et al.  Adaptation, Learning, and Optimization over Networks , 2014, Found. Trends Mach. Learn..

[36]  Ali H. Sayed,et al.  Robust distributed detection over adaptive diffusion networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  Ali H. Sayed,et al.  On the Learning Behavior of Adaptive Networks—Part II: Performance Analysis , 2013, IEEE Transactions on Information Theory.

[38]  Angelia Nedic,et al.  Distributed Optimization Over Time-Varying Directed Graphs , 2015, IEEE Trans. Autom. Control..

[39]  Ali H. Sayed,et al.  On the Learning Behavior of Adaptive Networks—Part I: Transient Analysis , 2013, IEEE Transactions on Information Theory.

[40]  Ali H. Sayed,et al.  Robust Adaptation in Impulsive Noise , 2016, IEEE Transactions on Signal Processing.