Distributed learning in wireless sensor networks

This paper discusses nonparametric distributed learning. After reviewing the classical learning model and highlighting the success of machine learning in centralized settings, the challenges that wireless sensor networks (WSN) pose for distributed learning are discussed, and research aimed at addressing these challenges is surveyed.

[1]  G. Roussas.  Nonparametric estimation in Markov processes , 1969 .

[2]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[3]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[4]  Gene H. Golub,et al.  Matrix computations , 1983 .

[5]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[6]  Adam Krzyzak,et al.  The rates of convergence of kernel regression estimates and classification rules , 1986, IEEE Trans. Inf. Theory.

[7]  Miroslaw Pawlak,et al.  Necessary and sufficient conditions for Bayes risk consistency of a recursive kernel classification rule , 1987, IEEE Trans. Inf. Theory.

[8]  S. Yakowitz.  Nonparametric density and regression estimation for Markov sequences without mixing assumptions , 1989 .

[9]  R. Viswanathan,et al.  Distributed detection of a signal in generalized Gaussian noise , 1989, IEEE Trans. Acoust. Speech Signal Process..

[10]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[11]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[12]  P. Varshney,et al.  Some results on distributed nonparametric detection , 1990, 29th IEEE Conference on Decision and Control.

[13]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[14]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[15]  J. Tsitsiklis.  Decentralized Detection' , 1993 .

[16]  David Haussler,et al.  How to use expert advice , 1993, STOC.

[17]  S. Yakowitz.  Nearest neighbor regression estimation for null-recurrent Markov time series , 1993 .

[18]  H. Sebastian Seung,et al.  Learning from a Population of Hypotheses , 1993, COLT '93.

[19]  Zhen Zhang,et al.  On the CEO problem , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[20]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[21]  Naoki Abe,et al.  Efficient Distribution-free Population Learning of Simple Concepts , 1994, AII/ALT.

[22]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[23]  Sanjeev R. Kulkarni,et al.  Rates of convergence of nearest neighbor estimation under arbitrary sampling , 1995, IEEE Trans. Inf. Theory.

[24]  Emad K. Al-Hussaini,et al.  Decentralized CFAR signal detection , 1995, Signal Process..

[25]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[26]  Foster J. Provost,et al.  Scaling Up: Distributed Machine Learning with Cooperation , 1996, AAAI/IAAI, Vol. 1.

[27]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[28]  Toby Berger,et al.  The CEO problem [multiterminal source coding] , 1996, IEEE Trans. Inf. Theory.

[29]  Pramod K. Varshney,et al.  Distributed Detection and Data Fusion , 1996 .

[30]  Yoram Singer,et al.  Using and combining predictors that specialize , 1997, STOC '97.

[31]  Toby Berger,et al.  The quadratic Gaussian CEO problem , 1997, IEEE Trans. Inf. Theory.

[32]  Rick S. Blum,et al.  Distributed detection with multiple sensors I. Advanced topics , 1997, Proc. IEEE.

[33]  Pramod K. Varshney,et al.  Distributed detection with multiple sensors I. Fundamentals , 1997, Proc. IEEE.

[34]  Kenji Yamanishi,et al.  Distributed cooperative Bayesian learning strategies , 1997, COLT '97.

[35]  Sawasd Tantaratana,et al.  Nonparametric distributed detector using Wilcoxon statistics , 1997, Signal Process..

[36]  Sanjeev R. Kulkarni,et al.  Density Estimation from an Individual Numerical Sequence , 1998, IEEE Trans. Inf. Theory.

[37]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Sanjeev R. Kulkarni,et al.  Learning Pattern Classification - A Survey , 1998, IEEE Trans. Inf. Theory.

[39]  Shun-ichi Amari,et al.  Statistical Inference Under Multiterminal Data Compression , 1998, IEEE Trans. Inf. Theory.

[40]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[41]  Peter L. Bartlett,et al.  Neural Network Learning - Theoretical Foundations , 1999 .

[42]  A. Nobel.  Limits to classification and regression estimation from ergodic processes , 1999 .

[43]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[44]  Robert J. McEliece,et al.  The generalized distributive law , 2000, IEEE Trans. Inf. Theory.

[45]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[46]  Andrzej Stachurski,et al.  Parallel Optimization: Theory, Algorithms and Applications , 2000, Parallel Distributed Comput. Pract..

[47]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[48]  Zoran Obradovic,et al.  The distributed boosting algorithm , 2001, KDD '01.

[49]  D. Bertsekas,et al.  Convergen e Rate of In remental Subgradient Algorithms , 2000 .

[50]  Andrew B. Nobel,et al.  Estimating a function from ergodic samples with additive noise , 2001, IEEE Trans. Inf. Theory.

[51]  Dimitri P. Bertsekas,et al.  Incremental Subgradient Methods for Nondifferentiable Optimization , 2001, SIAM J. Optim..

[52]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[53]  Venugopal V. Veeravalli.  Decentralized quickest change detection , 2001, IEEE Trans. Inf. Theory.

[54]  Yu Hen Hu,et al.  Detection, classification, and tracking of targets , 2002, IEEE Signal Process. Mag..

[55]  Sanjeev R. Kulkarni,et al.  Data-dependent kn-NN and kernel estimators consistent for arbitrary processes , 2002, IEEE Trans. Inf. Theory.

[56]  X. Jin.  Factor graphs and the Sum-Product Algorithm , 2002 .

[57]  Ian F. Akyildiz,et al.  Sensor Networks , 2002, Encyclopedia of GIS.

[58]  Feng Zhao,et al.  Collaborative signal and information processing in microsensor networks , 2002, IEEE Signal Processing Magazine.

[59]  Sergio D. Servetto.  On the Feasibility of Large-Scale Wireless Sensor Networks , 2002 .

[60]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[61]  Edward J. Coyle,et al.  An energy efficient hierarchical clustering algorithm for wireless sensor networks , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[62]  Akbar M. Sayeed,et al.  Collaborative Signal Processing for Distributed Classification in Sensor Networks , 2003, IPSN.

[63]  Benjamin Van Roy,et al.  Distributed Optimization in Adaptive Networks , 2003, NIPS.

[64]  Robert D. Nowak,et al.  Distributed EM algorithms for density estimation and clustering in sensor networks , 2003, IEEE Trans. Signal Process..

[65]  Mark A. Paskin,et al.  Junction tree algorithms for solving sparse linear systems , 2003 .

[66]  Akbar M. Sayeed,et al.  Distributed Multi-target Classification in Wireless Sensor Networks , 2003 .

[67]  Urbashi Mitra,et al.  Boundary Estimation in Sensor Networks: Theory and Methods , 2003, IPSN.

[68]  Slobodan N. Simic,et al.  A Learning-Theory Approach to Sensor Networks , 2003, IEEE Pervasive Comput..

[69]  Martin J. Wainwright,et al.  Decentralized detection and classification using kernel methods , 2004, ICML.

[70]  Panganamala Ramana Kumar,et al.  Extended message passing algorithm for inference in loopy Gaussian graphical models , 2004, Ad Hoc Networks.

[71]  H. Vincent Poor,et al.  Consistency in a model for distributed learning with specialists , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[72]  PROCEssIng magazInE.  IEEE Signal Processing Magazine , 2004 .

[73]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[74]  V. Ramachandran,et al.  Distributed classification of Gaussian space-time sources in wireless sensor networks , 2004, IEEE Journal on Selected Areas in Communications.

[75]  H. Vincent Poor,et al.  Consistency in Models for Communication Constrained Distributed Learning , 2004, COLT.

[76]  Igor Durdanovic,et al.  Parallel Support Vector Machines: The Cascade SVM , 2004, NIPS.

[77]  Robert Nowak,et al.  Distributed optimization in sensor networks , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[78]  Nitesh V. Chawla,et al.  Learning Ensembles from Bites: A Scalable and Accurate Approach , 2004, J. Mach. Learn. Res..

[79]  Brian M. Sadler,et al.  Information retrieval and processing in sensor networks: deterministic scheduling vs. random access , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[80]  Venugopal V. Veeravalli,et al.  Asymptotic results for decentralized detection in power constrained wireless sensor networks , 2004, IEEE Journal on Selected Areas in Communications.

[81]  C. Guestrin,et al.  Distributed regression: an efficient framework for modeling sensor network data , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[82]  V. Delouille,et al.  Robust distributed estimation in sensor networks using the embedded polygons algorithm , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[83]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[84]  Alfred O. Hero,et al.  Distributed maximum likelihood estimation for sensor networks , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[85]  H.-A. Loeliger,et al.  An introduction to factor graphs , 2004, IEEE Signal Process. Mag..

[86]  Sanjeev R. Kulkarni,et al.  Communication-estimation tradeoffs in wireless sensor networks , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[87]  Jason Weston,et al.  Fast Kernel Classifiers with Online and Active Learning , 2005, J. Mach. Learn. Res..

[88]  Zhi-Quan Luo,et al.  Universal decentralized detection in a bandwidth-constrained sensor network , 2004, IEEE Transactions on Signal Processing.

[89]  G.B. Giannakis,et al.  Localization via ultra-wideband radios: a look at positioning aspects for future sensor networks , 2005, IEEE Signal Processing Magazine.

[90]  Zhi-Quan Luo.  An isotropic universal decentralized estimation scheme for a bandwidth constrained ad hoc sensor network , 2005, IEEE J. Sel. Areas Commun..

[91]  H. Vincent Poor,et al.  Regression in sensor networks: training distributively with alternating projections , 2005, SPIE Optics + Photonics.

[92]  Robert D. Nowak,et al.  Quantized incremental algorithms for distributed optimization , 2005, IEEE Journal on Selected Areas in Communications.

[93]  R.L. Moses,et al.  Locating the nodes: cooperative localization in wireless sensor networks , 2005, IEEE Signal Processing Magazine.

[94]  Zhi-Quan Luo.  Universal decentralized estimation in a bandwidth constrained sensor network , 2005, IEEE Trans. Inf. Theory.

[95]  Michael I. Jordan,et al.  Nonparametric decentralized detection using kernel methods , 2005, IEEE Transactions on Signal Processing.

[96]  H. Vincent Poor,et al.  Neyman-pearson detection of gauss-Markov signals in noise: closed-form error exponentand properties , 2005, IEEE Transactions on Information Theory.

[97]  Bruno Sinopoli,et al.  A kernel-based learning approach to ad hoc sensor network localization , 2005, TOSN.

[98]  H. Vincent Poor,et al.  Neyman-Pearson Detection of Gauss-Markov Signals in Noise: Closed-Form Error Exponent and Properties , 2005, ISIT.

[99]  Carlos Guestrin,et al.  A robust architecture for distributed inference in sensor networks , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[100]  Andreas F. Molisch,et al.  Localization via Ultra- Wideband Radios , 2005 .

[101]  Zhi-Quan Luo,et al.  Decentralized estimation in an inhomogeneous sensing environment , 2005, IEEE Transactions on Information Theory.

[102]  Zhi-Quan Luo,et al.  Universal decentralized detection in a bandwidth-constrained sensor network , 2005, IEEE Trans. Signal Process..

[103]  V. Ramachandran,et al.  Distributed multitarget classification in wireless sensor networks , 2005, IEEE Journal on Selected Areas in Communications.

[104]  Zhi-Quan Luo.  An isotropic universal decentralized estimation scheme for a bandwidth constrained ad hoc sensor network , 2005, IEEE Journal on Selected Areas in Communications.

[105]  H. Vincent Poor,et al.  A large deviations approach to sensor scheduling for detection of correlated random fields , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[106]  Zhi-Quan Luo,et al.  Universal decentralized estimation in a bandwidth constrained sensor network , 2005, IEEE Transactions on Information Theory.

[107]  Mung Chiang,et al.  The value of clustering in distributed estimation for sensor networks , 2005, 2005 International Conference on Wireless Networks, Communications and Mobile Computing.

[108]  Alejandro Ribeiro,et al.  Bandwidth-constrained distributed estimation for wireless sensor Networks-part I: Gaussian case , 2006, IEEE Transactions on Signal Processing.

[109]  G.B. Giannakis,et al.  Distributed compression-estimation using wireless sensor networks , 2006, IEEE Signal Processing Magazine.

[110]  P.K. Varshney,et al.  Channel-aware distributed detection in wireless sensor networks , 2006, IEEE Signal Processing Magazine.

[111]  A.S. Willsky,et al.  Distributed fusion in sensor networks , 2006, IEEE Signal Processing Magazine.

[112]  M. Vetterli,et al.  Sensing reality and communicating bits: a dangerous liaison , 2006, IEEE Signal Processing Magazine.

[113]  H. Vincent Poor,et al.  Consistency in models for distributed learning under communication constraints , 2005, IEEE Transactions on Information Theory.

[114]  H. Vincent Poor,et al.  Distributed Kernel Regression: An Algorithm for Training Collaboratively , 2006, 2006 IEEE Information Theory Workshop - ITW '06 Punta del Este.

[115]  Martin J. Wainwright,et al.  Distributed fusion in sensor networks: a graphical models perspective , 2006 .

[116]  Balázs Kégl,et al.  Privacy-preserving boosting , 2007, Data Mining and Knowledge Discovery.

[117]  Alejandro Ribeiro,et al.  Bandwidth-constrained distributed estimation for wireless sensor networks-part II: unknown probability density function , 2006, IEEE Transactions on Signal Processing.

[118]  Sanjeev R. Kulkarni,et al.  Regression estimation from an individual stable sequence , 2007, ArXiv.

[119]  Sunita Sarawagi.  Learning with Graphical Models , 2008 .