Communication requirement for distributed statistical machine learning with application in waveform cognition

Distributed learning is an effective approach to mitigate the data communications in machine learning when the data is stored in a distributed manner, particularly in the era of big data. In the distributed learning procedure, learners can send intermediate computation results instead of raw data, thus reducing the communication cost. In this paper, the communication requirement for distributed learning is studied in the scenario of multiple data storage nodes having the capability of learning and a fusion center. Lower bounds for communications are derived based on VC-entropy of modeling in the machine learning. Numerical results are provided to show the communication requirement for typical learning problems.

[1]  John S. Baras,et al.  A distributed learning algorithm with bit-valued communications for multi-agent welfare optimization , 2013, 52nd IEEE Conference on Decision and Control.

[2]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[3]  John Langford,et al.  Scaling up machine learning: parallel and distributed approaches , 2011, KDD '11 Tutorials.

[4]  Maria-Florina Balcan,et al.  Distributed Learning, Communication Complexity and Privacy , 2012, COLT.

[5]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[6]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[7]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[8]  H. Vincent Poor,et al.  Consistency in models for distributed learning under communication constraints , 2005, IEEE Transactions on Information Theory.

[9]  Qing Zhao,et al.  Distributed Learning in Wireless Sensor Networks , 2007 .

[10]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[11]  Raheem A. Beyah,et al.  A Passive Approach to Wireless NIC Identification , 2006, 2006 IEEE International Conference on Communications.

[12]  M.M. Buddhikot,et al.  Passive Steady State RF Fingerprinting: A Cognitive Technique for Scalable Deployment of Co-Channel Femto Cell Underlays , 2008, 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks.

[13]  Li Yu-bin,et al.  On Distributed Learning , 2006 .