Distributed training of multiclass conic-segmentation support vector machines on communication constrained networks

We present a distributed algorithm for training multiclass conic-segmentation support vector machines (CS-SVMs) on communication-constrained networks. The proposed algorithm takes advantage of the sparsity of the CS-SVM to minimise the communication overhead between nodes during training to obtain classifiers at each node which closely approximate the optimal (centralised) classifier. The proposed algorithm is also suited for wireless sensor networks where inter-node communication is limited by power restrictions and bandwidth. We demonstrate our algorithm by applying it to two datasets, one simulated and one benchmark dataset, to show that the global decision functions found by the nodes closely approximate the optimal decision function found by a centralised algorithm possessing all training data in one batch.

[1]  Giles M. Foody,et al.  Multiclass and Binary SVM Classification: Implications for Training and Classification Users , 2008, IEEE Geoscience and Remote Sensing Letters.

[2]  Sutharshan Rajasegarar,et al.  Anomaly detection by clustering ellipsoids in wireless sensor networks , 2009, 2009 International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP).

[3]  Mani Srivastava,et al.  Energy-aware wireless microsensor networks , 2002, IEEE Signal Process. Mag..

[4]  D Thukaram,et al.  Comparison of Multiclass SVM Classification Methods to Use in a Supportive System for Distance Relay Coordination , 2010, IEEE Transactions on Power Delivery.

[5]  Gert Cauwenberghs,et al.  SVM incremental learning, adaptation and optimization , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[6]  Emilio Parrado-Hernández,et al.  Distributed support vector machines , 2006, IEEE Trans. Neural Networks.

[7]  Bo-Suk Yang,et al.  Application of nonlinear feature extraction and support vector machines for fault diagnosis of induction motors , 2007, Expert Syst. Appl..

[8]  Marimuthu Palaniswami,et al.  Incremental training of support vector machines , 2005, IEEE Transactions on Neural Networks.

[9]  R. Shah,et al.  Least Squares Support Vector Machines , 2022 .

[10]  Marimuthu Palaniswami,et al.  Clustering ellipses for anomaly detection , 2011, Pattern Recognit..

[11]  Panagiotis Tsakalides,et al.  Training a SVM-based classifier in distributed sensor networks , 2006, 2006 14th European Signal Processing Conference.

[12]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[13]  Robert T. Schultz,et al.  Nonlinear Estimation and Modeling of fMRI Data Using Spatio-temporal Support Vector Regression , 2003, IPMI.

[14]  Leonidas J. Guibas,et al.  Collaborative signal and information processing: an information-directed approach , 2003 .

[15]  M. Palaniswami,et al.  Distributed Anomaly Detection in Wireless Sensor Networks , 2006, 2006 10th IEEE Singapore International Conference on Communication Systems.

[16]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[17]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[18]  Daniel Minoli,et al.  Wireless Sensor Networks: Technology, Protocols, and Applications , 2007 .

[19]  Joseph M. Hellerstein,et al.  Public Health for the Internet (') Towards A New Grand Challenge for Information Management , 2007 .

[21]  Ye Li,et al.  Fault diagnosis based on support vector machine ensemble , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[22]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[23]  Peter Desnoyers,et al.  Ultra-low power data storage for sensor networks , 2009, TOSN.

[24]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[25]  Baltasar Beferull-Lozano,et al.  Distributed consensus algorithms for SVM training in wireless sensor networks , 2008, 2008 16th European Signal Processing Conference.

[26]  Marimuthu Palaniswami,et al.  Centered Hyperspherical and Hyperellipsoidal One-Class Support Vector Machines for Anomaly Detection in Sensor Networks , 2010, IEEE Transactions on Information Forensics and Security.

[27]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[28]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[29]  Jason Weston,et al.  Support vector machines for multi-class pattern recognition , 1999, ESANN.

[30]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[31]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[32]  Bhavani M. Thuraisingham,et al.  A new intrusion detection system using support vector machines and hierarchical clustering , 2007, The VLDB Journal.

[33]  Marimuthu Palaniswami,et al.  Elliptical anomalies in wireless sensor networks , 2009, TOSN.

[34]  Mario Di Francesco,et al.  Energy conservation in wireless sensor networks: A survey , 2009, Ad Hoc Networks.

[35]  KhanLatifur,et al.  A new intrusion detection system using support vector machines and hierarchical clustering , 2007, VLDB 2007.

[36]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[37]  Marimuthu Palaniswami,et al.  Protein Secondary Structure Prediction Using Support Vector Machines and a New Feature Representation , 2006, Int. J. Comput. Intell. Appl..

[38]  Kotagiri Ramamohanarao,et al.  Survey of network-based defense mechanisms countering the DoS and DDoS problems , 2007, CSUR.

[39]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[40]  S. Abe,et al.  Fuzzy support vector machines for pattern classification , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[41]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .