Distributed incremental wLPSVM learning

Weighted linear proximal support vector machine (wLPSVM) is known as an efficient binary classification algorithm with good accuracy and class-imbalance robustness. In this work, original batch wLPSVM is facilitated with distributed incremental learning capability, which allows simultaneously learning from multiple streaming data sources that are geographically distributed. In our approach, incremental and distributed learning are solved as a merging problem at the same time. A new wLPSVM expression is derived. In the new expression, knowledge from samples are presented as a set of class-wised core matrices, and merging knowledge from two subsets of data can be simply accomplished by matrix addition. With the new expression, we are able to conduct incremental and distributed learning at the same time via merging knowledge from multiple incremental stages and multiple data sources.

[1]  Edward Y. Chang,et al.  Parallelizing Support Vector Machines on Distributed Computers , 2007, NIPS.

[2]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[3]  Georgios B. Giannakis,et al.  Consensus-based distributed linear support vector machines , 2010, IPSN '10.

[4]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[5]  Glenn Fung,et al.  Multicategory Proximal Support Vector Machine Classifiers , 2005, Machine Learning.

[6]  Mohammed J. Zaki,et al.  Parallel classification for data mining on shared-memory multiprocessors , 1999, Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337).

[7]  Ichiro Takeuchi,et al.  Multiple Incremental Decremental Learning of Support Vector Machines , 2009, IEEE Transactions on Neural Networks.

[8]  Alexander J. Smola,et al.  Scaling Distributed Machine Learning with the Parameter Server , 2014, OSDI.

[9]  Georgios B. Giannakis,et al.  Consensus-Based Distributed Support Vector Machines , 2010, J. Mach. Learn. Res..

[10]  Dmitry Pechyony,et al.  Solving Large Scale Linear SVM with Distributed Block Minimization , 2011 .

[11]  T. Yamasaki,et al.  Incremental SVMs and Their Geometrical Analyses , 2005, 2005 International Conference on Neural Networks and Brain.

[12]  Ji Hong-bing,et al.  A Modified PSVM and its Application to Unbalanced Data Classification , 2007, Third International Conference on Natural Computation (ICNC 2007).

[13]  Ying Chen,et al.  Efficient text classification by weighted proximal SVM , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[14]  Fuzhen Zhuang,et al.  A parallel incremental extreme SVM classifier , 2011, Neurocomputing.

[15]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[16]  Torsten Rohlfing,et al.  Nonrigid image registration in shared-memory multiprocessor environments with application to brains, breasts, and bees , 2003, IEEE Transactions on Information Technology in Biomedicine.