Soft Margin Consistency Based Scalable Multi-View Maximum Entropy Discrimination

Multi-view learning receives increasing interest in recent years to analyze complex data. Lately, multiview maximum entropy discrimination (MVMED) and alternative MVMED (AMVMED) were proposed as extensions of maximum entropy discrimination (MED) to the multi-view learning setting, which use the hard margin consistency principle that enforces two view margins to be the same. In this paper, we propose soft margin consistency based multi-view MED (SMVMED) achieving margin consistency in a less strict way, which minimizes the relative entropy between the posteriors of two view margins. With a trade-off parameter balancing large margin and margin consistency, SMVMED is more flexible. We also propose a sequential minimal optimization (SMO) algorithm to efficiently train SMVMED and make it scalable to large datasets. We evaluate the performance of SMVMED on multiple real-world datasets and get encouraging results.

[1]  Tony Jebara,et al.  Multitask Sparsity via Maximum Entropy Discrimination , 2011, J. Mach. Learn. Res..

[2]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[3]  Shiliang Sun,et al.  Alternative Multiview Maximum Entropy Discrimination , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[5]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[6]  Adolfo Martínez Usó,et al.  UJIIndoorLoc: A new multi-building and multi-floor database for WLAN fingerprint-based indoor localization problems , 2014, 2014 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[7]  Jaime S. Cardoso,et al.  Neural Computing and Applications Robust Classification with Reject Option Using the Self-Organizing Map , 2014 .

[8]  Robert H. Sloan,et al.  Proceedings of the 15th Annual Conference on Computational Learning Theory , 2002 .

[9]  Peter A. Flach,et al.  Proceedings of the 28th International Conference on Machine Learning , 2011 .

[10]  Bo Zhang,et al.  Laplace maximum margin Markov networks , 2008, ICML '08.

[11]  Chih-Jen Lin,et al.  Working Set Selection Using Second Order Information for Training Support Vector Machines , 2005, J. Mach. Learn. Res..

[12]  Dinh Phung,et al.  Journal of Machine Learning Research: Preface , 2014 .

[13]  Shiliang Sun,et al.  Multi-View Maximum Entropy Discrimination , 2013, IJCAI.

[14]  Bo Zhang,et al.  Partially Observed Maximum Entropy Discrimination Markov Networks , 2008, NIPS.

[15]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[16]  Tommi S. Jaakkola,et al.  Feature Selection and Dualities in Maximum Entropy Discrimination , 2000, UAI.

[18]  Nicholas Kushmerick,et al.  Learning to remove Internet advertisements , 1999, AGENTS '99.

[19]  Craig Boutilier,et al.  Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence , 2000 .

[20]  Jun Zhu,et al.  Maximum Entropy Discrimination Markov Networks , 2009, J. Mach. Learn. Res..

[21]  Tony Jebara,et al.  Machine Learning: Discriminative and Generative , 2012 .

[22]  J. Weston,et al.  Support Vector Machine Solvers , 2007 .

[23]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[24]  Michael J. Watts,et al.  IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS Publication Information , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Tommi S. Jaakkola,et al.  Maximum Entropy Discrimination , 1999, NIPS.

[26]  John Shawe-Taylor,et al.  Two view learning: SVM-2K, Theory and Practice , 2005, NIPS.