Semi-supervised multi-view maximum entropy discrimination with expectation Laplacian regularization

Abstract Semi-supervised multi-view learning has attracted considerable attention and achieved great success in the machine learning field. This paper proposes a semi-supervised multi-view maximum entropy discrimination approach (SMVMED) with expectation Laplacian regularization for data classification. It takes advantage of the geometric information of the marginal distribution embedded in unlabeled data to construct a semi-supervised classifier. Different from existing methods using Laplacian regularization, we propose to use expectation Laplacian regularization for semi-supervised learning in probabilistic models. We give two implementations of SMVMED and provide their kernel variants. One of them can be relaxed and formulated as a quadratic programming problem that is solved easily. Therefore, for this implementation, we provided two versions which are approximate and exact ones. The experiments on one synthetic and multiple real-world data sets show that SMVMED demonstrates superior performance over semi-supervised single-view maximum entropy discrimination, MVMED and other state-of-the-art semi-supervised multi-view learning methods.

[1]  Jun Zhu,et al.  Maximum Entropy Discrimination Markov Networks , 2009, J. Mach. Learn. Res..

[2]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[3]  Tony Jebara,et al.  Multitask Sparsity via Maximum Entropy Discrimination , 2011, J. Mach. Learn. Res..

[4]  Bo Zhang,et al.  Laplace maximum margin Markov networks , 2008, ICML '08.

[5]  Sotirios Chatzis,et al.  Infinite Markov-Switching Maximum Entropy Discrimination Machines , 2013, ICML.

[6]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[7]  R. Bharat Rao,et al.  Bayesian Co-Training , 2007, J. Mach. Learn. Res..

[8]  Mikhail Belkin,et al.  A Co-Regularization Approach to Semi-supervised Learning with Multiple Views , 2005 .

[9]  Yun Fu,et al.  Coupled Marginalized Auto-Encoders for Cross-Domain Multi-View Learning , 2016, IJCAI.

[10]  Jianjun Li.  A two-step rejection procedure for testing multiple hypotheses , 2008 .

[11]  Xuelong Li,et al.  Convex Multiview Semi-Supervised Classification , 2017, IEEE Transactions on Image Processing.

[12]  Tommi S. Jaakkola,et al.  Maximum Entropy Discrimination , 1999, NIPS.

[13]  John Shawe-Taylor,et al.  Two view learning: SVM-2K, Theory and Practice , 2005, NIPS.

[14]  Xuelong Li,et al.  Auto-Weighted Multi-View Learning for Image Clustering and Semi-Supervised Classification , 2018, IEEE Transactions on Image Processing.

[15]  Rayid Ghani,et al.  Analyzing the effectiveness and applicability of co-training , 2000, CIKM '00.

[16]  Vikas Sindhwani,et al.  An RKHS for multi-view learning and manifold co-regularization , 2008, ICML '08.

[17]  Shiliang Sun,et al.  Multi-View Maximum Entropy Discrimination , 2013, IJCAI.

[18]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[19]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[20]  Shiliang Sun,et al.  Multi-kernel maximum entropy discrimination for multi-view learning , 2016, Intell. Data Anal..

[21]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[22]  Shiliang Sun,et al.  Consensus and complementarity based maximum entropy discrimination for multi-view classification , 2016, Inf. Sci..

[23]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[24]  Shiliang Sun,et al.  Applying a multitask feature sparsity method for the classification of semantic relations between nominals , 2012, 2012 International Conference on Machine Learning and Cybernetics.

[25]  K. Doksum.  Robust Procedures for Some Linear Models with one Observation per Cell , 1967 .

[26]  Tony Jebara,et al.  Multi-task feature and kernel selection for SVMs , 2004, ICML.

[27]  Feiping Nie,et al.  The Constrained Laplacian Rank Algorithm for Graph-Based Clustering , 2016, AAAI.

[28]  O. J. Dunn.  Multiple Comparisons among Means , 1961 .

[29]  Xuelong Li,et al.  Self-weighted Multiview Clustering with Multiple Graphs , 2017, IJCAI.

[30]  Zhi-Hua Zhou,et al.  A New Analysis of Co-Training , 2010, ICML.

[31]  G. Hommel.  A stagewise rejective multiple test procedure based on a modified Bonferroni test , 1988 .

[32]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[33]  Shiliang Sun,et al.  Alternative Multiview Maximum Entropy Discrimination , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[34]  John Shawe-Taylor,et al.  Synthesis of maximum margin and multiview learning using unlabeled data , 2007, ESANN.

[35]  David S. Rosenberg,et al.  Multiview point cloud kernels for semisupervised learning [Lecture Notes] , 2009, IEEE Signal Processing Magazine.

[36]  Alfred O. Hero,et al.  Semi-supervised multi-sensor classification via consensus-based Multi-View Maximum Entropy Discrimination , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  Tommi S. Jaakkola,et al.  Feature Selection and Dualities in Maximum Entropy Discrimination , 2000, UAI.

[38]  Shiliang Sun,et al.  Multi-view Laplacian Support Vector Machines , 2011, ADMA.

[39]  Shiliang Sun,et al.  Semi-supervised Multitask Learning via Self-training and Maximum Entropy Discrimination , 2012, ICONIP.

[40]  Shiliang Sun,et al.  Soft Margin Consistency Based Scalable Multi-View Maximum Entropy Discrimination , 2016, IJCAI.

[41]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[42]  Shiliang Sun,et al.  Multi-view Laplacian twin support vector machines , 2014, Applied Intelligence.