Nonlinear Semi-Supervised Metric Learning Via Multiple Kernels and Local Topology

Changing the metric on the data may change the data distribution, hence a good distance metric can promote the performance of learning algorithm. In this paper, we address the semi-supervised distance metric learning (ML) problem to obtain the best nonlinear metric for the data. First, we describe the nonlinear metric by the multiple kernel representation. By this approach, we project the data into a high dimensional space, where the data can be well represented by linear ML. Then, we reformulate the linear ML by a minimization problem on the positive definite matrix group. Finally, we develop a two-step algorithm for solving this model and design an intrinsic steepest descent algorithm to learn the positive definite metric matrix. Experimental results validate that our proposed method is effective and outperforms several state-of-the-art ML methods.

[1]  Josef Kittler,et al.  Multiple Kernel Learning via Distance Metric Learning for Interactive Image Retrieval , 2011, MCS.

[2]  David Griol,et al.  A Two-Stage Combining Classifier Model for the Development of Adaptive Dialog Systems , 2016, Int. J. Neural Syst..

[3]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[4]  Antonio Torralba,et al.  Small codes and large image databases for recognition , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Xuelong Li,et al.  Visual-Textual Joint Relevance Learning for Tag-Based Social Image Search , 2013, IEEE Transactions on Image Processing.

[6]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[7]  Jun Wang,et al.  Metric Learning with Multiple Kernels , 2011, NIPS.

[8]  David Zhang,et al.  A Kernel Classification Framework for Metric Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[10]  Katya Scheinberg,et al.  Block Coordinate Descent Methods for Semidefinite Programming , 2012 .

[11]  Thorsten Joachims,et al.  Learning a Distance Metric from Relative Comparisons , 2003, NIPS.

[12]  Hojjat Adeli,et al.  Spiking Neural Networks , 2009, Int. J. Neural Syst..

[13]  Ethem Alpaydin,et al.  Multiple Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[14]  Hojjat Adeli,et al.  An adaptive conjugate gradient learning algorithm for efficient training of neural networks , 1994 .

[15]  Julio Ortega Lopera,et al.  Leveraging cooperation for parallel multi‐objective feature selection in high‐dimensional EEG data , 2015, Concurr. Comput. Pract. Exp..

[16]  Hojjat Adeli,et al.  A parallel genetic/neural network learning algorithm for MIMD shared memory machines , 1994, IEEE Trans. Neural Networks.

[17]  Pong C. Yuen,et al.  Semi-supervised metric learning via topology preserving multiple semi-supervised assumptions , 2013, Pattern Recognit..

[18]  Changshui Zhang,et al.  Learning Kernels with Radiuses of Minimum Enclosing Balls , 2010, NIPS.

[19]  Qi Wu,et al.  Epileptic Seizure Detection with Log-Euclidean Gaussian Kernel-Based Sparse Representation , 2016, Int. J. Neural Syst..

[20]  Hyo Seon Park,et al.  Neurocomputing for Design Automation , 2018 .

[21]  Hong Qiao,et al.  A New Algorithm for Robust Pedestrian Tracking Based on Manifold Learning and Feature Selection , 2011, IEEE Transactions on Intelligent Transportation Systems.

[22]  Yang Gao,et al.  Structurally Enhanced Incremental Neural Learning for Image Classification with Subgraph Extraction , 2014, Int. J. Neural Syst..

[23]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[24]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[25]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[26]  Qi Wu,et al.  Kernel Collaborative Representation-Based Automatic Seizure Detection in Intracranial EEG , 2015, Int. J. Neural Syst..

[27]  Jun Wang,et al.  A metric learning perspective of SVM: on the relation of LMNN and SVM , 2012, AISTATS.

[28]  Kaizhu Huang,et al.  Low Rank Metric Learning with Manifold Regularization , 2011, 2011 IEEE 11th International Conference on Data Mining.

[29]  Dongmei Fu,et al.  Semi-supervised classification with Laplacian multiple kernel learning , 2014, Neurocomputing.

[30]  T. Lane,et al.  A Framework for Multiple Kernel Support Vector Regression and Its Applications to siRNA Efficacy Prediction , 2009, TCBB.

[31]  Wei Liu,et al.  Semi-supervised distance metric learning for Collaborative Image Retrieval , 2008, CVPR.

[32]  Nicholas Ayache,et al.  Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices , 2007, SIAM J. Matrix Anal. Appl..

[33]  Kilian Q. Weinberger,et al.  Large Margin Multi-Task Metric Learning , 2010, NIPS.

[34]  Hong Chang,et al.  A Kernel Approach for Semisupervised Metric Learning , 2007, IEEE Transactions on Neural Networks.

[35]  Lawrence K. Saul,et al.  Latent Coincidence Analysis: A Hidden Variable Model for Distance Metric Learning , 2012, NIPS.

[36]  Luigi Grippo,et al.  On the convergence of the block nonlinear Gauss-Seidel method under convex constraints , 2000, Oper. Res. Lett..

[37]  Qi Wu,et al.  Using Fractal and Local Binary Pattern Features for Classification of ECOG Motor Imagery Tasks Obtained from the Right Brain Hemisphere , 2016, Int. J. Neural Syst..

[38]  Hojjat Adeli,et al.  Parallel backpropagation learning algorithms on CRAY Y-MP8/864 supercomputer , 1993, Neurocomputing.

[39]  Lorenzo Torresani,et al.  Large Margin Component Analysis , 2006, NIPS.

[40]  Mahdieh Soleymani Baghshah,et al.  Semi-Supervised Metric Learning Using Pairwise Constraints , 2009, IJCAI.

[41]  Hojjat Adeli,et al.  A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection , 2009, Neural Networks.

[42]  Hong Qiao,et al.  Learning an Intrinsic-Variable Preserving Manifold for Dynamic Visual Tracking , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[43]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[44]  Feiping Nie,et al.  Learning a Mahalanobis distance metric for data clustering and classification , 2008, Pattern Recognit..

[45]  Wei Liu,et al.  Semi-supervised distance metric learning for collaborative image retrieval and clustering , 2010, ACM Trans. Multim. Comput. Commun. Appl..

[46]  Radu Horaud,et al.  Acoustic Space Learning for Sound-Source Separation and Localization on Binaural Manifolds , 2014, Int. J. Neural Syst..

[47]  Hojjat Adeli,et al.  Improved spiking neural networks for EEG classification and epilepsy and seizure detection , 2007, Integr. Comput. Aided Eng..

[48]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[49]  Hojjat Adeli,et al.  Enhanced probabilistic neural network with local decision circles: A robust classifier , 2010, Integr. Comput. Aided Eng..

[50]  Maher Moakher,et al.  A Differential Geometric Approach to the Geometric Mean of Symmetric Positive-Definite Matrices , 2005, SIAM J. Matrix Anal. Appl..

[51]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[52]  Joel E. W. Koh,et al.  Computer-Aided Diagnosis of Depression Using EEG Signals , 2015, European Neurology.

[53]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[54]  Yunsong Guo,et al.  Metric Learning: A Support Vector Approach , 2008, ECML/PKDD.