Integrating local and global topological structures for semi-supervised dimensionality reduction

Dimensionality reduction plays an important role in many machine learning tasks. This paper studies semi-supervised dimensionality reduction using pairwise constraints. In this setting, domain knowledge is given in the form of pairwise constraint, which specifies whether a pair of instances belongs to the same class (must-link constraint) or different classes (cannot-link constraint). In this paper, a novel semi-supervised dimensionality reduction method called LGS3DR is proposed, which can integrate both local and global topological structures of the data as well as pairwise constraints. The LGS3DR method is effective and has a closed form solution. Experiments on data visualization and face recognition show that LGS3DR is superior to many existing dimensionality reduction methods.

[1]  Witold Pedrycz,et al.  Knowledge-based clustering - from data to information granules , 2007 .

[2]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[3]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[4]  Witold Pedrycz,et al.  Algorithms of fuzzy clustering with partial supervision , 1985, Pattern Recognit. Lett..

[5]  Witold Pedrycz,et al.  Fuzzy clustering with partial supervision , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[6]  Jun-Hai Zhai,et al.  An improved algorithm for calculating fuzzy attribute reducts , 2013, J. Intell. Fuzzy Syst..

[7]  Ian Davidson,et al.  Knowledge Driven Dimension Reduction for Clustering , 2009, IJCAI.

[8]  Witold Pedrycz,et al.  Semantic Web Content Analysis: A Study in Proximity-Based Collaborative Clustering , 2007, IEEE Transactions on Fuzzy Systems.

[9]  Claire Cardie,et al.  Clustering with Instance-Level Constraints , 2000, AAAI/IAAI.

[10]  Witold Pedrycz,et al.  Collaborative fuzzy clustering , 2002, Pattern Recognit. Lett..

[11]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Bernhard Schölkopf,et al.  Local learning projections , 2007, ICML '07.

[13]  Jian Yang,et al.  A two-step framework for highly nonlinear data unfolding , 2010, Neurocomputing.

[14]  Witold Pedrycz,et al.  P-FCM: a proximity-based fuzzy clustering for user-centered web applications , 2003, Int. J. Approx. Reason..

[15]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[16]  Tomer Hertz,et al.  Learning a Mahalanobis Metric from Equivalence Constraints , 2005, J. Mach. Learn. Res..

[17]  Xizhao Wang,et al.  Maximum Ambiguity-Based Sample Selection in Fuzzy Decision Tree Induction , 2012, IEEE Transactions on Knowledge and Data Engineering.

[18]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[19]  Hakan Cevikalp,et al.  Semi-Supervised Dimensionality Reduction Using Pairwise Equivalence Constraints , 2008, VISAPP.

[20]  Deli Zhao,et al.  Linear local tangent space alignment and application to face recognition , 2007, Neurocomputing.

[21]  Boonserm Kijsirikul,et al.  A unified semi-supervised dimensionality reduction framework for manifold learning , 2008, Neurocomputing.

[22]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[23]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[24]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[25]  Dan Klein,et al.  From Instance-level Constraints to Space-Level Constraints: Making the Most of Prior Knowledge in Data Clustering , 2002, ICML.

[26]  Witold Pedrycz,et al.  P-FCM: a proximity -- based fuzzy clustering , 2004, Fuzzy Sets Syst..

[27]  Feiping Nie,et al.  A unified framework for semi-supervised dimensionality reduction , 2008, Pattern Recognit..

[28]  Honggang Zhang,et al.  Comments on "Globally Maximizing, Locally Minimizing: Unsupervised Discriminant Projection with Application to Face and Palm Biometrics" , 2007, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Dongwon Lee,et al.  Semi-supervised dimensionality reduction for analyzing high-dimensional data with constraints , 2012, Neurocomputing.

[30]  Jun-Hai Zhai,et al.  Fuzzy decision tree based on fuzzy-rough technique , 2011, Soft Comput..

[31]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Hong Chang,et al.  Extending the relevant component analysis algorithm for metric learning using both positive and negative equivalence constraints , 2006, Pattern Recognit..

[33]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Konstantinos N. Plataniotis,et al.  Face recognition using LDA-based algorithms , 2003, IEEE Trans. Neural Networks.

[35]  Jieping Ye,et al.  Integrating Global and Local Structures: A Least Squares Framework for Dimensionality Reduction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Svetha Venkatesh,et al.  Exploiting side information in locality preserving projection , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Mahdieh Soleymani Baghshah,et al.  Semi-Supervised Metric Learning Using Pairwise Constraints , 2009, IJCAI.

[38]  Daoqiang Zhang,et al.  Semisupervised Dimensionality Reduction With Pairwise Constraints for Hyperspectral Image Classification , 2011, IEEE Geoscience and Remote Sensing Letters.

[39]  I. Jolliffe.  Principal Component Analysis , 2002 .

[40]  Wei Tang,et al.  Pairwise Constraints-Guided Dimensionality Reduction , 2007 .

[41]  Xizhao Wang,et al.  Dynamic ensemble extreme learning machine based on sample entropy , 2012, Soft Comput..

[42]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[43]  Daoqiang Zhang,et al.  Semi-Supervised Dimensionality Reduction ∗ , 2007 .

[44]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[45]  David G. Stork,et al.  Pattern Classification , 1973 .

[46]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[47]  Jia Wei,et al.  Neighbourhood preserving based semi-supervised dimensionality reduction , 2008 .