Learning a Propagable Graph for Semisupervised Learning: Classification and Regression

In this paper, we present a novel framework, called learning by propagability, for two essential data mining tasks, i.e., classification and regression. The whole learning process is driven by the philosophy that the data labels and the optimal feature representation jointly constitute a harmonic system, where the data labels are invariant with respect to the propagation on the similarity graph constructed based on the optimal feature representation. Based on this philosophy, a unified framework of learning by propagability is proposed for the purposes of both classification and regression. Specifically, this framework has three characteristics: 1) the formulation unifies the label propagation and optimal feature representation pursuing, and thus the label propagation process is enhanced by benefiting from the refined similarity graph constructed with the derived optimal feature representation instead of the original representation; 2) it unifies the formulations for supervised and semisupervised learning in both classification and regression tasks; and 3) it can directly deal with the multiclass classification problems. Extensive experiments for the classification task on UCI toy data sets, handwritten digit recognition, face recognition, and microarray recognition as well as for the regression task of human age estimation on the FG-NET aging database, all validate the effectiveness of our proposed learning framework, compared with the state-of-the-art counterparts.

[1]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[2]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[3]  Hiroyasu Koshimizu,et al.  Method for estimating and modeling age and gender using facial image processing , 2001, Proceedings Seventh International Conference on Virtual Systems and Multimedia.

[4]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[5]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[6]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[7]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[8]  Yi Liu,et al.  An Efficient Algorithm for Local Distance Metric Learning , 2006, AAAI.

[9]  Yun Fu,et al.  Human Age Estimation With Regression on Discriminative Aging Manifold , 2008, IEEE Transactions on Multimedia.

[10]  Shuicheng Yan,et al.  Regression From Uncertain Labels and Its Applications to Soft Biometrics , 2008, IEEE Transactions on Information Forensics and Security.

[11]  Weiguo Fan,et al.  Effective and efficient dimensionality reduction for large-scale and streaming data preprocessing , 2006, IEEE Transactions on Knowledge and Data Engineering.

[12]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Bingbing Ni,et al.  Learning by Propagability , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[14]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Yun Fu,et al.  Image-Based Human Age Estimation by Manifold Learning and Locally Adjusted Robust Regression , 2008, IEEE Transactions on Image Processing.

[16]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[17]  C. Christodoulou,et al.  Comparing different classifiers for automatic age estimation , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Zhi-Hua Zhou,et al.  Automatic Age Estimation Based on Facial Aging Patterns , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Mikhail Belkin,et al.  Regularization and Semi-supervised Learning on Large Graphs , 2004, COLT.

[20]  Tomer Hertz,et al.  Learning Distance Functions using Equivalence Relations , 2003, ICML.

[21]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[22]  Geoffrey E. Hinton,et al.  Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure , 2007, AISTATS.

[23]  Mikhail Belkin,et al.  Tikhonov regularization and semi-supervised learning on large graphs , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[24]  Niels da Vitoria Lobo,et al.  Age Classification from Facial Images , 1999, Comput. Vis. Image Underst..

[25]  Ming Liu,et al.  Regression from patch-kernel , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Jiawei Han,et al.  Semi-supervised Discriminant Analysis , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[27]  Shuicheng Yan,et al.  Ranking with Uncertain Labels , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[28]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[29]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[30]  I. Jolliffe Principal Component Analysis , 2002 .

[31]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.