Heteroscedastic max-min distance analysis

Many discriminant analysis methods such as LDA and HLDA actually maximize the average pairwise distances between classes, which often causes the class separation problem. Max-min distance analysis (MMDA) addresses this problem by maximizing the minimum pairwise distance in the latent subspace, but it is developed under the homoscedastic assumption. This paper proposes Heteroscedastic MMDA (HMMDA) methods that explore the discriminative information in the difference of intra-class scatters for dimensionality reduction. WHMMDA maximizes the minimal pairwise Chenoff distance in the whitened space. OHMMDA incorporates this objective and the minimization of class compactness into a trace quotient formulation and imposes an orthogonal constraint to the final transformation, which can be solved by a bisection search algorithm. Two variants of OHMMDA are further proposed to encode the margin information. Experiments on several UCI Machine Learning datasets and the Yale Face database demonstrate the effectiveness of the proposed HMMDA methods.

[1]  Robert P. W. Duin,et al.  Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  C. R. Rao,et al.  The Utilization of Multiple Measurements in Problems of Biological Classification , 1948 .

[3]  Alexandros Iosifidis,et al.  On the Optimal Class Representation in Linear Discriminant Analysis , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Luis Rueda,et al.  On the Performance of Chernoff-Distance-Based Linear Dimensionality Reduction Techniques , 2006, Canadian Conference on AI.

[5]  Xiaoou Tang,et al.  Tensor linear Laplacian discrimination (TLLD) for feature extraction , 2009, Pattern Recognit..

[6]  Yaoliang Yu,et al.  Distance metric learning by minimal distance maximization , 2011, Pattern Recognit..

[7]  Robert P. W. Duin,et al.  Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Hongdong Li,et al.  Supervised dimensionality reduction via sequential semidefinite programming , 2008, Pattern Recognit..

[9]  Luis Rueda,et al.  Linear dimensionality reduction by maximizing the Chernoff distance in the transformed space , 2008, Pattern Recognit..

[10]  Aleix M. Martínez,et al.  Subclass discriminant analysis , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Dacheng Tao,et al.  Max-Min Distance Analysis by Using Sequential SDP Relaxation for Dimension Reduction , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Xiaoqing Ding,et al.  Linear Sequence Discriminant Analysis: A Model-Based Dimensionality Reduction Method for Vector Sequences , 2013, 2013 IEEE International Conference on Computer Vision.

[14]  Guna Seetharaman,et al.  Chernoff Dimensionality Reduction-Where Fisher Meets FKT , 2011, SDM.

[15]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[16]  Andreas G. Andreou,et al.  Heteroscedastic discriminant analysis and reduced rank HMMs for improved speech recognition , 1998, Speech Commun..

[17]  David J. Kriegman,et al.  The yale face database , 1997 .

[18]  Michael L. Overton,et al.  On the Sum of the Largest Eigenvalues of a Symmetric Matrix , 1992, SIAM J. Matrix Anal. Appl..

[19]  Cheng-Lin Liu,et al.  Evaluation of weighted Fisher criteria for large category dimensionality reduction in application to Chinese handwriting recognition , 2013, Pattern Recognit..

[20]  Jiawei Han,et al.  Orthogonal Laplacianfaces for Face Recognition , 2006, IEEE Transactions on Image Processing.

[21]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Aleix M. Martínez,et al.  Bayes Optimality in Linear Discriminant Analysis , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Gang Hua,et al.  Face Recognition using Discriminatively Trained Orthogonal Rank One Tensor Projections , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[25]  Kim-Chuan Toh,et al.  Solving semidefinite-quadratic-linear programs using SDPT3 , 2003, Math. Program..

[26]  Masashi Sugiyama,et al.  Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis , 2007, J. Mach. Learn. Res..

[27]  Dit-Yan Yeung,et al.  Worst-Case Linear Discriminant Analysis , 2010, NIPS.

[28]  Thomas S. Huang,et al.  Facial expression recognition: A clustering-based approach , 2003, Pattern Recognit. Lett..