Find the intrinsic space for multiclass classification

Multiclass classification is one of the core problems in many applications. High classification accuracy is fundamental to be accepted as a valuable or even indispensable tool in the work flow. In the classification problem, each sample is usually represented as a vector of features. Most of the cases, some features are usually redundant or misleading, and high dimension is not necessary. Therefore, it is important to find the intrinsically lower dimensional space to get the most representative features that contain the best information for classification. In this paper, we propose a novel dimension reduction method for multiclass classification. Using the constraint of the triplet set, our proposed method projects the original high dimensional feature space to a much lower dimensional feature space. This method enables faster computation, reduce the space needed, and mostly importantly produces more meaningful representations that leads to better classification accuracy.

[1]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[2]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[3]  Michael C. Hout,et al.  Multidimensional Scaling , 2003, Encyclopedic Dictionary of Archaeology.

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[6]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[7]  Amaury Lendasse,et al.  A robust nonlinear projection method , 2000 .

[8]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, CVPR.

[9]  Jinbo Bi,et al.  Dimensionality Reduction via Sparse Support Vector Machines , 2003, J. Mach. Learn. Res..

[10]  Frank Nielsen,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. 1 Total Bregman Divergence and its Applications to DTI Analysis , 2022 .

[11]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[12]  Lei Wang,et al.  Positive Semidefinite Metric Learning with Boosting , 2009, NIPS.

[13]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[14]  David G. Stork,et al.  Pattern Classification , 1973 .

[15]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[16]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[17]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[18]  Frank Nielsen,et al.  Total Bregman divergence and its applications to shape retrieval , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Le Lu,et al.  Sparse Classification for Computer Aided Diagnosis Using Learned Dictionaries , 2011, MICCAI.

[20]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Xiaojing Ye,et al.  Coarse-to-fine classification via parametric and nonparametric models for computer-aided diagnosis , 2011, CIKM '11.