Dimension Reduction by Maximizing Pairwise Discriminations

Dimension reduction is an important pre-processing technique for high-dimensional data analysis. In this paper, we consider the process of linear dimension reduction (LDR) in multiclass problems. We propose a novel feature extraction method based on Minimax Probability Machine (MPM), named MPMbased Dimension Reduction (DR-MPM). Its objective naturally combines the discriminative information of all the class pairs and each pair of classes is well separated in the projected subspace. The algorithm is robust in the sense that it is insensitive to 'outlier' classes which lie far away from other classes. We evaluate DR-MPM on a number of synthetic and real-world data sets, and show that it outperforms other state-of-art feature extraction methods in terms of visual intuition and classification accuracy, especially when the distances between classes are unevenly distributed.