Classification based on local feature selection via linear programming

This paper presents a novel local feature selection and classification method, which finds the most discriminative features for different regions of the feature space. To this end, we consider each sample of the training set to be a “representative point” of its associated class. A feature set (possibly different in size and members) is assigned to each representative point. The process of finding a feature set for each representative point is independent of the others and can be performed in parallel. The proposed method makes no assumptions about the underlying structure of the training set; hence the method is insensitive to the distribution of the data over the feature space. The method is formulated as a linear programming optimization problem, which has a very efficient realization. Experimental results demonstrate the viability of the formulation and the effectiveness of the proposed algorithm.

[1]  Andrea Lagorio,et al.  On the Use of SIFT Features for Face Authentication , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[2]  Derek Sleeman,et al.  Proceedings of the Ninth International Workshop on Machine Learning , 1992 .

[3]  Lei Wang,et al.  Feature Selection with Kernel Class Separability , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Hiroshi Motoda,et al.  Computational Methods of Feature Selection , 2022 .

[5]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  George Mavrotas,et al.  Effective implementation of the epsilon-constraint method in Multi-Objective Mathematical Programming problems , 2009, Appl. Math. Comput..

[7]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[8]  Thomas G. Dietterich.  Machine-Learning Research Four Current Directions , 1997 .

[9]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[10]  Zheng Bao,et al.  Large Margin Feature Weighting Method via Linear Programming , 2009, IEEE Transactions on Knowledge and Data Engineering.

[11]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[12]  Thomas G. Dietterich.  Machine-Learning Research , 1997, AI Mag..

[13]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.

[14]  Jian Li,et al.  Iterative RELIEF for feature weighting , 2006, ICML.

[15]  P. Langley.  Selection of Relevant Features in Machine Learning , 1994 .

[16]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[17]  Majid Nili Ahmadabadi,et al.  Optimal Local Basis: A Reinforcement Learning Approach for Face Recognition , 2009, International Journal of Computer Vision.

[18]  Sinisa Todorovic,et al.  Local-Learning-Based Feature Selection for High-Dimensional Data Analysis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[20]  Chong-Ho Choi,et al.  Input Feature Selection by Mutual Information Based on Parzen Window , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Naftali Tishby,et al.  Margin based feature selection - theory and algorithms , 2004, ICML.

[23]  C. Hwang.  Multiple Objective Decision Making - Methods and Applications: A State-of-the-Art Survey , 1979 .

[24]  Yijun Sun,et al.  Iterative RELIEF for Feature Weighting: Algorithms, Theories, and Applications , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Jun Luo,et al.  Person-Specific SIFT Features for Face Recognition , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[26]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.