A Discrimination Analysis for Unsupervised Feature Selection via Optic Diffraction Principle

This paper proposes an unsupervised discrimination analysis for feature selection based on a property of the Fourier transform of the probability density distribution. Each feature is evaluated on the basis of a simple observation motivated by the concept of optical diffraction, which is invariant under feature scaling. The time complexity is O(mn), where m is number of features and n is number of instances when being applied directly to the given data. This approach is also extended to deal with data orientation, which is the direction of data alignment. Therefore, the discrimination score of any transformed space can be used for evaluating the original features. The experimental results on several real-world datasets demonstrate the effectiveness of the proposed method.

[1]  Yiu-ming Cheung,et al.  A new feature selection method for Gaussian mixture clustering , 2009, Pattern Recognit..

[2]  C. A. Murthy,et al.  Unsupervised Feature Selection Using Feature Similarity , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Dawei Song,et al.  Beyond Redundancies: A Metric-Invariant Method for Unsupervised Feature Selection , 2010, IEEE Transactions on Knowledge and Data Engineering.

[4]  Kari Torkkola,et al.  Feature Extraction by Non-Parametric Mutual Information Maximization , 2003, J. Mach. Learn. Res..

[5]  S. Niijima,et al.  Laplacian Linear Discriminant Analysis Approach to Unsupervised Feature Selection , 2009, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[6]  B. W. Silverman,et al.  Probability, Statistics and Analysis: Some properties of a test for multimodality based on kernel density estimates , 1983 .

[7]  Deniz Erdogmus,et al.  Feature extraction using information-theoretic learning , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Jieping Ye,et al.  Generalized Linear Discriminant Analysis: A Unified Framework and Efficient Model Selection , 2008, IEEE Transactions on Neural Networks.

[9]  Anil K. Jain,et al.  Simultaneous feature selection and clustering using mixture models , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[11]  Michal Linial,et al.  Novel Unsupervised Feature Filtering of Biological Data , 2006, ISMB.

[12]  George Kesidis,et al.  Margin-Maximizing Feature Elimination Methods for Linear and Nonlinear Kernel-Based Discriminant Functions , 2010, IEEE Transactions on Neural Networks.

[13]  E. Lander,et al.  MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia , 2002, Nature Genetics.

[14]  Dirk P. Kroese,et al.  Kernel density estimation via diffusion , 2010, 1011.2602.

[15]  Jacek M. Zurada,et al.  Normalized Mutual Information Feature Selection , 2009, IEEE Transactions on Neural Networks.

[16]  Yiu-ming Cheung,et al.  Local Kernel Regression Score for Selecting Features of High-Dimensional Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[17]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[18]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[19]  Martin Vetterli,et al.  Fast Fourier transforms: a tutorial review and a state of the art , 1990 .

[20]  T. Caliński,et al.  A dendrite method for cluster analysis , 1974 .

[21]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[22]  Pierre Comon,et al.  A Contrast Function for Independent Component Analysis Without Permutation Ambiguity , 2010, IEEE Transactions on Neural Networks.

[23]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[24]  M. Ringnér,et al.  Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks , 2001, Nature Medicine.

[25]  Yi Li,et al.  Multimodality as a criterion for feature selection in unsupervised analysis of gene expression data , 2005, Fifth IEEE Symposium on Bioinformatics and Bioengineering (BIBE'05).

[26]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[27]  David T. Neilson,et al.  Diffraction-grating-based (de)multiplexer using image plane transformations , 2002 .

[28]  J. Goodman.  Introduction to Fourier optics , 1969 .

[29]  Jing Hua,et al.  Simultaneous Localized Feature Selection and Model Detection for Gaussian Mixtures , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[31]  Huan Liu,et al.  Efficient Feature Selection via Analysis of Relevance and Redundancy , 2004, J. Mach. Learn. Res..

[32]  Samuel Kaski,et al.  Discriminative components of data , 2005, IEEE Transactions on Neural Networks.

[33]  Lianwen Jin,et al.  An unsupervised feature ranking scheme by discovering biclusters , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[34]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.