Feature Extraction Using ICA

In manipulating data such as in supervised learning, we often extract new features from original features for the purpose of reducing the dimensions of feature space and achieving better performances. In this paper, we propose a new feature extraction algorithm using independent component analysis (ICA) for classification problems. By using ICA in solving supervised classification problems, we can get new features which are made as independent from each other as possible and also convey the output information faithfully. Using the new features along with the conventional feature selection algorithms, we can greatly reduce the dimension of feature space without degrading the performance of classifying systems.

[1]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[2]  Chong-Ho Choi,et al.  Improved mutual information feature selector for neural networks in supervised learning , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[3]  Thomas P. Trappenberg,et al.  Input variable selection using independent component analysis , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  John E. Moody,et al.  Data Visualization and Feature Selection: New Algorithms for Nongaussian Data , 1999, NIPS.

[6]  Robert A. Lordo,et al.  Learning from Data: Concepts, Theory, and Methods , 2001, Technometrics.

[7]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[8]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection , 1998 .

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Erkki Oja,et al.  Image feature extraction by sparse coding and independent component analysis , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[11]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..