Feature extraction by supervised independent component analysis based on category information

Independent component analysis (ICA) is a technique of transforming observation signals into their unknown independent components; hence, ICA has often been applied to blind signal separation problems. In this application, it is expected that the obtained independent components extract essential information of independent signal sources from input data in an unsupervised fashion. Based on such characteristics, ICA is currently utilized as a feature extraction method for images and sounds for recognition purposes. However, since ICA is an unsupervised learning, the obtained independent components are not always useful in recognition. To overcome this problem, we propose a supervised approach to ICA using category information. The proposed method is implemented in a conventional three‐layered neural network, but its objective function to be minimized is defined not only for the output layer but also for the hidden layer. The objective function consists of the following two terms: one evaluates the kurtosis of hidden unit outputs and the other evaluates the error between output signals and their teacher signals. The experiments are performed using several standard datasets to evaluate performance of the proposed algorithm. It is confirmed that a higher recognition accuracy is attained by the proposed method as compared with a conventional ICA algorithm. © 2007 Wiley Periodicals, Inc. Electr Eng Jpn, 161(2): 25–32, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/eej.20522

[1]  Shigeo Abe,et al.  Support Vector Machines for Pattern Classification , 1999, Advances in Pattern Recognition.

[2]  E. Oja,et al.  Independent Component Analysis , 2001 .

[3]  Isaac G. Councill,et al.  An Introduction to Support Vector Machines: A Review , 2003, AI Mag..

[4]  Manabu Kotani,et al.  Feature Extraction of Digit Patterns Utilizing Independent Component Analysis , 2002 .

[5]  Lakhmi C. Jain,et al.  Self-Organizing Neural Networks , 2002 .

[6]  Manabu Kotani,et al.  Feature extraction using supervised independent component analysis by maximizing class distance , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[7]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[8]  Manabu Kotani,et al.  Signal Processing of Speech Using Independent Component Analysis Based on Information Maximization Algorithm , 2000 .

[9]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[10]  P O Hoyer,et al.  Independent component analysis applied to feature extraction from colour and stereo images , 2000, Network.

[11]  Seiichi Ozawa,et al.  A Study of Feature Extraction and Selection Using Independent Component Analysis , 2000 .

[12]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[13]  Mark Girolami,et al.  Self-Organizing Neural Networks , 1999 .

[14]  K. Bennett,et al.  A support vector machine approach to decision trees , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[15]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[16]  T J Sejnowski,et al.  Learning the higher-order structure of a natural sound. , 1996, Network.

[17]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.