A study of feature extraction using supervised independent component analysis

Recently, independent component analysis (ICA) has been applied to not only problems of blind signal separation, but also feature extraction of images and sounds. In this paper, we study the effectiveness of Umeyama's (1999) supervised ICA (SICA) for feature extraction of handwritten characters. Two types of control vectors (supervisor) are proposed for SICA: 1) average patterns (Type-I); and 2) eigen-patterns (Type-II). To demonstrate the usefulness of SICA, recognition performance is evaluated for handwritten digits that are included in the MNIST database. From the results of recognition experiments, we certify that SICAs with both types of control vectors work effective for feature extraction. Actually, the within-class variance between-class variance ratio of SICA features with Type-I control vectors becomes slightly larger as compared with a conventional ICA.

[1]  H. Martin,et al.  ndependent component representations for face recognition * , 2022 .

[2]  Seiichi Ozawa,et al.  A Study of Feature Extraction and Selection Using Independent Component Analysis , 2000 .

[3]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[4]  Norio Baba,et al.  Application of independent component analysis to handwritten Japanese character recognition , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[5]  Erkki Oja,et al.  Applications of neural blind separation to signal and image processing , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[6]  Manabu Kotani,et al.  Application of independent component analysis to feature extraction of speech , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[7]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[8]  Marian Stewart Bartlett,et al.  Independent component representations for face recognition , 1998, Electronic Imaging.

[9]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[10]  Aapo Hyvärinen,et al.  Survey on Independent Component Analysis , 1999 .

[11]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[12]  J. Karhunen,et al.  A bigradient optimization approach for robust PCA, MCA, and source separation , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.