Feature selection algorithms for the generation of multiple classifier systems and their application to handwritten word recognition

The study of multiple classifier systems has become an area of intensive research in pattern recognition recently. Also in handwriting recognition, systems combining several classifiers have been investigated. In this paper new methods for the creation of classifier ensembles based on feature selection algorithms are introduced. Those new methods are evaluated and compared to existing approaches in the context of handwritten word recognition, using a hidden Markov model recognizer as basic classifier.

[1]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[2]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[3]  Pavel Pudil,et al.  Oscillating search algorithms for feature selection , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[4]  H.M.R. Abou-zeid,et al.  Computer recognition of unconstrained handwritten numerals , 2003, 2003 46th Midwest Symposium on Circuits and Systems.

[5]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[6]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[7]  Horst Bunke,et al.  Automatic bankcheck processing , 1997 .

[8]  Josef Kittler,et al.  Multiple Classifier Systems: First International Workshop, MCS 2000 Cagliari, Italy, June 21-23, 2000 Proceedings , 2000 .

[9]  Horst Bunke,et al.  Automatic segmentation of the IAM off-line database for handwritten English text , 2002, Object recognition supported by user interaction for service robots.

[10]  Horst Bunke,et al.  Using a Statistical Language Model to Improve the Performance of an HMM-Based Cursive Handwriting Recognition System , 2001, Int. J. Pattern Recognit. Artif. Intell..

[11]  David A. Hull.  Using statistical testing in the evaluation of retrieval experiments , 1993, SIGIR.

[12]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[13]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[14]  Gyeonghwan Kim,et al.  An architecture for handwritten text recognition systems , 1999, International Journal on Document Analysis and Recognition.

[15]  Gerard Salton,et al.  Research and Development in Information Retrieval , 1982, Lecture Notes in Computer Science.

[16]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Seong-Whan Lee,et al.  Advances in Handwriting Recognition , 1999, Series in Machine Perception and Artificial Intelligence.

[18]  J.-C. Simon,et al.  Off-line cursive word recognition , 1992, Proc. IEEE.

[19]  Horst Bunke,et al.  Creation of classifier ensembles for handwritten word recognition using feature selection algorithms , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[20]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[21]  David D. Denison,et al.  Nonlinear estimation and classification , 2003 .

[22]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[23]  Ching Y. Suen,et al.  Combination of multiple classifiers with measurement values , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[24]  O. Firschein,et al.  Syntactic pattern recognition and applications , 1983, Proceedings of the IEEE.

[25]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[27]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1995, COLT '90.

[28]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Feature Subset Selection , 1977, IEEE Transactions on Computers.

[29]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[30]  Horst Bunke,et al.  The IAM-database: an English sentence database for offline handwriting recognition , 2002, International Journal on Document Analysis and Recognition.

[31]  Torsten Caesar,et al.  Sophisticated topology of hidden Markov models for cursive script recognition , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[32]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[33]  Kagan Tumer,et al.  Input Decimation Ensembles: Decorrelation through Dimensionality Reduction , 2001, Multiple Classifier Systems.

[34]  K. Maruyama,et al.  RECOGNITION METHOD FOR CURSIVE JAPANESE WORD WRITTEN IN LATIN CHARACTERS , 2004 .

[35]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Ching Y. Suen,et al.  Multiple Classifier Combination Methodologies for Different Output Levels , 2000, Multiple Classifier Systems.

[37]  Sebastiano Impedovo,et al.  Frontiers in Handwriting Recognition , 1994 .

[38]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[39]  Lee Luan Ling,et al.  Disconnected handwritten numeral image recognition , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[40]  Josef Kittler,et al.  Advances in Statistical Feature Selection , 2001, ICAPR.