Multilevel Data Classification and Function Approximation Using Hierarchical Neural Networks

Combining diverse features and multiple classifiers is an open research area in which no optimal strategy is found but successful experimental studies have been performed depending on a specific task at hand. In this chapter, a strategy for combining diverse features and multiple classifiers is presented as an exemplary new model in multilevel data classification using hierarchical neural networks. In the proposed strategy, each feature set and each classifier extracts its own representation from the raw data which results with measurements extracted from the original data (or a subset of original data) that are unique to each level of approximation/classification. Later on, the results of each level are linearly combined in function approximation or merged in classification. It is shown by advanced signal and image processing applications that proposed model of combining features/classifiers is especially important for applications that require integration of different types of features and classifiers.

[1]  Ke Chen A connectionist method for pattern classification with diverse features , 1998, Pattern Recognit. Lett..

[2]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Hsi-Jian Lee,et al.  Multi-stage pre-candidate selection in handwritten chinese character recognition systems , 1994, Pattern Recognit..

[4]  Fumitaka Kimura,et al.  Handwritten numerical recognition based on multiple algorithms , 1991, Pattern Recognit..

[5]  D. H. Kil,et al.  Pattern recognition and prediction with applications to signal characterization , 1996 .

[6]  Ke Chen,et al.  A method of combining multiple probabilistic classifiers through soft competition on different feature sets , 1998, Neurocomputing.

[7]  Dmitry A. Denisov,et al.  Model-based chromosome recognition via hypotheses construction/verification , 1994, Pattern Recognit. Lett..

[8]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Luis-Manuel Tomás-Balibrea,et al.  Automatic system for quality-based classification of marble textures , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[10]  H. Hackbarth,et al.  Modular connectionist structure for 100-word recognition , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[11]  Jianhua Zhang,et al.  Combining multiple precision-boosted classifiers for indoor-outdoor scene classification , 2005, Third International Conference on Information Technology and Applications (ICITA'05).

[12]  Shun-ichi Amari,et al.  Combining Classifiers and Learning Mixture-of-Experts , 2009, Encyclopedia of Artificial Intelligence.

[13]  Okan Önal,et al.  Cascaded and Hierarchical Neural Networks for Classifying Surface Images of Marble Slabs , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[14]  Nicola D'Apuzzo,et al.  Modeling human faces with multi-image photogrammetry , 2002, IS&T/SPIE Electronic Imaging.

[15]  Luís A. Alexandre,et al.  On combining classifiers using sum and product rules , 2001, Pattern Recognit. Lett..

[16]  Bruce W. Schmeiser,et al.  Improving model accuracy using optimal linear combinations of trained neural networks , 1995, IEEE Trans. Neural Networks.

[17]  Lalit Gupta,et al.  Three-layer perceptron based classifiers for the partial shape classification problem , 1994, Pattern Recognit..

[18]  Ludmila I. Kuncheva,et al.  "Fuzzy" versus "nonfuzzy" in combining classifiers designed by Boosting , 2003, IEEE Trans. Fuzzy Syst..

[19]  Daw-Tung Lin,et al.  Facial expressions classification with hierarchical radial basis function networks , 1999, ICONIP'99. ANZIIS'99 & ANNES'99 & ACNN'99. 6th International Conference on Neural Information Processing. Proceedings (Cat. No.99EX378).

[20]  Marek Kurzynski On the identity of optimal strategies for multistage classifiers , 1989, Pattern Recognit. Lett..

[21]  M. Perrone Improving regression estimation: Averaging methods for variance reduction with extensions to general convex measure optimization , 1993 .

[22]  Josef Kittler,et al.  Weighting Factors in Multiple Expert Fusion , 1997, BMVC.

[23]  Josef Kittler,et al.  Automatic system for surface inspection and sorting of tiles , 1998 .

[24]  Mauro Maggioni,et al.  Multiscale approximation with hierarchical radial basis functions networks , 2004, IEEE Transactions on Neural Networks.

[25]  Zixing Cai,et al.  Advances of Research in Fuzzy Integral for Classifiers' fusion , 2007, Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing.

[26]  Derek Partridge,et al.  Multiple Classifier Systems: Software Engineered, Automatically Modular Leading to a Taxonomic Overview , 2002, Pattern Analysis & Applications.

[27]  Roberto Battiti,et al.  Democracy in neural nets: Voting schemes for classification , 1994, Neural Networks.

[28]  Yuehui Chen,et al.  Hierarchical Radial Basis Function Neural Networks for Classification Problems , 2006, ISNN.

[29]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[30]  Cüneyt Güzelis,et al.  Semiautomatic Transfer Function Initialization for Abdominal Visualization Using Self-Generating Hierarchical Radial Basis Function Networks , 2009, IEEE Transactions on Visualization and Computer Graphics.

[31]  L. Kuncheva ‘ Fuzzy ’ vs ‘ Non-fuzzy ’ in Combining Classifiers Designed by Boosting , 2003 .

[32]  Vincenzo Piuri,et al.  Automatic multiscale meshing through HRBF networks , 2005, IEEE Transactions on Instrumentation and Measurement.

[33]  Hisham El-Shishiny,et al.  A multistage algorithm for fast classification of patterns , 1989, Pattern Recognit. Lett..

[34]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[35]  J. Kittler,et al.  Multistage pattern recognition with reject option , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[36]  Ke Chen,et al.  Methods of Combining Multiple Classifiers with Different Features and Their Applications to Text-Independent Speaker Identification , 1997, Int. J. Pattern Recognit. Artif. Intell..

[37]  Jiangying Zhou,et al.  Discrimination of characters by a multi-stage recognition process , 1994, Pattern Recognit..

[38]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[39]  Ke Chen,et al.  On the use of different speech representations for speaker modeling , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[40]  Michael C. Fairhurst,et al.  An interactive two-level architecture for a memory network pattern classifier , 1990, Pattern Recognit. Lett..

[41]  Ching Y. Suen,et al.  Building a new generation of handwriting recognition systems , 1993, Pattern Recognit. Lett..

[42]  Ethem Alpaydin,et al.  Multiple networks for function learning , 1993, IEEE International Conference on Neural Networks.

[43]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[44]  Ying Gao Periodic Growing of Two Populations Intergrowth System , 2007 .

[45]  Jacek M. Zurada,et al.  Advances in Neural Networks - ISNN 2006, Third International Symposium on Neural Networks, Chengdu, China, May 28 - June 1, 2006, Proceedings, Part I , 2006, International Symposium on Neural Networks.

[46]  Ky Van Ha Hierarchical radial basis function networks , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[47]  Majid Ahmadi,et al.  Fusion of classifiers with fuzzy integrals , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[48]  Joydeep Ghosh,et al.  Hierarchical Fusion of Multiple Classifiers for Hyperspectral Data Analysis , 2002, Pattern Analysis & Applications.

[49]  Ching Y. Suen,et al.  A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Michael I. Jordan,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1994, Neural Computation.

[51]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[52]  Rosa Maria Valdovinos,et al.  Dynamic and Static Weighting in Classifier Fusion , 2005, IbPRIA.

[53]  Du-Ming Tsai,et al.  Automated surface inspection for statistical textures , 2003, Image Vis. Comput..

[54]  L. M. Tomas-Balibrea,et al.  Classification of marble surfaces using wavelets , 2003 .

[55]  J. Martinez-Alajarin,et al.  Supervised classification of marble textures using support vector machines , 2004 .