Fusing Neural Networks Through Space Partitioning and Fuzzy Integration

To improve recognition results, decisions of multiple neural networks can be aggregated into a committee decision. Aggregation weights assigned to neural networks or groups of networks can be the same in the entire data space or can be different (data dependent) in various regions of the space. In this paper, we propose a method for obtaining data dependent aggregation weights. The proposed approach is tested in two aggregation schemes, namely aggregation through neural network selection, and aggregation by the Choquet integral with respect to the λ-fuzzy measure. The effectiveness of the approach is demonstrated on two artificial and three real data sets.

[1]  Antanas Verikas,et al.  Soft combination of neural classifiers: A comparative study , 1999, Pattern Recognit. Lett..

[2]  Kagan Tumer,et al.  Linear and order statistics combiners for reliable pattern classification , 1996 .

[3]  Michael I. Jordan,et al.  Convergence results for the EM approach to mixtures of experts architectures , 1995, Neural Networks.

[4]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[5]  Antanas Verikas,et al.  FUSING NEURAL NETWORKS THROUGH FUZZY INTEGRATION , 2001 .

[6]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[7]  Kevin W. Bowyer,et al.  Combination of Multiple Classifiers Using Local Accuracy Estimates , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Michel Grabisch,et al.  Classification by fuzzy integral: performance and tests , 1994, CVPR 1994.

[9]  Paul D. Gader,et al.  Fusion of handwritten word classifiers , 1996, Pattern Recognit. Lett..

[10]  Steve R. Waterhouse,et al.  Ensemble Methods for Phoneme Classification , 1996, NIPS.

[11]  Roberto Battiti,et al.  Democracy in neural nets: Voting schemes for classification , 1994, Neural Networks.

[12]  A. Sharkey.  Linear and Order Statistics Combiners for Pattern Classification , 1999 .

[13]  James C. Bezdek,et al.  Decision templates for multiple classifier fusion: an experimental comparison , 2001, Pattern Recognit..

[14]  P. Gader,et al.  Advances in fuzzy integration for pattern recognition , 1994, CVPR 1994.

[15]  Ching Y. Suen,et al.  A method of combining multiple classifiers-a neural network approach , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[16]  Xiuzhen Cheng,et al.  An asymptotic analysis of some expert fusion methods , 2001, Pattern Recognit. Lett..

[17]  Anders Krogh,et al.  Learning with ensembles: How overfitting can be useful , 1995, NIPS.

[18]  Antanas Verikas,et al.  Training neural networks by stochastic optimisation , 2000, Neurocomputing.

[19]  W. Peizhuang.  Pattern Recognition with Fuzzy Objective Function Algorithms (James C. Bezdek) , 1983 .

[20]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[21]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[22]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  David W. Opitz,et al.  Generating Accurate and Diverse Members of a Neural-Network Ensemble , 1995, NIPS.

[24]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Antanas Verikas,et al.  Adaptive character recognition system , 1992, Pattern Recognit. Lett..

[26]  Leo Breiman,et al.  HALF&HALF BAGGING AND HARD BOUNDARY POINTS , 1998 .

[27]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[28]  Volker Tresp,et al.  Combining Estimators Using Non-Constant Weighting Functions , 1994, NIPS.

[29]  Volker Tresp,et al.  Averaging Regularized Estimators , 1997, Neural Computation.

[30]  Sherif Hashem,et al.  Optimal Linear Combinations of Neural Networks , 1997, Neural Networks.

[31]  James C. Bezdek,et al.  Fuzzy Kohonen clustering networks , 1994, Pattern Recognit..

[32]  M. Sugeno.  FUZZY MEASURES AND FUZZY INTEGRALS—A SURVEY , 1993 .

[33]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[34]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[35]  Paul D. Gader,et al.  Improved dynamic-programming-based handwritten word recognition using optimal order statistics , 1997, Optics & Photonics.