Multi-Net Systems

This chapter provides an introduction to the main methods and issues in the combination of Artificial Neural Nets. A distinction is made between ensemble and modular modes of combination, and the two are then treated separately. The reasons for ensemble combination are considered, and an account is provided of the main methods for creating and combining ANNs in ensembles. This account is accompanied by a discussion of the relative effectiveness of these methods, in which the concepts of diversity and selection are explained. The review of modular combination outlines the main methods of creating and combining modules, depending on whether the relationship between the modules is co-operative, competitive, sequential or supervisory. An overivew of the chapters in the book forms the conclusion section.

[1]  J. von Neumann,et al.  Probabilistic Logic and the Synthesis of Reliable Organisms from Unreliable Components , 1956 .

[2]  J. M. Bates,et al.  The Combination of Forecasts , 1969 .

[3]  J. Fodor.  The Modularity of mind. An essay on faculty psychology , 1986 .

[4]  Algirdas Avizienis,et al.  Fault Tolerance by Design Diversity: Concepts and Experiments , 1984, Computer.

[5]  J. Fodor,et al.  The Modularity of Mind: An Essay on Faculty Psychology , 1984 .

[6]  Dave E. Eckhardt,et al.  A Theoretical Basis for the Analysis of Multiversion Software Subject to Coincident Errors , 1985, IEEE Transactions on Software Engineering.

[7]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[8]  B. Grofman.  Information Pooling and Group Decision Making , 1986 .

[9]  Christian Genest,et al.  Combining Probability Distributions: A Critique and an Annotated Bibliography , 1986 .

[10]  Lawrence D. Jackel,et al.  Large Automatic Learning, Rule Extraction, and Generalization , 1987, Complex Syst..

[11]  Kiyohiro Shikano,et al.  Modularity and scaling in large phonemic neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[12]  R. Clemen,et al.  Collinearity and the use of latent root regression for combining GNP forecasts , 1989 .

[13]  Bev Littlewood,et al.  Conceptual Modeling of Coincident Failures in Multiversion Software , 1989, IEEE Trans. Software Eng..

[14]  John F. Kolen,et al.  Backpropagation is Sensitive to Initial Conditions , 1990, Complex Syst..

[15]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  David J. Chalmers,et al.  Syntactic Transformations on Distributed Representations , 1990 .

[17]  Jack Mostow,et al.  Direct Transfer of Learned Information Among Neural Networks , 1991, AAAI.

[18]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[19]  Tomas Hrycej,et al.  Modular Learning in Neural Networks: A Modularized Approach to Neural Network Classification , 1992 .

[20]  L. Cooper,et al.  When Networks Disagree: Ensemble Methods for Hybrid Neural Networks , 1992 .

[21]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[22]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[23]  Alexander H. Waibel,et al.  The Meta-Pi Network: Building Distributed Knowledge Representations for Robust Multisource Pattern Recognition , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Mongi A. Abidi,et al.  Data fusion in robotics and machine intelligence , 1992 .

[25]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[26]  William G. Baxt,et al.  Improving the Accuracy of an Artificial Neural Network Using Multiple Differently Trained Networks , 1992, Neural Computation.

[27]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[28]  Harris Drucker,et al.  Boosting and Other Ensemble Methods , 1994, Neural Computation.

[29]  Michael I. Jordan,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1994, Neural Computation.

[30]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[31]  Galina L. Rogova,et al.  Combining the results of several neural network classifiers , 1994, Neural Networks.

[32]  Kumpati S. Narendra,et al.  Adaptation and learning using multiple models, switching, and tuning , 1995 .

[33]  Noel E. Sharkey,et al.  Neural Nets and Diversity , 1995, SAFECOMP.

[34]  Bhagavatula Vijaya Kumar,et al.  Learning ranks with neural networks , 1995, SPIE Defense + Commercial Sensing.

[35]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[36]  Noel E. Sharkey,et al.  An Analysis of Catastrophic Interference , 1995, Connect. Sci..

[37]  Kishan G. Mehrotra,et al.  Efficient classification for multiclass problems using modular neural networks , 1995, IEEE Trans. Neural Networks.

[38]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[39]  Robert A. Jacobs,et al.  Methods For Combining Experts' Probability Assessments , 1995, Neural Computation.

[40]  Eric B. Bartlett,et al.  Error Estimation by Series Association for Neural Network Systems , 1995, Neural Computation.

[41]  Anders Krogh,et al.  Learning with ensembles: How overfitting can be useful , 1995, NIPS.

[42]  Nathan Intrator,et al.  Bootstrapping with Noise: An Effective Regularization Technique , 1996, Connect. Sci..

[43]  Robin R. Murphy,et al.  Biological and cognitive foundations of intelligent sensor fusion , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[44]  Sherif Hashem.  Effects of Collinearity on Combining Neural Networks , 1996, Connect. Sci..

[45]  Amanda J. C. Sharkey,et al.  On Combining Artificial Neural Nets , 1996, Connect. Sci..

[46]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[47]  Ray L. Somorjai,et al.  Application of Several Methods of Classification Fusion to Magnetic Resonance Spectra , 1996, Connect. Sci..

[48]  Bruce E. Rosen,et al.  Ensemble Learning Using Decorrelated Neural Networks , 1996, Connect. Sci..

[49]  David W. Opitz,et al.  Actively Searching for an E(cid:11)ective Neural-Network Ensemble , 1996 .

[50]  R. Tibshirani,et al.  Combining Estimates in Regression and Classification , 1996 .

[51]  Kagan Tumer,et al.  Error Correlation and Error Reduction in Ensemble Classifiers , 1996, Connect. Sci..

[52]  Tony J. Prescott,et al.  Spatial Representation for Navigation in Animats , 1996, Adapt. Behav..

[53]  Paul W. Munro,et al.  Reducing Variance of Committee Prediction with Resampling Techniques , 1996, Connect. Sci..

[54]  Colin McCormack.  Adaptation of Learning Rule Parameters Using a Meta Neural Network , 1997, Connect. Sci..

[55]  Noel E. Sharkey,et al.  Combining diverse neural nets , 1997, The Knowledge Engineering Review.

[56]  Roderick Murray-Smith,et al.  Multiple Model Approaches to Modelling and Control , 1997 .

[57]  Sherif Hashem,et al.  Optimal Linear Combinations of Neural Networks , 1997, Neural Networks.

[58]  Mark A. Musen,et al.  Modular Neural Networks for Medical Prognosis: Quantifying the Benefits of Combining Neural Networks for Survival Prediction , 1997, Connect. Sci..

[59]  Thierry Catfolis,et al.  Hybridization and Specialization of Real-time Recurrent Learning-based Neural Networks , 1997, Connect. Sci..

[60]  Noel E. Sharkey,et al.  Artificial neural networks for coordination and control: The portability of experiential representations , 1997, Robotics Auton. Syst..

[61]  Noel E. Sharkey,et al.  Adapting an Ensemble Approach for the Diagnosis of Breast Cancer , 1998 .

[62]  P. Gallinari,et al.  Modular neural net systems, training of , 1998 .

[63]  Masami Ito,et al.  Task decomposition and module combination based on class relations: a modular neural network for pattern classification , 1999, IEEE Trans. Neural Networks.