A neural model for multi-expert architectures

We present a generalization of conventional artificial neural networks that allows for a functional equivalence to multi-expert systems. The new model provides an architectural freedom going beyond existing multi-expert models and an integrative formalism for comparing and combining various techniques of learning. We consider the gradient, EM, reinforcement, and unsupervised learning. Its uniform representation aims at a simple genetic encoding and evolutionary structure optimization of multi-expert systems. This paper contains a detailed description of the model and learning rules, empirically validates its functionality, and discusses future perspectives.

[1]  R. Jacobs.  Computational studies of the development of functionally specialized neural modules , 1999, Trends in Cognitive Sciences.

[2]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[3]  Marc Toussaint,et al.  On model selection and the disability of neural networks to decompose tasks , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[4]  Charles W. Anderson,et al.  Reinforcement Learning with Modular Neural Networks for Control , 1994 .

[5]  Michael I. Jordan,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1994, Neural Computation.

[6]  Henrik Gollee,et al.  Modular Neural Networks and Self-Decomposition , 1997 .

[7]  P. Frasconi,et al.  An EM Approach to Learning Sequential , 1994 .

[8]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[9]  Fuad Rahman,et al.  Serial Combination of Multiple Experts: A Unified Evaluation , 1999, Pattern Analysis & Applications.

[10]  Michael I. Jordan,et al.  Task Decomposition Through Competition in a Modular Connectionist Architecture: The What and Where Vision Tasks , 1990, Cogn. Sci..

[11]  P. Anandan,et al.  Pattern-recognizing stochastic learning automata , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[12]  Steven J. Nowlan,et al.  Mixtures of Controllers for Jump Linear and Non-Linear Plants , 1993, NIPS.

[13]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .