Modular Neural Network Classifiers: A Comparative Study

There is a wide variety of Modular Neural Network (MNN) classifiers in the literature. They differ according to the design of their architecture, task-decomposition scheme, learning procedure, and multi-module decision-making strategy. Meanwhile, there is a lack of comparative studies in the MNN literature. This paper compares ten MNN classifiers which give a good representation of design varieties, viz., Decoupled; Other-output; ART-BP; Hierarchical; Multiple-experts; Ensemble (majority vote); Ensemble (average vote); Merge-glue; Hierarchical Competitive Neural Net; and Cooperative Modular Neural Net. Two benchmark applications of different degree and nature of complexity are used for performance comparison, and the strength-points and drawbacks of the different networks are outlined. The aim is to help a potential user to choose an appropriate model according to the application in hand and the available computational resources.

[1]  Michael I. Jordan,et al.  Task Decomposition Through Competition in a Modular Connectionist Architecture: The What and Where Vision Tasks , 1990, Cogn. Sci..

[2]  Roberto Battiti,et al.  Democracy in neural nets: Voting schemes for classification , 1994, Neural Networks.

[3]  Mohamed S. Kamel,et al.  Modular neural network architectures for classification , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[4]  Alexander H. Waibel,et al.  Modular Construction of Time-Delay Neural Networks for Speech Recognition , 1989, Neural Computation.

[5]  M. Kamel,et al.  Voting schemes for cooperative neural network classifiers , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[6]  G. Bartfai,et al.  Hierarchical clustering with ART neural networks , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[7]  Mohsen Rashwan,et al.  A tree structured neural network , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[8]  H. Hackbarth,et al.  Modular connectionist structure for 100-word recognition , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[9]  Mohamed S. Kamel,et al.  Cooperative modular neural network classifiers , 1996 .

[10]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[11]  P. Gallinari,et al.  Cooperation of neural nets and task decomposition , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[12]  Ethem Alpaydin,et al.  Multiple networks for function learning , 1993, IEEE International Conference on Neural Networks.