Improved generalization ability using constrained neural network architectures

The function of generalization is indispensable for training artificial neural networks to robustly recognize patterns. The ability to generalize is acquired by placing constraints on the network's architecture. In order to enable an artificial network to emulate the same function of generalization as human beings, it is essential to design the network with the same architecture as that of the real biological brain and use similar learning rules to train it. The author is attempting to determine the constraints controlling biological neural networks, and to introduce them in the design of artificial neural networks. This paper offers some of the results of such trials, taking the "neocognitron" as the primary example. These constraints, however, are useful not only for neocognitron-like models but also for most artificial neural networks in general.