Generalisation Performance vs. Architecture Variations in Constructive Cascade Networks

Constructive cascade algorithms are powerful methods for training feedforward neural networks with automation of the task of specifying the size and topology of network to use. A series of empirical studies were performed to examine the effect of imposing constraints on constructive cascade neural network architectures. Building a priori knowledge of the task into the network gives better generalisation performance. We introduce our Local Feature Constructive Cascade (LoCC) and Symmetry Local Feature Constructive Cascade (SymLoCC) algorithms, and show them to have good generalisation and network construction properties on face recognition tasks.

[1]  James T. Kwok,et al.  Constructive algorithms for structure learning in feedforward neural networks for regression problems , 1997, IEEE Trans. Neural Networks.

[2]  M. A. Grudin,et al.  On internal representations in face recognition systems , 2000, Pattern Recognit..

[3]  Yann LeCun,et al.  Generalization and network design strategies , 1989 .

[4]  Tom Gedeon,et al.  Exploring architecture variations in constructive cascade networks , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[5]  Bernard Widrow,et al.  Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[6]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[8]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[9]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[10]  Tamás D. Gedeon,et al.  Increased generalization through selective decay in a constructive cascade network , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).