Improving the Performance of Cascade Correlation Neural Networks on Multimodal Functions

Intrinsic qualities of the cascade correlation algorithm make it a popular choice for many researchers wishing to utilize neural networks. Problems arise when the outputs required are highly multimodal over the input domain. The mean squared error of the approximation increases significantly as the number of modes increases. By applying ensembling and early stopping, we show that this error can be reduced by a factor of three. We also present a new technique based on subdivision that we call patchworking. When used in combination with early stopping and ensembling the mean improvement in error is over 10 in some cases.

[1]  Anju Vyas.  Print , 2003 .

[2]  Shumeet Baluja,et al.  Reducing Network Depth in the Cascade-Correlation Learning Architecture, , 1994 .

[3]  Lutz Prechelt,et al.  Investigation of the CasCor Family of Learning Algorithms , 1997, Neural Networks.

[4]  S. Waugh,et al.  Function evaluation and the cascade-correlation architecture , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[5]  Igor V. Tetko,et al.  An Enhancement of Generalization Ability in Cascade Correlation Algorithm by Avoidance of Overfitting/Overtraining Problem , 2004, Neural Processing Letters.

[6]  Ramana V. Grandhi,et al.  Improved Distributed Hypercube Sampling , 2002 .

[7]  Kishan G. Mehrotra,et al.  Elements of artificial neural networks , 1996 .

[8]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[9]  Hamid Hefazi,et al.  Constructive neural network ensemble for regression tasks in high dimensional spaces , 2007, Sixth International Conference on Machine Learning and Applications (ICMLA 2007).

[10]  Jude Shavlik,et al.  EXPERIMENTAL ANALYSIS OF ASPECTS OF THE CASCADE-CORRELATION LEARNING ARCHITECTURE , 1991 .

[11]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[12]  Sandro Ridella,et al.  On the convergence of a growing topology neural algorithm , 1996, Neurocomputing.

[13]  James T. Kwok,et al.  Objective functions for training new hidden units in constructive neural networks , 1997, IEEE Trans. Neural Networks.