A Constructive Cascade Network with Adaptive Regularisation

Determining the optimum amount of regularisation to obtain the best generalisation performance in feedforward neural networks is a difficult problem. This problem is addressed in the casper algorithm, a constructive cascade algorithm that uses regularisation. Previously the amount of regularisation used by this algorithm was set by a parameter prior to training. This work explores the use of an adaptive method to automatically set the amount of regularisation as the network is constructed. This technique is compared against the original method of user optimised regularisation settings and is shown to maintain, and some-times improve the generalisation results, while also constructing smaller networks. Further benchmarking on the Proben1 series of data sets is performed and the results compared to an optimised Cascade Correlation algorithm.