A statistical procedure for determining the optimal number of hidden neurons of a neural model

This paper proposes a novel model selection procedure for neural networks based on least squares estimation and statistical tests. The procedure is performed systematically and automatically in two phases. In the first (bottom-up) phase, the parameters of candidate neural models with an increasing number of hidden neurons are estimated until they cannot be approved anymore, i.e. until the neural models become ill-conditioned. In the second (top-down) phase, a selection among approved candidate models using statistical Fisher tests is performed; the series of tests starts from an appropriate full model chosen with the help of computationally inexpensive estimates of the performance of the candidates, and ends with the smallest candidate whose hidden neurons all have a statistically significant contribution to the estimation of the regression. Large scale simulation experiments illustrate the efficiency and the parsimony of the proposed procedure, and allow a comparison to other approaches.