Robust training of multilayer perceptrons: some experimental results

Robust regression methods are a useful alternative to least squares for modelling real data sets, which generally contain outliers. Various authors have previously examined the use of these methods for modelling such data using multilayer perceptrons (MLPs), with impressive results. This paper describes some experimental experiences with the use of some simple robust methods for MLP training. The use of robust error measures for testing is demonstrated, and the use of more than one robust error for testing MLPs during the training process is recommended. The failure of one simple robust training method due to leverage points is demonstrated, and preliminary results from a method which may assist in the identification of these points are provided. Finally, the problem of overfitting when using robust methods is discussed briefly.