Reducing noise impact on MLP training

In this paper we propose and discuss several new approaches to noise-resistant training of multilayer perceptron neural networks. Two groups of approaches: input ones, based on instance selection and outlier detection, and output ones, based on modified robust error objective functions, are presented and compared. In addition we compare them to some known methods. The experimental evaluation of the methods on classification and regression tasks and comparison of their performances for different amounts of noise in the training data, proves the effectiveness of the proposed approaches.

[1]  Marek Grochowski,et al.  Comparison of Instance Selection Algorithms II. Results and Comments , 2004, ICAISC.

[2]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[3]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[4]  Shun-Feng Su,et al.  The annealing robust backpropagation (ARBP) learning algorithm , 2000, IEEE Trans. Neural Networks Learn. Syst..

[5]  Ramesh C. Jain,et al.  A robust backpropagation learning algorithm for function approximation , 1994, IEEE Trans. Neural Networks.

[6]  Marcin Blachnik,et al.  Instance Selection in Logical Rule Extraction for Regression Problems , 2013, ICAISC.

[7]  Moumen T. El-Melegy,et al.  Robust Training of Artificial Feedforward Neural Networks , 2009, Foundations of Computational Intelligence.

[8]  Moumen T. El-Melegy,et al.  Random sampler M-estimator algorithm for robust function approximation via feed-forward neural networks , 2011, The 2011 International Joint Conference on Neural Networks.

[9]  Andrzej Rusiecki.  Robust LTS Backpropagation Learning Algorithm , 2007, IWANN.

[10]  Moumen T. El-Melegy,et al.  Random Sampler M-Estimator Algorithm With Sequential Probability Ratio Test for Robust Function Approximation Via Feed-Forward Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[11]  P. Rousseeuw.  Least Median of Squares Regression , 1984 .

[12]  Dennis L. Wilson,et al.  Asymptotic Properties of Nearest Neighbor Rules Using Edited Data , 1972, IEEE Trans. Syst. Man Cybern..

[13]  Ana González-Marcos,et al.  TAO-robust backpropagation learning algorithm , 2005, Neural Networks.

[14]  J. Tolvi,et al.  Genetic algorithms for outlier detection and variable selection in linear regression models , 2004, Soft Comput..

[15]  Andrzej Rusiecki,et al.  Robust learning algorithm based on LTA estimator , 2013, Neurocomputing.

[16]  Ginés Rubio,et al.  Applying Mutual Information for Prototype or Instance Selection in Regression Problems , 2009, ESANN.

[17]  Andrzej Rusiecki,et al.  Improving MLP Neural Network Performance by Noise Reduction , 2013, TPNC.

[18]  Moumen T. El-Melegy,et al.  RANSAC algorithm with sequential probability ratio test for robust training of feed-forward neural networks , 2011, The 2011 International Joint Conference on Neural Networks.

[19]  Marcin Blachnik,et al.  Bagging of Instance Selection Algorithms , 2014, ICAISC.

[20]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[21]  Jianping Zhang,et al.  Intelligent Selection of Instances for Prediction Functions in Lazy Learning Algorithms , 1997, Artificial Intelligence Review.

[22]  Kadir Liano,et al.  Robust error measure for supervised neural network learning with outliers , 1996, IEEE Trans. Neural Networks.

[23]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[24]  Wlodzislaw Duch,et al.  Variable step search algorithm for feedforward networks , 2008, Neurocomputing.

[25]  Francisco Herrera,et al.  Prototype Selection for Nearest Neighbor Classification: Taxonomy and Empirical Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Andrzej Rusiecki.  Robust Learning Algorithm Based on Iterative Least Median of Squares , 2012, Neural Processing Letters.