A comparative study of optimization methods for improving artificial neural network performance

This paper proposes a comparative study of commonly-used global optimization methods to improve training performance of back-propagation neural networks. The optimization methods adopted herein include Simulated annealing, Direct search, and Genetic algorithm. These methods are used to optimize neural networks' weights and biases before using back-propagation algorithm in order to prevent the networks from local minima. Four benchmark datasets of prediction (regression) task were used to evaluate the established models. The experimental results indicated that optimizing neural network's parameters is a complicated problem due to its high dimension of variables to be optimized. And only genetic algorithm was able to solve this difficult optimization problem. In addition, this paper also applied this success method to predict monthly rainfall time series data in the northeast region of Thailand. The results indicated that using of genetic algorithm with back-propagation neural network is a recommended combination.

[1]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[2]  Kevin Kok Wai Wong,et al.  Cluster validation methods for localization of spatial rainfall data in the northeast region of Thailand , 2013, 2013 International Conference on Machine Learning and Cybernetics.

[3]  I-Cheng Yeh,et al.  Analysis of Strength of Concrete Using Design of Experiments and Neural Networks , 2006 .

[4]  Thomas F. Brooks,et al.  Airfoil self-noise and prediction , 1989 .

[5]  Lance Chun Che Fung,et al.  Porosity Prediction Using Bagging of Complementary Neural Networks , 2009, ISNN.

[6]  P. Cortez,et al.  A data mining approach to predict forest fires using meteorological data , 2007 .

[7]  Jay Cheng,et al.  A New Technique for Searching the Global Minimum of Supervised Neural Network , 2010, 2010 Fourth International Conference on Genetic and Evolutionary Computing.

[8]  Athanasios Tsanas,et al.  Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools , 2012 .

[9]  Michael Negnevitsky,et al.  Artificial Intelligence: A Guide to Intelligent Systems , 2001 .

[10]  Lance Chun Che Fung,et al.  A modular technique for monthly rainfall time series prediction , 2013, 2013 IEEE Symposium on Computational Intelligence in Dynamic and Uncertain Environments (CIDUE).

[11]  Lance Chun Che Fung,et al.  An integrated intelligent technique for monthly rainfall time series prediction , 2014, 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[12]  Lester Ingber,et al.  Adaptive simulated annealing (ASA): Lessons learned , 2000, ArXiv.