A new probabilistic neural network model based on backpropagation algorithm

Since the traditional probabilistic neural network (PNN) cannot systematically solve the difficulty of estimating probability function and the high space complexity, this paper introduces backpropagation (BP) algorithm into the classical PNN. By designing appropriate error function and BP algorithm based on the steepest descent, an improved BP-PNN is presented, with its algorithm and effectiveness deduced. Three synthetic datasets and ten benchmark problems have been tested, compared with Probabilistic Neural Networks (PNN), Multi-Layered Perceptron (MLP) and Support Vector Machine (SVM). The results prove that (1) the accuracy of classification of BP-PNN is much higher than PNN, and it has a significant advantage compared with MLP and SVM; (2) BP-PNN has strong capacity to identify the importance of input indicators; (3) BP-PNN is a new pattern classification method to estimate the probabilistic function, reduce the space complexity and identify the importance of the indicators.

[1]  D. F. Specht,et al.  Enhancements to probabilistic neural networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[2]  Xie Yongfang.  Cost-sensitive probabilistic neural network with its application in fault diagnosis , 2010 .

[3]  Pang Su-lin.  Study on Credit Scoring Model and Forecasting Based on Probabilistic Neural Network , 2005 .

[4]  A. E. Amin,et al.  A novel classification model for cotton yarn quality based on trained neural network using genetic algorithm , 2013, Knowl. Based Syst..

[5]  D. F. Specht,et al.  Generalization accuracy of probabilistic neural networks compared with backpropagation networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[6]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[7]  Hongye Su,et al.  Forecasting building energy consumption with hybrid genetic algorithm-hierarchical adaptive network-based fuzzy inference system , 2010 .

[8]  De-Shuang Huang,et al.  Determining the centers of radial basis probabilistic neural networks by recursive orthogonal least square algorithms , 2005, Appl. Math. Comput..

[9]  Ivica Kostanic,et al.  Principles of Neurocomputing for Science and Engineering , 2000 .

[10]  Mehdi Khashei,et al.  Hybridization of the probabilistic neural networks with feed-forward neural networks for forecasting , 2012, Eng. Appl. Artif. Intell..

[11]  Sukumar Mishra,et al.  Detection and classification of power quality disturbances using S-Transform and probabilistic neural network , 2009, 2009 IEEE/PES Power Systems Conference and Exposition.

[12]  Rahman Ashena,et al.  Bottom hole pressure estimation using evolved neural networks by real coded ant colony optimization and genetic algorithm , 2011 .

[13]  Leonardo Trujillo,et al.  Hybrid back-propagation training with evolutionary strategies , 2014, Soft Comput..

[14]  Sung-Nien Yu,et al.  Electrocardiogram beat classification based on wavelet transformation and probabilistic neural network , 2007, Pattern Recognit. Lett..

[15]  Xiao De-yun.  PCA-based probability neural network structure optimization , 2008 .

[16]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[17]  Sepehr Sanaye,et al.  Thermal modeling of gas engine driven air to water heat pump systems in heating mode using genetic algorithm and Artificial Neural Network methods , 2013 .

[18]  Mo M. Jamshidi,et al.  A Modified Probabilistic Neural Network for Partial Volume Segmentation in Brain MR Image , 2007, IEEE Transactions on Neural Networks.

[19]  Tommy W. S. Chow,et al.  A weight initialization method for improving training speed in feedforward neural network , 2000, Neurocomputing.

[20]  Michael R. Berthold,et al.  Constructive training of probabilistic neural networks , 1998, Neurocomputing.

[21]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[22]  Yuping Jin,et al.  Lithology identification based on genetic optimized radial basis probabilistic neural network: Lithology identification based on genetic optimized radial basis probabilistic neural network , 2013 .

[23]  S. Ramakrishnan,et al.  Comparative study between traditional and modified probabilistic neural networks , 2009, Telecommun. Syst..

[24]  Marjorie B. Platt,et al.  Probabilistic Neural Networks in Bankruptcy Prediction , 1999 .

[25]  D. F. Specht,et al.  Probabilistic neural networks for classification, mapping, or associative memory , 1988, IEEE 1988 International Conference on Neural Networks.

[26]  Deug-Woo Lee,et al.  A combined method for improving backpropagation algorithm , 1991, Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics.

[27]  Thierry Denoeux,et al.  A k-nearest neighbor classification rule based on Dempster-Shafer theory , 1995, IEEE Trans. Syst. Man Cybern..

[28]  Hojjat Adeli,et al.  A probabilistic neural network for earthquake magnitude prediction , 2009, Neural Networks.

[29]  Li Baolin.  Lithology identification based on genetic optimized radial basis probabilistic neural network , 2013 .

[30]  Ryan P. Adams,et al.  Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks , 2015, ICML.

[31]  Christian Igel,et al.  Empirical evaluation of the improved Rprop learning algorithms , 2003, Neurocomputing.

[32]  Andrey V. Savchenko,et al.  Probabilistic neural network with homogeneity testing in recognition of discrete patterns set , 2013, Neural Networks.

[33]  Sammy Siu,et al.  Improving the Back-Propagation Algorithm Using Evolutionary Strategy , 2007, IEEE Transactions on Circuits and Systems II: Express Briefs.

[34]  Yashwant Prasad Singh,et al.  Dynamic tunneling technique for efficient training of multilayer perceptrons , 1999, IEEE Trans. Neural Networks.