A New Breakpoint in Hybrid Particle Swarm-Neural Network Architecture: Individual Boundary Adjustment

Neural Network (NN) is an effective classifier, but it generally uses the Backpropagation type algorithms which are insufficient because of trapping to local minimum of error rate. For elimination of this handicap, stochastic optimization algorithms are used to update the parameters of NN. Particle Swarm Optimization (PSO) is one of these providing a robust coherence with NN. In realized studies about Hybrid PSO-NN, position and velocity boundaries of weight and bias are chosen equal or set free in space which leave the performance of PSO-NN in suspense. In this paper, the limitations of weight velocity (wv), weight position (wp), bias velocity (bv) and bias position (bp) are diversely changed and their effects on the output of hybrid structure are examined. Concerning this, the formed structure is called as Bounded PSO-NN on account of adjusting the optimum operating conditions (intervals). On performance evaluation, proposed method is tested on binary and multiclass pattern classification by using six medical datasets: Wisconsin Breast Cancer (WBC), Pima Indian Diabetes (PID), Bupa Liver Disorders (BLD), Heart Statlog (HS), Breast Tissue (BT) and Dermatology Data (DD). Upon analyzing the results, it was revealed that Bounded PSO-NN has a faster processing time than general PSO-NNs in which set-free and wpi=bpi and wvi=bvi conditions are settled. The superiority in terms of processing time is about 199s (set-free) and 307s (wpi=bpi and wvi=bvi) for training, about 16ms (set-free) and 9ms (wpi=bpi and wvi=bvi) for test. In terms of classification performance, PSO-NN (set-free condition), PSO-NN (wpi=bpi & wvi=bvi) and PSO-NN with individual boundary adjustment (bounded PSO-NN) respectively achieves to accuracy rates as 69.84%, 95.31% and 97.22% on WBC, 47.01%, 76.69% and 77.73% on PID, 55.36%, 67.54% and 73.91% on BLD, 64.82%, 81.48% and 85.56% on HS, 75%, 92.31% and 100% on BT, 27.47%, 92.31% and 100% on DD. In the light of experiments, it is seen that Bounded PSO-NN is better than general PSO-NNs for obtaining the optimum results. Consequently, the importance of limitations is clarified and it is proven that each limitation must be adjusted individually, not be set free or not be chosen equal.

[1]  Zhengxin Chen,et al.  A Descriptive Framework for the Field of Data Mining and Knowledge Discovery , 2008, Int. J. Inf. Technol. Decis. Mak..

[2]  Alireza Bahadori,et al.  Estimation of the water content of natural gas dried by solid calcium chloride dehydrator units , 2014 .

[3]  P. Luukka.  A New Nonlinear Fuzzy Robust PCA Algorithm and Similarity Classifier in Classification of Medical Data Sets , 2011 .

[4]  Paulo Cortez,et al.  Particle swarms for feedforward neural network training , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[5]  A. Rezaee Jordehi,et al.  Parameter selection in particle swarm optimisation: a survey , 2013, J. Exp. Theor. Artif. Intell..

[6]  Sohrab Zendehboudi,et al.  A new screening tool for evaluation of steamflooding performance in Naturally Fractured Carbonate Reservoirs , 2013 .

[7]  Yi Peng,et al.  Evaluation of Classification Algorithms Using MCDM and Rank Correlation , 2012, Int. J. Inf. Technol. Decis. Mak..

[8]  Rudolf Kruse,et al.  Obtaining interpretable fuzzy classification rules from medical data , 1999, Artif. Intell. Medicine.

[9]  A. Elkamel,et al.  Asphaltene precipitation and deposition in oil reservoirs –technical aspects, experimental and hybrid neural network predictive tools , 2014 .

[10]  Islamic Azad,et al.  Comparison of Particle Swarm Optimization and Backpropagation Algorithms for Training Feedforward Neural Network , 2014 .

[11]  Andrew J. Chipperfield,et al.  Simplifying Particle Swarm Optimization , 2010, Appl. Soft Comput..

[12]  C. L. Wu,et al.  Methods to improve neural network performance in daily flows prediction , 2009 .

[13]  Shanwen Zhang,et al.  Dimension Reduction Using Semi-Supervised Locally Linear Embedding for Plant Leaf Classification , 2009, ICIC.

[14]  N. B. Venkateswarlu,et al.  A Critical Study of Selected Classification Algorithms for Liver Disease Diagnosis , 2011 .

[15]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[16]  J. Ross Quinlan,et al.  Improved Use of Continuous Attributes in C4.5 , 1996, J. Artif. Intell. Res..

[17]  Chidchanok Lursinsap,et al.  Contemporary Classification on Medical Data based on Non-Linear Feature Extraction , 2009, 2009 International Conference on Computational Science and Its Applications.

[18]  M.M.B.R. Vellasco,et al.  Inverted hierarchical neuro-fuzzy BSP system: a novel neuro-fuzzy model for pattern classification and rule extraction in databases , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[19]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[20]  Yuh-Jye Lee,et al.  SSVM: A Smooth Support Vector Machine for Classification , 2001, Comput. Optim. Appl..

[21]  Ferenc Szeifert,et al.  Supervised fuzzy clustering for the identification of fuzzy classifiers , 2003, Pattern Recognit. Lett..

[22]  K. Chau,et al.  A hybrid model coupled with singular spectrum analysis for daily rainfall prediction , 2010 .

[23]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[24]  Kwok-wing Chau,et al.  Improving Forecasting Accuracy of Annual Runoff Time Series Using ARIMA Based on EEMD Decomposition , 2015, Water Resources Management.

[25]  Sun I. Kim,et al.  Nonlinear Support Vector Machine Visualization for Risk Factor Analysis Using Nomograms and Localized Radial Basis Function Kernels , 2008, IEEE Transactions on Information Technology in Biomedicine.

[26]  Kwok-wing Chau,et al.  Data-driven input variable selection for rainfall-runoff modeling using binary-coded particle swarm optimization and Extreme Learning Machines , 2015 .

[27]  Alireza Bahadori,et al.  Novel methods predict equilibrium vapor methanol content during gas hydrate inhibition , 2013 .

[28]  Siti Mariyam Hj. Shamsuddin,et al.  Bounded PSO Vmax Function in Neural Network Learning , 2008, 2008 Eighth International Conference on Intelligent Systems Design and Applications.

[29]  Andreas Stafylopatis,et al.  Data Mining based on Gene Expression Programming and Clonal Selection , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[30]  Devashish Sharma,et al.  THE CONCEPT OF SENSITIVITY AND SPECIFICITY IN RELATION TO TWO TYPES OF ERRORS AND ITS APPLICATION IN MEDICAL RESEARCH , 2009 .

[31]  Lifeng Xi,et al.  Evolving artificial neural networks using an improved PSO and DPSO , 2008, Neurocomputing.

[32]  Jun Zhang,et al.  Multilayer Ensemble Pruning via Novel Multi-sub-swarm Particle Swarm Optimization , 2009, J. Univers. Comput. Sci..

[33]  Nicos G. Pavlidis,et al.  Optimizing the Performance of Probabilistic Neural Networks in a BioinformaticsTa sk , 2004 .

[34]  Yi Peng,et al.  Evaluation of clustering algorithms for financial risk analysis using MCDM methods , 2014, Inf. Sci..

[35]  Zhang Li-ping,et al.  Optimal choice of parameters for particle swarm optimization , 2005 .

[36]  Yi Peng,et al.  FAMCDM: A fusion approach of MCDM methods to rank multiclass classification algorithms , 2011 .