Evaluation of PNN pattern-layer activation function approximations in different training setups

The processing of inputs in the first two layers of the probabilistic neural network (PNN) is highly parallel which makes it quite appropriate for hardware implementations with FPGA. One of the main inconveniences however remains the implementation of the nonlinear activation function of the pattern layer neurons. In the present study, we investigate the applicability of three approximations of the exponential activation function with look-up tables of different precision and the effect this has on the training process and the classification accuracy. Furthermore, seeking for a highly-parallel hardware-friendly algorithm for the automated adjustment of the spread factor $$\sigma _i$$, we investigated the performance of fifteen PNN training setups, which are based on the differential evolution (DE) or unified particle swarm optimization (UPSO) methods. The experimental evaluation was performed following a common experimental protocol, which makes use of the Parkinson Speech Dataset, as this research aims to support the development of portable medical devices that are capable to detect episodes with exacerbation in patients with Parkinson’s disease. The performance of the most successful setups is discussed in terms of error rates and from the perspective of the resources required for an FPGA-based implementation.

[1]  Michael N. Vrahatis,et al.  Particle Swarm Optimization and Intelligence: Advances and Applications , 2010 .

[2]  Maciej Kusy,et al.  Application of Reinforcement Learning Algorithms for the Adaptive Computation of the Smoothing Parameter for Probabilistic Neural Network , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Alex Pappachen James,et al.  Probabilistic Neural Network with Memristive Crossbar Circuits , 2019, 2019 IEEE International Symposium on Circuits and Systems (ISCAS).

[4]  Fikret S. Gürgen,et al.  Collection and Analysis of a Parkinson Speech Dataset With Multiple Types of Sound Recordings , 2013, IEEE Journal of Biomedical and Health Informatics.

[5]  Ching-Han Chen,et al.  Adaptive image interpolation using probabilistic neural network , 2009, Expert Syst. Appl..

[6]  Ihsan Ullah,et al.  An Automated System for Epilepsy Detection using EEG Brain Signals based on Deep Learning Approach , 2018, Expert Syst. Appl..

[7]  Alex Pappachen James,et al.  Approximate Probabilistic Neural Networks with Gated Threshold Logic , 2018, 2018 IEEE 18th International Conference on Nanotechnology (IEEE-NANO).

[8]  Xiongxiong He,et al.  Auxiliary Diagnosis of Breast Tumor Based on PNN Classifier Optimized by PCA and PSO Algorithm , 2017, 2017 9th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC).

[9]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[10]  Jürgen Becker,et al.  Comparison of processing performance and architectural efficiency metrics for FPGAs and GPUs in 3D Ultrasound Computer Tomography , 2012, 2012 International Conference on Reconfigurable Computing and FPGAs.

[11]  Mihir Narayan Mohanty,et al.  Recognition of Human Speech Emotion Using Variants of Mel-Frequency Cepstral Coefficients , 2018 .

[12]  Ronald Marsh,et al.  Conjugate gradient and approximate Newton methods for an optimal probabilistic neural network for food color classification , 1998 .

[13]  Kizheppatt Vipin,et al.  FAPNN: An FPGA based Approximate Probabilistic Neural Network Library , 2018, 2018 International Conference on Computing and Network Communications (CoCoNet).

[14]  Jeffrey S. Vetter,et al.  A Survey of Methods for Analyzing and Improving GPU Energy Efficiency , 2014, ACM Comput. Surv..

[15]  Majid Sarrafzadeh,et al.  Remote health monitoring: Predicting outcome success based on contextual features for cardiovascular disease , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[16]  Michael N. Vrahatis,et al.  Recent approaches to global optimization problems through Particle Swarm Optimization , 2002, Natural Computing.

[17]  P. R. Anurenjan,et al.  Early Diagnosis of Alzheimer's Disease from MRI Images Using PNN , 2018, 2018 International CET Conference on Control, Communication, and Computing (IC4).

[18]  Yu Cao,et al.  PNN for EEG-based Emotion Recognition , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[19]  Fan Zhou,et al.  Field-programmable gate array implementation of a probabilistic neural network for motor cortical decoding in rats , 2010, Journal of Neuroscience Methods.

[20]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[21]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[22]  R. Dellavalle,et al.  Mobile medical and health apps: state of the art, concerns, regulatory control and certification , 2014, Online journal of public health informatics.

[23]  Louise E. Moser,et al.  Personal Health Monitoring Using a Smartphone , 2015, 2015 IEEE International Conference on Mobile Services.

[24]  Narrendar RaviChandran,et al.  Feature-driven machine learning to improve early diagnosis of Parkinson's disease , 2018, Expert Syst. Appl..

[25]  Vassilis P. Plagianakos,et al.  Parallel evolutionary training algorithms for “hardware-friendly” neural networks , 2002, Natural Computing.

[26]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[27]  Michael N. Vrahatis,et al.  Parameter selection and adaptation in Unified Particle Swarm Optimization , 2007, Math. Comput. Model..

[28]  R. Lavanyadevi,et al.  Brain tumor classification and segmentation in MRI images using PNN , 2017, 2017 IEEE International Conference on Electrical, Instrumentation and Communication Engineering (ICEICE).

[29]  David A. Copland,et al.  Mobile computing technology and aphasia: An integrated review of accessibility and potential uses , 2013 .

[30]  S Vinitha Sree,et al.  Evolutionary algorithm-based classifier parameter tuning for automatic ovarian cancer tissue characterization and classification. , 2014, Ultraschall in der Medizin.

[31]  Wee Ser,et al.  Probabilistic neural-network structure determination for pattern classification , 2000, IEEE Trans. Neural Networks Learn. Syst..

[32]  Todor Ganchev,et al.  Enhanced Training for the Locally Recurrent Probabilistic Neural Networks , 2009, Int. J. Artif. Intell. Tools.

[33]  John J Heine,et al.  Statistical learning methods as a preprocessing step for survival analysis: evaluation of concept using lung cancer data , 2011, Biomedical engineering online.

[34]  Huseyin Seker,et al.  Highly Parameterized K-means Clustering on FPGAs: Comparative Results with GPPs and GPUs , 2011, 2011 International Conference on Reconfigurable Computing and FPGAs.

[35]  Nicos G. Pavlidis,et al.  New Self-adaptive Probabilistic Neural Networks in Bioinformatic and Medical Tasks , 2006, Int. J. Artif. Intell. Tools.

[36]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[37]  Biswanath Samanta,et al.  Artificial neural networks and genetic algorithm for bearing fault detection , 2006, Soft Comput..

[38]  Michael N. Vrahatis,et al.  Optimal power allocation and joint source-channel coding for wireless DS-CDMA visual sensor networks using the Nash Bargaining Solution , 2005, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  Ken Tough.  Optimizing the Performance , 1999 .

[40]  Dong Wang,et al.  FPGA implementation of hardware processing modules as coprocessors in brain-machine interfaces , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[41]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[42]  Ching-Han Chen,et al.  VLSI implementation of anisotropic probabilistic neural network for real-time image scaling , 2018, Journal of Real-Time Image Processing.

[43]  Elias Oliveira,et al.  Particle Swarm Optimization Applied to Parameters Learning of Probabilistic Neural Networks for Classification of Economic Activities , 2009 .

[44]  De-Shuang Huang,et al.  A Constructive Hybrid Structure Optimization Methodology for Radial Basis Probabilistic Neural Networks , 2008, IEEE Transactions on Neural Networks.

[45]  Michael N. Vrahatis,et al.  Novel Approaches to Probabilistic Neural Networks Through Bagging and Evolutionary Estimating of Prior Probabilities , 2008, Neural Processing Letters.

[46]  Walker H. Land,et al.  A Multi-class Probabilistic Neural Network for Pathogen Classification , 2013, Complex Adaptive Systems.

[47]  Maya Gokhale,et al.  Accelerating a Random Forest Classifier: Multi-Core, GP-GPU, or FPGA? , 2012, 2012 IEEE 20th International Symposium on Field-Programmable Custom Computing Machines.

[48]  Samuel Williams,et al.  [Personal health]. , 1969, Kango kyoshitsu. [Nursing classroom].

[49]  LunBo Li,et al.  Optimizing the Performance of Probabilistic Neural Networks Using PSO in the Task of Traffic Sign Recognition , 2008, ICIC.