Assessment of Gradient Descent Trained Rule-Fact Network Expert System Multi-Path Training Technique Performance

The use of gradient descent training to optimize the performance of a rule-fact network expert system via updating the network’s rule weightings was previously demonstrated. Along with this, four training techniques were proposed: two used a single path for optimization and two use multiple paths. The performance of the single path techniques was previously evaluated under a variety of experimental conditions. The multiple path techniques, when compared, outperformed the single path ones; however, these techniques were not evaluated with different network types, training velocities or training levels. This paper considers the multi-path techniques under a similar variety of experimental conditions to the prior assessment of the single-path techniques and demonstrates their effectiveness under multiple operating conditions.

[1]  Dawn Song,et al.  Robust Physical-World Attacks on Deep Learning Models , 2017, 1707.08945.

[2]  Francis Bach,et al.  Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss , 2020, COLT.

[3]  Gerardo Rubino,et al.  Levenberg - Marquardt Training Algorithms for Random Neural Networks , 2011, Comput. J..

[4]  G. Giannakis,et al.  Federated Variance-Reduced Stochastic Gradient Descent With Robustness to Byzantine Attacks , 2019, IEEE Transactions on Signal Processing.

[5]  Christian Fieseler,et al.  Towards a deliberative framework for responsible innovation in artificial intelligence , 2021 .

[6]  Osonde Osoba,et al.  Noise can speed backpropagation learning and deep bidirectional pretraining , 2020, Neural Networks.

[7]  Kaushik Roy,et al.  Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures , 2019, Frontiers in Neuroscience.

[8]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[9]  W. Bastiaan Kleijn,et al.  The HSIC Bottleneck: Deep Learning without Back-Propagation , 2019, AAAI.

[10]  Taeweon Suh,et al.  Speculative Backpropagation for CNN Parallel Training , 2020, IEEE Access.

[11]  Faith-Michael E. Uzoka,et al.  Neuro-Fuzzy Expert System for evaluating the performance of Distributed Software System Architecture , 2013, Expert Syst. Appl..

[12]  Christian Poellabauer,et al.  Crafting Adversarial Examples For Speech Paralinguistics Applications , 2017, ArXiv.

[13]  Timothée Masquelier,et al.  STiDi-BP: Spike time displacement based error backpropagation in multilayer spiking neural networks , 2021, Neurocomputing.

[14]  Jian Li,et al.  Dimensions of artificial intelligence anxiety based on the integrated fear acquisition theory , 2020 .

[15]  Raúl Rojas,et al.  The Backpropagation Algorithm , 1996 .

[16]  J. M. McKinion,et al.  Expert systems for agriculture , 1985 .

[17]  Randall S. Sexton,et al.  Comparing backpropagation with a genetic algorithm for neural network training , 1999 .

[18]  P. Fettke,et al.  Explainable Artificial Intelligence for Process Mining: A General Overview and Application of a Novel Local Explanation Approach for Predictive Process Monitoring , 2020, Studies in Computational Intelligence.

[19]  Radhya Sahal,et al.  Explainable Artificial Intelligence (XAI) to Enhance Trust Management in Intrusion Detection Systems Using Decision Tree Model , 2021, Complex..

[20]  Byoung Chul Ko,et al.  Building Deep Random Ferns Without Backpropagation , 2020, IEEE Access.

[21]  Guang-Zhong Yang,et al.  XAI—Explainable artificial intelligence , 2019, Science Robotics.

[22]  Emily B. Fox,et al.  Adaptively Truncating Backpropagation Through Time to Control Gradient Bias , 2019, UAI.

[23]  Tony Doyle,et al.  Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy , 2017, Inf. Soc..

[24]  Sankar K. Pal,et al.  Neuro-Fuzzy Expert Systems: Relevance, Features and Methodologies , 1996 .

[25]  V. Sundarapandian,et al.  A NEURO FUZZY EXPERT SYSTEM FOR HEART DISEASE DIAGNOSIS , 2012 .

[26]  John F. Kolen,et al.  Backpropagation is Sensitive to Initial Conditions , 1990, Complex Syst..

[27]  E. Styvaktakis,et al.  Expert System for Classification and Analysis of Power System Events , 2002, IEEE Power Engineering Review.

[28]  S. Kalogirou Expert systems and GIS: an application of land suitability evaluation , 2002 .

[29]  Dongyan Zhao,et al.  Explainable AI: A Brief Survey on History, Research Areas, Approaches and Challenges , 2019, NLPCC.

[30]  W. A. Sandham,et al.  Neural network and neuro-fuzzy systems for improving diabetes therapy , 1998, Proceedings of the 20th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Vol.20 Biomedical Engineering Towards the Year 2000 and Beyond (Cat. No.98CH36286).

[31]  Jean-Michel Renders,et al.  Optimization of fuzzy expert systems using genetic algorithms and neural networks , 1995, IEEE Trans. Fuzzy Syst..

[32]  Jeremy Straub Expert System Gradient Descent Style Training: Development of a Defensible Artificial Intelligence Technique , 2021, Knowl. Based Syst..

[33]  Roberto Battiti,et al.  Accelerated Backpropagation Learning: Two Optimization Methods , 1989, Complex Syst..

[34]  J. Straub Machine learning performance validation and training using a ‘perfect’ expert system , 2021, MethodsX.

[35]  Hussein A. Abbass,et al.  Speeding Up Backpropagation Using Multiobjective Evolutionary Algorithms , 2003, Neural Computation.

[36]  Seda Sahin,et al.  Hybrid expert systems: A survey of current approaches and applications , 2012, Expert Syst. Appl..

[37]  Pu Zhao,et al.  Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent , 2020, AAAI.

[38]  Joshua Lederberg,et al.  DENDRAL: A Case Study of the First Expert System for Scientific Hypothesis Formation , 1993, Artif. Intell..

[39]  Lujo Bauer,et al.  Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition , 2016, CCS.

[40]  Samarjit Kar,et al.  Hypertension diagnosis: A comparative study using fuzzy expert system and neuro fuzzy system , 2013, 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[41]  Ankur Taly,et al.  Explainable AI in Industry , 2019, KDD.

[42]  Farhad Kolahan,et al.  Optimization of backpropagation neural network-based models in EDM process using particle swarm optimization and simulated annealing algorithms , 2020 .

[43]  Francisco Herrera,et al.  Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI , 2020, Inf. Fusion.

[44]  Jeremy Straub,et al.  An expert system for the prediction of student performance in an initial computer science course , 2017, 2017 IEEE International Conference on Electro Information Technology (EIT).

[45]  Amine Chohra,et al.  Neuro-fuzzy expert system E_S_CO_V for the obstacle avoidance behavior of intelligent autonomous vehicles , 1997, Adv. Robotics.