Evolutionary Optimization for Neuromorphic Systems

Designing and training an appropriate spiking neural network for neuromorphic deployment remains an open challenge in neuromorphic computing. In 2016, we introduced an approach for utilizing evolutionary optimization to address this challenge called Evolutionary Optimization for Neuromorphic Systems (EONS). In this work, we present an improvement to this approach that enables rapid prototyping of new applications of spiking neural networks in neuromorphic systems. We discuss the overall EONS framework and its improvements over the previous implementation. We present several case studies of how EONS can be used, including to train spiking neural networks for classification and control tasks, to train under hardware constraints, to evolve a reservoir for a liquid state machine, and to evolve smaller networks using multi-objective optimization.

[1]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[2]  Catherine D. Schuman,et al.  The TENNLab Suite of LIDAR-Based Control Applications for Recurrent , Spiking , Neuromorphic Systems , 2019 .

[3]  Garrick Orchard,et al.  SLAYER: Spike Layer Error Reassignment in Time , 2018, NeurIPS.

[4]  Andrew S. Cassidy,et al.  Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware , 2016, 2016 IEEE International Conference on Rebooting Computing (ICRC).

[5]  Catherine D. Schuman,et al.  An evolutionary optimization framework for neural networks and neuromorphic architectures , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[6]  Catherine D. Schuman,et al.  Biomimetic, Soft-Material Synapse for Neuromorphic Computing: from Device to Network , 2018, 2018 IEEE 13th Dallas Circuits and Systems Conference (DCAS).

[7]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[8]  Catherine D. Schuman,et al.  Caspian: A Neuromorphic Development Platform , 2020, NICE.

[9]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[10]  Franck Mamalet,et al.  Unsupervised Feature Learning With Winner-Takes-All Based STDP , 2018, Front. Comput. Neurosci..

[11]  Craig M. Vineyard,et al.  Training deep neural networks for binary communication with the Whetstone method , 2019 .

[12]  Cory Merkel,et al.  Design and Analysis of a Neuromemristive Reservoir Computing Architecture for Biosignal Processing , 2016, Front. Neurosci..

[13]  Priya Shah,et al.  A Neuromorphic Sparse Coding Defense to Adversarial Images , 2019, ICONS.

[14]  Seung Hwan Lee,et al.  Reservoir computing using dynamic memristors for temporal information processing , 2017, Nature Communications.

[15]  Catherine D. Schuman,et al.  Intelligent Reservoir Generation for Liquid State Machines using Evolutionary Optimization , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[16]  Muhaini Othman,et al.  Evolving spiking neural networks for personalised modelling, classification and prediction of spatio-temporal patterns with a case study on stroke , 2014, Neurocomputing.

[17]  Catherine D. Schuman,et al.  Neuromorphic computing for temporal scientific data classification , 2017, NCS.

[18]  Catherine D. Schuman,et al.  The TENNLab Exploratory Neuromorphic Computing Framework , 2018, IEEE Letters of the Computer Society.

[19]  Risto Miikkulainen,et al.  Accelerated Neural Evolution through Cooperatively Coevolved Synapses , 2008, J. Mach. Learn. Res..

[20]  Catherine D. Schuman,et al.  Parallel Evolutionary Optimization for Neuromorphic Network Training , 2016, 2016 2nd Workshop on Machine Learning in HPC Environments (MLHPC).

[21]  Catherine D. Schuman,et al.  Multi-Objective Optimization for Size and Resilience of Spiking Neural Networks , 2019, 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON).

[22]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[23]  James S. Plank,et al.  Four Simulators of the DANNA Neuromorphic Computing Architecture , 2018, Proceedings of the International Conference on Neuromorphic Systems.

[24]  Liam McDaid,et al.  Hardware spiking neural network prototyping and application , 2011, Genetic Programming and Evolvable Machines.

[25]  Catherine D. Schuman,et al.  DANNA: A neuromorphic software ecosystem ☆ , 2016, BICA 2016.

[26]  Antoine Dupret,et al.  Event-Based, Timescale Invariant Unsupervised Online Deep Learning With STDP , 2018, Front. Comput. Neurosci..

[27]  Catherine D. Schuman,et al.  Understanding Selection And Diversity For Evolution Of Spiking Recurrent Neural Networks , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[28]  Steven R. Young,et al.  Optimizing deep learning hyper-parameters through an evolutionary algorithm , 2015, MLHPC@SC.

[29]  Elliot Meyerson,et al.  Evolving Deep Neural Networks , 2017, Artificial Intelligence in the Age of Neural Networks and Brain Computing.

[30]  Tobi Delbrück,et al.  Training Deep Spiking Neural Networks Using Backpropagation , 2016, Front. Neurosci..

[31]  Daniel Rasmussen,et al.  NengoDL: Combining Deep Learning and Neuromorphic Modelling Methods , 2018, Neuroinformatics.

[32]  Benjamin Schrauwen,et al.  An overview of reservoir computing: theory, applications and implementations , 2007, ESANN.

[33]  Catherine D. Schuman,et al.  Energy and Area Efficiency in Neuromorphic Computing for Resource Constrained Devices , 2018, ACM Great Lakes Symposium on VLSI.

[34]  Dharmendra S. Modha,et al.  Backpropagation for Energy-Efficient Neuromorphic Computing , 2015, NIPS.

[35]  Sander M. Bohte,et al.  SpikeProp: backpropagation for networks of spiking neurons , 2000, ESANN.

[36]  Nicholas Soures,et al.  Digital neuromorphic design of a Liquid State Machine for real-time processing , 2016, 2016 IEEE International Conference on Rebooting Computing (ICRC).

[37]  Catherine D. Schuman,et al.  Design of Superconducting Optoelectronic Networks for Neuromorphic Computing , 2018, 2018 IEEE International Conference on Rebooting Computing (ICRC).