A Swarm Optimization Solver Based on Ferroelectric Spiking Neural Networks

As computational models inspired by the biological neural system, spiking neural networks (SNN) continue to demonstrate great potential in the landscape of artificial intelligence, particularly in tasks such as recognition, inference, and learning. While SNN focuses on achieving high-level intelligence of individual creatures, Swarm Intelligence (SI) is another type of bio-inspired models that mimic the collective intelligence of biological swarms, i.e., bird flocks, fish school and ant colonies. SI algorithms provide efficient and practical solutions to many difficult optimization problems through multi-agent metaheuristic search. Bridging these two distinct subfields of artificial intelligence has the potential to harness collective behavior and learning ability of biological systems. In this work, we explore the feasibility of connecting these two models by implementing a generalized SI model on SNN. In the proposed computing paradigm, we use SNNs to represent agents in the swarm and encode problem solutions with the spike firing rate and with spike timing. The coupled neurons communicate and modulate each other's action potentials through event-driven spikes and synchronize their dynamics around the states of optimal solutions. We demonstrate that such an SI-SNN model is capable of efficiently solving optimization problems, such as parameter optimization of continuous functions and a ubiquitous combinatorial optimization problem, namely, the traveling salesman problem with near-optimal solutions. Furthermore, we demonstrate an efficient implementation of such neural dynamics on an emerging hardware platform, namely ferroelectric field-effect transistor (FeFET) based spiking neurons. Such an emerging in-silico neuron is composed of a compact 1T-1FeFET structure with both excitatory and inhibitory inputs. We show that the designed neuromorphic system can serve as an optimization solver with high-performance and high energy-efficiency.

[1]  M. Luisier,et al.  Physical modeling of ferroelectric field-effect transistors in the negative capacitance regime , 2016, 2016 International Conference on Simulation of Semiconductor Processes and Devices (SISPAD).

[2]  Murray Shanahan,et al.  Training a spiking neural network to control a 4-DoF robotic arm based on Spike Timing-Dependent Plasticity , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[3]  Saibal Mukhopadhyay,et al.  ReRAM Crossbar based Recurrent Neural Network for human activity detection , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[4]  Charles D. Michener,et al.  Comparative Social Behavior of Bees , 1969 .

[5]  Andrew D. Back,et al.  A spiking neural network architecture for nonlinear function approximation , 2001, Neural Networks.

[6]  Donald M. Chiarulli,et al.  Modeling oscillator arrays for video analytic applications , 2014, 2014 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[7]  Ian O'Connor,et al.  Computing with ferroelectric FETs: Devices, models, systems, and applications , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[8]  Michael Rosenblum,et al.  Neural Synchronization from the Perspective of Non-linear Dynamics , 2017, Front. Comput. Neurosci..

[9]  Andrew S. Cassidy,et al.  Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware , 2016, 2016 IEEE International Conference on Rebooting Computing (ICRC).

[10]  Piotr Dudek,et al.  Compact silicon neuron circuit with spiking and bursting behaviour , 2008, Neural Networks.

[11]  S. Datta,et al.  Physics-Based Circuit-Compatible SPICE Model for Ferroelectric Transistors , 2016, IEEE Electron Device Letters.

[12]  Yunus Babacan,et al.  A spiking and bursting neuron circuit based on memristor , 2016, Neurocomputing.

[13]  Carlos D. Brody,et al.  Simple Networks for Spike-Timing-Based Computation, with Application to Olfactory Processing , 2003, Neuron.

[14]  Giacomo Indiveri,et al.  Integration of nanoscale memristor synapses in neuromorphic computing architectures , 2013, Nanotechnology.

[15]  Zheng Wang,et al.  Ferroelectric Oscillators and Their Coupled Networks , 2017, IEEE Electron Device Letters.

[16]  Filip Ponulak,et al.  Introduction to spiking neural networks: Information processing, learning and applications. , 2011, Acta neurobiologiae experimentalis.

[17]  Suman Datta,et al.  Vertex coloring of graphs via phase dynamics of coupled oscillatory networks , 2016, Scientific Reports.

[18]  Xiaodong Li,et al.  Swarm Intelligence in Optimization , 2008, Swarm Intelligence.

[19]  Daniele Ielmini,et al.  Brain-inspired computing with resistive switching memory (RRAM): Devices, synapses and neural networks , 2018 .

[20]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[21]  Hartmut Pohlheim,et al.  Genetic and evolutionary algorithm toolbox for use with matlab , 1994 .

[22]  Louis B. Rosenberg,et al.  Crowds vs swarms, a comparison of intelligence , 2016, 2016 Swarm/Human Blended Intelligence Workshop (SHBI).

[23]  Haibin Duan,et al.  New progresses in swarm intelligence-based computation , 2015, Int. J. Bio Inspired Comput..

[24]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[25]  Yan Fang,et al.  Achieving Swarm Intelligence with Spiking Neural Oscillators , 2017, 2017 IEEE International Conference on Rebooting Computing (ICRC).

[26]  Nikola K. Kasabov,et al.  NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data , 2014, Neural Networks.

[27]  G. Di Caro,et al.  Ant colony optimization: a new meta-heuristic , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[28]  Giacomo Indiveri,et al.  A low-power adaptive integrate-and-fire neuron circuit , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[29]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[30]  Xin-She Yang,et al.  A New Metaheuristic Bat-Inspired Algorithm , 2010, NICSO.

[31]  Stefan Habenschuss,et al.  Solving Constraint Satisfaction Problems with Networks of Spiking Neurons , 2016, Front. Neurosci..

[32]  Giacomo Indiveri,et al.  An event-based architecture for solving constraint satisfaction problems , 2015, Nature Communications.

[33]  Janez Brest,et al.  A comprehensive review of firefly algorithms , 2013, Swarm Evol. Comput..

[34]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[35]  Adel M. Alimi,et al.  ACO-PSO Optimization for Solving TSP Problem with GPU Acceleration , 2016, ISDA.

[36]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[37]  Giacomo Indiveri,et al.  Frontiers in Neuromorphic Engineering , 2011, Front. Neurosci..

[38]  Marjan Mernik,et al.  Analysis of exploration and exploitation in evolutionary algorithms by ancestry trees , 2011 .

[39]  Kenneth S. Norris,et al.  Cooperative societies in three- dimensional space: On the origins of aggregations, flocks, and schools, with special reference to dolphins and fish , 1988 .

[40]  Shimeng Yu,et al.  Synaptic electronics: materials, devices and applications , 2013, Nanotechnology.

[41]  Jacques-Olivier Klein,et al.  Spin-Transfer Torque Magnetic Memory as a Stochastic Memristive Synapse for Neuromorphic Systems , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[42]  Zoran Krivokapic,et al.  Experimental Demonstration of Ferroelectric Spiking Neurons for Unsupervised Clustering , 2018, 2018 IEEE International Electron Devices Meeting (IEDM).

[43]  Steve B. Furber,et al.  Using Stochastic Spiking Neural Networks on SpiNNaker to Solve Constraint Satisfaction Problems , 2017, Front. Neurosci..

[44]  Suman Datta,et al.  Stochastic IMT (Insulator-Metal-Transition) Neurons: An Interplay of Thermal and Threshold Noise at Bifurcation , 2017, Front. Neurosci..

[45]  Jongkil Park,et al.  Live demonstration: Hierarchical Address-Event Routing architecture for reconfigurable large scale neuromorphic systems , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[46]  Deepak Khosla,et al.  Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition , 2014, International Journal of Computer Vision.

[47]  Russell C. Eberhart,et al.  The particle swarm: social adaptation in information-processing systems , 1999 .

[48]  Hojjat Adeli,et al.  Spiking Neural Networks , 2009, Int. J. Neural Syst..

[49]  J. Deneubourg,et al.  Self-organized shortcuts in the Argentine ant , 1989, Naturwissenschaften.

[50]  Chenming Hu,et al.  Impact of Parasitic Capacitance and Ferroelectric Parameters on Negative Capacitance FinFET Characteristics , 2017, IEEE Electron Device Letters.

[51]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[52]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[53]  Romain Brette,et al.  Equation-oriented specification of neural models for simulations , 2013, Front. Neuroinform..

[54]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[55]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[56]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[57]  S. Datta,et al.  Neuro-Mimetic Dynamics of a Ferroelectric FET-Based Spiking Neuron , 2019, IEEE Electron Device Letters.

[58]  Thomas Stützle,et al.  MAX-MIN Ant System , 2000, Future Gener. Comput. Syst..

[59]  Damien Querlioz,et al.  Vowel recognition with four coupled spin-torque nano-oscillators , 2017, Nature.