A Unified Software/Hardware Scalable Architecture for Brain-Inspired Computing Based on Self-Organizing Neural Models

The field of artificial intelligence has significantly advanced over the past decades, inspired by discoveries from the fields of biology and neuroscience. The idea of this work is inspired by the process of self-organization of cortical areas in the human brain from both afferent and lateral/internal connections. In this work, we develop an original brain-inspired neural model associating Self-Organizing Maps (SOM) and Hebbian learning in the Reentrant SOM (ReSOM) model. The framework is applied to multimodal classification problems. Compared to existing methods based on unsupervised learning with post-labeling, the model enhances the state-of-the-art results. This work also demonstrates the distributed and scalable nature of the model through both simulation results and hardware execution on a dedicated FPGA-based platform named SCALP (Self-configurable 3D Cellular Adaptive Platform). SCALP boards can be interconnected in a modular way to support the structure of the neural model. Such a unified software and hardware approach enables the processing to be scaled and allows information from several modalities to be merged dynamically. The deployment on hardware boards provides performance results of parallel execution on several devices, with the communication between each board through dedicated serial links. The proposed unified architecture, composed of the ReSOM model and the SCALP hardware platform, demonstrates a significant increase in accuracy thanks to multimodal association, and a good trade-off between latency and power consumption compared to a centralized GPU implementation.

[1]  Matin Hashemi,et al.  ECG Classification Algorithm Based on STDP and R-STDP Neural Networks for Real-Time Monitoring on Ultra Low-Power Personal Wearable Devices , 2019, IEEE Transactions on Biomedical Circuits and Systems.

[2]  M. Casasola,et al.  Acquisition of word-object associations by 14-month-old infants. , 1998, Developmental psychology.

[3]  Carlos Gershenson,et al.  The Meaning of Self-organization in Computing , 2003 .

[4]  Bernard Girau,et al.  Pruning Self-Organizing Maps for Cellular Hardware Architectures , 2018, 2018 NASA/ESA Conference on Adaptive Hardware and Systems (AHS).

[5]  E. M. Rouiller,et al.  Multisensory anatomical pathways , 2009, Hearing Research.

[6]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[7]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[8]  Yann Boniface,et al.  Dynamic self-organising map , 2011, Neurocomputing.

[9]  A. Giraud,et al.  Implicit Multisensory Associations Influence Voice Recognition , 2006, PLoS biology.

[10]  T. Kohonen.  SELF-ORGANIZING MAPS: OPHMIZATION APPROACHES , 1991 .

[11]  Stefan Wermter,et al.  Emergence of multimodal action representations from neural network self-organization , 2017, Cognitive Systems Research.

[12]  Bruno Lara-Guzmán,et al.  A Self-Organized Internal Models Architecture for Coding Sensory–Motor Schemes , 2016, Front. Robot. AI.

[13]  G. Edelman.  Neural Darwinism: The Theory Of Neuronal Group Selection , 1989 .

[14]  Serge Weber,et al.  High performance scalable hardware SOM architecture for real-time vector quantization , 2018, 2018 IEEE International Conference on Image Processing, Applications and Systems (IPAS).

[15]  R. Douglas,et al.  Neuronal circuits of the neocortex. , 2004, Annual review of neuroscience.

[16]  Pete Warden,et al.  Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition , 2018, ArXiv.

[17]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[18]  S. T. Brassai.  FPGA based hardware implementation of a self-organizing map , 2014, IEEE 18th International Conference on Intelligent Engineering Systems INES 2014.

[19]  Nicholas Cain,et al.  The computational properties of a simplified cortical column model , 2014, BMC Neuroscience.

[20]  Mohamed Hédi Bedoui,et al.  A scalable and adaptable hardware NoC-based self organizing map , 2018, Microprocess. Microsystems.

[21]  Andrew S. Cassidy,et al.  Cognitive computing building block: A versatile and efficient digital neuron model for neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[22]  Mathieu Lefort,et al.  Self-organization of neural maps using a modulated BCM rule within a multimodal architecture , 2010, BICS 2010.

[23]  G. Edelman.  Neural Darwinism: Selection and reentrant signaling in higher brain function , 1993, Neuron.

[24]  G. Edelman,et al.  Degeneracy and complexity in biological systems , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[25]  Erzsébet Merényi,et al.  A reconfigurable neuroprocessor for self-organizing feature maps , 2013, Neurocomputing.

[26]  Sungroh Yoon,et al.  Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection , 2020, AAAI.

[27]  Kaushik Roy,et al.  STDP Based Unsupervised Multimodal Learning With Cross-Modal Processing in Spiking Neural Networks , 2021, IEEE Transactions on Emerging Topics in Computational Intelligence.

[28]  Kevin Warwick,et al.  The plastic self organising map , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[29]  Alexander N. Gorban,et al.  Elastic Principal Graphs and Manifolds and their Practical Applications , 2005, Computing.

[30]  Hananel Hazan,et al.  Unsupervised Learning with Self-Organizing Spiking Neural Networks , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[31]  Yusuke Mukuta,et al.  Fully Spiking Variational Autoencoder , 2021, AAAI.

[32]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[33]  Peyman Najafirad,et al.  Generalized Zero-Shot Learning Using Multimodal Variational Auto-Encoder With Semantic Concepts , 2021, 2021 IEEE International Conference on Image Processing (ICIP).

[34]  Jorge Peña,et al.  Digital Hardware Architectures of Kohonen's Self Organizing Feature Maps with Exponential Neighboring Function , 2006, 2006 IEEE International Conference on Reconfigurable Computing and FPGA's (ReConFig 2006).

[35]  Hojjat Adeli,et al.  Third Generation Neural Networks: Spiking Neural Networks , 2009 .

[36]  Bernard Girau,et al.  Cellular Self-Organising Maps - CSOM , 2019, WSOM+.

[37]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[38]  G. Edelman,et al.  Reentry: a key mechanism for integration of brain function , 2013, Front. Integr. Neurosci..

[39]  D. Querlioz,et al.  Visual Pattern Extraction Using Energy-Efficient “2-PCM Synapse” Neuromorphic Architecture , 2012, IEEE Transactions on Electron Devices.

[40]  Saemundur O. Haraldsson,et al.  Exploring the Accuracy – Energy Trade-off in Machine Learning , 2021, 2021 IEEE/ACM International Workshop on Genetic Improvement (GI).

[41]  Hao Dong,et al.  Contrastive Multimodal Fusion with TupleInfoNCE , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[42]  Pedro H. M. Braga,et al.  Deep Categorization with Semi-Supervised Self-Organizing Maps , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[43]  Laurent Rodriguez,et al.  Improving Self-Organizing Maps with Unsupervised Feature Extraction , 2020, ICONIP.

[44]  Dezhi Han,et al.  Multimodal Encoder-Decoder Attention Networks for Visual Question Answering , 2020, IEEE Access.

[45]  Takuya Akiba,et al.  Optuna: A Next-generation Hyperparameter Optimization Framework , 2019, KDD.

[46]  Tracy Brown,et al.  The Embodied Mind: Cognitive Science and Human Experience , 2002, Cybern. Hum. Knowing.

[47]  W. Singer,et al.  The formation of cooperative cell assemblies in the visual cortex. , 1990, The Journal of experimental biology.

[48]  Vincent Gripon,et al.  GPU-based Self-Organizing Maps for Post-Labeled Few-Shot Unsupervised Learning , 2020, ICONIP.

[49]  Arda Mavi,et al.  A New Dataset and Proposed Convolutional Neural Network Architecture for Classification of American Sign Language Digits , 2020, ArXiv.

[50]  G. Edelman.  Group selection and phasic reentrant signaling a theory of higher brain function , 1982 .

[51]  Andreas Rauber,et al.  The growing hierarchical self-organizing map , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[52]  K. Plunkett,et al.  Timing matters: The impact of label synchrony on infant categorisation , 2015, Cognition.

[53]  Steve B. Furber,et al.  Modeling Spiking Neural Networks on SpiNNaker , 2010, Computing in Science & Engineering.

[54]  Andres Upegui,et al.  SCALP: Self-configurable 3-D Cellular Adaptive Platform , 2018, 2018 IEEE Symposium Series on Computational Intelligence (SSCI).

[55]  Irem Dikmen,et al.  Comparing the performance of traditional cluster analysis, self-organizing maps and fuzzy C-means method for strategic grouping , 2009, Expert Syst. Appl..

[56]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[57]  Laurent Rodriguez,et al.  A distributed cellular approach of large scale SOM models for hardware implementation , 2018, 2018 IEEE International Conference on Image Processing, Applications and Systems (IPAS).

[58]  Stefan Schliebs,et al.  Evolving spiking neural network—a survey , 2013, Evolving Systems.

[59]  Jianguo Xin,et al.  Supervised learning with spiking neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[60]  E. Culurciello,et al.  NeuFlow: Dataflow vision processing system-on-a-chip , 2012, 2012 IEEE 55th International Midwest Symposium on Circuits and Systems (MWSCAS).

[61]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[62]  Nadja Althaus,et al.  Modeling Cross-Modal Interactions in Early Word Learning , 2013, IEEE Transactions on Autonomous Mental Development.

[63]  Angelo Cangelosi,et al.  Epigenetic Robotics Architecture (ERA) , 2010, IEEE Transactions on Autonomous Mental Development.

[64]  Emilio Del-Moral-Hernandez,et al.  Comparison of three FPGA architectures for embedded multidimensional categorization through Kohonen's self-organizing maps , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[65]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[66]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[67]  Giacomo Indiveri,et al.  Frontiers in Neuromorphic Engineering , 2011, Front. Neurosci..

[68]  Zhenzhong Chen,et al.  A Multimodal Variational Encoder-Decoder Framework for Micro-video Popularity Prediction , 2020, WWW.

[69]  Michael Gasser,et al.  The Development of Embodied Cognition: Six Lessons from Babies , 2005, Artificial Life.

[70]  T. Masquelier,et al.  S4NN: temporal backpropagation for spiking neural networks with one spike per neuron , 2019, Int. J. Neural Syst..

[71]  Emilio Del-Moral-Hernandez,et al.  An FPGA distributed implementation model for embedded SOM with on-line learning , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[72]  Jean-Marc Philippe,et al.  NeuroDSP Accelerator for Face Detection Application , 2015, ACM Great Lakes Symposium on VLSI.

[73]  Basil Sh. Mahmood,et al.  Reconfigurable Self-Organizing Neural Network Design and it's FPGA Implementation , 2009 .

[74]  C. Shatz.  How are specific connections formed between thalamus and cortex? , 1992, Current Opinion in Neurobiology.

[75]  Tomoaki Nakamura,et al.  Bag of multimodal LDA models for concept formation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[76]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[77]  Hiroomi Hikawa.  FPGA implementation of self organizing map with digital phase locked loops , 2005, Neural Networks.

[78]  Christian Jutten,et al.  Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects , 2015, Proceedings of the IEEE.

[79]  Eugenio Culurciello,et al.  Convolutional Clustering for Unsupervised Learning , 2015, ArXiv.

[80]  G. Calvert.  Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[81]  Meng Dong,et al.  Unsupervised speech recognition through spike-timing-dependent plasticity in a convolutional spiking neural network , 2018, PloS one.

[82]  Hélène Paugam-Moisy,et al.  Bidirectional Associative Memory for Multimodal Fusion : a Depression Evaluation Case Study , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[83]  Andres Upegui,et al.  Self-organizing neurons: toward brain-inspired unsupervised learning , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[84]  Garrick Orchard,et al.  Efficient Neuromorphic Signal Processing with Loihi 2 , 2021, 2021 IEEE Workshop on Signal Processing Systems (SiPS).

[85]  Francisco S. Melo,et al.  MHVAE: a Human-Inspired Deep Hierarchical Generative Model for Multimodal Representation Learning , 2020, ArXiv.

[86]  Catherine D. Schuman,et al.  Dynamic adaptive neural network arrays: a neuromorphic architecture , 2015, MLHPC@SC.

[87]  Peter Ford Dominey,et al.  Multi-modal convergence maps: from body schema and self-representation to mental imagery , 2013, Adapt. Behav..

[88]  Mohamed Hédi Bedoui,et al.  A Scalable Flexible SOM NoC-Based Hardware Architecture , 2016, WSOM.

[89]  Daswin De Silva,et al.  Unsupervised skill transfer learning for autonomous robots using distributed Growing Self Organizing Maps , 2021, Robotics Auton. Syst..

[90]  Sander M. Bohte,et al.  Unsupervised clustering with spiking neurons by sparse temporal coding and multilayer RBF networks , 2002, IEEE Trans. Neural Networks.