Convolutional Spiking Neural Networks for Detecting Anticipatory Brain Potentials Using Electroencephalogram

Spiking neural networks (SNNs) are receiving increased attention as a means to develop"biologically plausible"machine learning models. These networks mimic synaptic connections in the human brain and produce spike trains, which can be approximated by binary values, precluding high computational cost with floating-point arithmetic circuits. Recently, the addition of convolutional layers to combine the feature extraction power of convolutional networks with the computational efficiency of SNNs has been introduced. In this paper, the feasibility of using a convolutional spiking neural network (CSNN) as a classifier to detect anticipatory slow cortical potentials related to braking intention in human participants using an electroencephalogram (EEG) was studied. The EEG data was collected during an experiment wherein participants operated a remote controlled vehicle on a testbed designed to simulate an urban environment. Participants were alerted to an incoming braking event via an audio countdown to elicit anticipatory potentials that were then measured using an EEG. The CSNN's performance was compared to a standard convolutional neural network (CNN) and three graph neural networks (GNNs) via 10-fold cross-validation. The results showed that the CSNN outperformed the other neural networks.

[1]  Michael S. Gaffrey,et al.  Spatiotemporal dynamics of EEG microstates in four- to eight-year-old children: Age- and sex-related effects , 2022, Developmental Cognitive Neuroscience.

[2]  A. Grunin,et al.  Neuromorphic artificial intelligence systems , 2022, Frontiers in Neuroscience.

[3]  Roman Moucek,et al.  Spiking Neural Networks for Classification of Brain-Computer Interface and Image Data , 2021, 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[4]  Martin J. McKeown,et al.  Toward Open-World Electroencephalogram Decoding Via Deep Learning: A comprehensive survey , 2021, IEEE Signal Processing Magazine.

[5]  Nikola Kasabov,et al.  Spiking Neural Networks for Computational Intelligence: An Overview , 2021, Big Data Cogn. Comput..

[6]  S. Ehrlich,et al.  Demonstrating the Viability of Mapping Deep Learning Based EEG Decoders to Spiking Networks on Low-powered Neuromorphic Chips , 2021, 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[7]  D. Jeong,et al.  Training Spiking Neural Networks Using Lessons From Deep Learning , 2021, Proceedings of the IEEE.

[8]  Toshiaki Koike-Akino,et al.  EEG-GNN: Graph Neural Networks for Classification of Electroencephalogram (EEG) Signals , 2021, 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[9]  Gouhei Tanaka,et al.  2022 roadmap on neuromorphic computing and engineering , 2021, Neuromorph. Comput. Eng..

[10]  Sami Barchid,et al.  Deep Spiking Convolutional Neural Network for Single Object Localization Based On Deep Continuous Local Learning , 2021, 2021 International Conference on Content-Based Multimedia Indexing (CBMI).

[11]  Garrick Orchard,et al.  Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook , 2021, Proceedings of the IEEE.

[12]  Hong Qu,et al.  Classification of Alzheimer’s Disease Using Deep Convolutional Spiking Neural Network , 2021, Neural Processing Letters.

[13]  Sai Kalyan Ranga Singanamalla,et al.  Spiking Neural Network for Augmenting Electroencephalographic Data for Brain Computer Interfaces , 2021, Frontiers in Neuroscience.

[14]  N. Kasabov,et al.  Brain-inspired spiking neural networks for decoding and understanding muscle activity and kinematics from electroencephalography signals during hand movements , 2021, Scientific Reports.

[15]  Gaetano Di Caterina,et al.  A New Spiking Convolutional Recurrent Neural Network (SCRNN) With Applications to Event-Based Hand Gesture Recognition , 2020, Frontiers in Neuroscience.

[16]  Nikola Kasabov,et al.  Spiking Neural Networks: Background, Recent Development and the NeuCube Architecture , 2020, Neural Processing Letters.

[17]  Y. Li,et al.  Attention-based Graph ResNet for Motor Intent Detection from Raw EEG signals , 2020, ArXiv.

[18]  Y. Li,et al.  GCNs-Net: A Graph Convolutional Neural Network Approach for Decoding Time-resolved EEG Motor Imagery Signals , 2020, IEEE transactions on neural networks and learning systems.

[19]  Norizam Sulaiman,et al.  Current Status, Challenges, and Possible Solutions of EEG-Based Brain-Computer Interface: A Comprehensive Review , 2020, Frontiers in Neurorobotics.

[20]  Yi Cao,et al.  EEG-Based Emotion Classification Using Spiking Neural Networks , 2020, IEEE Access.

[21]  Maryam Gholami Doborjeh,et al.  Deep Learning of EEG Data in the NeuCube Brain-Inspired Spiking Neural Network Architecture for a Better Understanding of Depression , 2019, ICONIP.

[22]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[23]  Gopalakrishnan Srinivasan,et al.  Deep Spiking Convolutional Neural Network Trained With Unsupervised Spike-Timing-Dependent Plasticity , 2019, IEEE Transactions on Cognitive and Developmental Systems.

[24]  Ruthvik Vaila,et al.  Feature Extraction using Spiking Convolutional Neural Networks , 2019, ICONS.

[25]  Chunyan Miao,et al.  EEG-Based Emotion Recognition Using Regularized Graph Neural Networks , 2019, IEEE Transactions on Affective Computing.

[26]  Wan-Young Chung,et al.  Detection of Driver Braking Intention Using EEG Signals During Simulated Driving , 2019, Sensors.

[27]  Yansong Chua,et al.  Hardware-friendly Neural Network Architecture for Neuromorphic Computing , 2019, ArXiv.

[28]  Ricardo Chavarriaga,et al.  Real-time Detection of Driver’s Movement Intention in Response to Traffic Lights , 2019, bioRxiv.

[29]  Meng Dong,et al.  Unsupervised speech recognition through spike-timing-dependent plasticity in a convolutional spiking neural network , 2018, PloS one.

[30]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[31]  Mohamed El-Habrouk,et al.  A survey of Automotive Driving Assistance Systems technologies , 2018, 2018 International Conference on Artificial Intelligence and Data Processing (IDAP).

[32]  Gopalakrishnan Srinivasan,et al.  Training Deep Spiking Convolutional Neural Networks With STDP-Based Unsupervised Pre-training Followed by Supervised Fine-Tuning , 2018, Front. Neurosci..

[33]  Luzheng Bi,et al.  EEG-Based Detection of Driver Emergency Braking Intention for Brain-Controlled Vehicles , 2018, IEEE Transactions on Intelligent Transportation Systems.

[34]  José Manuel Ferrández,et al.  EEG-Based Detection of Braking Intention Under Different Car Driving Conditions , 2018, Front. Neuroinform..

[35]  T. Masquelier,et al.  Deep Learning in Spiking Neural Networks , 2018, Neural Networks.

[36]  Seong-Whan Lee,et al.  Detecting Driver's Braking Intention Using Recurrent Convolutional Neural Networks Based EEG Analysis , 2017, 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR).

[37]  Yufei Huang,et al.  Prediction of fatigue-related driver performance from EEG data by deep Riemannian model , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[38]  Bao-Liang Lu,et al.  A multimodal approach to estimating vigilance using EEG and forehead EOG , 2016, Journal of neural engineering.

[39]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[40]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[41]  Ricardo Chavarriaga,et al.  Action prediction based on anticipatory brain potentials during simulated driving , 2015, Journal of neural engineering.

[42]  Stefan Haufe,et al.  Detection of braking intention in diverse situations during simulated driving based on EEG feature combination , 2015, Journal of neural engineering.

[43]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[44]  J. Millán,et al.  Single trial prediction of self-paced reaching directions from EEG signals , 2014, Front. Neurosci..

[45]  J. Millán,et al.  Single trial analysis of slow cortical potentials: a study on anticipation related potentials , 2013, Journal of neural engineering.

[46]  J. Millán,et al.  Detection of self-paced reaching movement intention from EEG signals , 2012, Front. Neuroeng..

[47]  Stefan Haufe,et al.  EEG potentials predict upcoming emergency brakings during simulated driving , 2011, Journal of neural engineering.

[48]  Chin-Teng Lin,et al.  EEG-based cognitive state monitoring and predition by using the self-constructing neural fuzzy system , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[49]  Chin-Teng Lin,et al.  Driver's cognitive state classification toward brain computer interface via using a generalized and supervised technology , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[50]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[51]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[52]  Katsuhiko Mori,et al.  Convolutional spiking neural network model for robust face detection , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[53]  Tony Travers,et al.  A Comprehensive Review , 1998 .

[54]  Mario Osvin Pavčević,et al.  Introduction to graph theory , 1973, The Mathematical Gazette.

[55]  Salimur Choudhury,et al.  A Comprehensive Survey on the Ambulance Routing and Location Problems , 2020, ArXiv.

[56]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[57]  Rajakrishnan Rajkumar,et al.  Grammar Engineering for CCG using Ant and XSLT ∗ , 2001 .