Brain2Object: Printing Your Mind from Brain Signals with Spatial Correlation Embedding

Electroencephalography (EEG) signals are known to manifest differential patterns when individuals visually concentrate on different objects (e.g., a car). In this work, we present an end-to-end digital fabrication system , Brain2Object, to print the 3D object that an individual is observing by solely decoding visually-evoked EEG brain signal streams. We propose a unified training framework which combines multi-class Common Spatial Pattern and deep Convolutional Neural Networks to support the backend computation. Specially, a Dynamical Graph Representation of EEG signals is learned for accurately capturing the structured spatial correlations of EEG channels in an adaptive manner. A user friendly interface is developed as the system front end. Brain2Object presents a streamlined end-to-end workflow which can serve as a template for deeper integration of BCI technologies to assist with our routine activities. The proposed system is evaluated extensively using offline experiments and through an online demonstrator. For the former, we use a rich widely used public dataset and a limited but locally collected dataset. The experiment results show that our approach consistently outperforms a wide range of baseline and state-of-the-art approaches. The proof-of-concept corroborates the practicality of our approach and illustrates the ease with which such a system could be deployed.

[1]  Lina Yao,et al.  Converting Your Thoughts to Texts: Enabling Brain Typing via Deep Feature Learning of EEG Signals , 2017, 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[2]  Liqing Zhang,et al.  Feature learning from incomplete EEG with denoising autoencoder , 2014, Neurocomputing.

[3]  Qin Lin,et al.  Classification of Epileptic EEG Signals with Stacked Sparse Autoencoder Based on Deep Learning , 2016, ICIC.

[4]  Oberdan R. Pinheiro,et al.  Wheelchair simulator game for training people with severe disabilities , 2016, 2016 1st International Conference on Technology and Innovation in Sports, Health and Wellbeing (TISHW).

[5]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[6]  Yijun Wang,et al.  Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[7]  Seungjin Choi,et al.  Bayesian common spatial patterns for multi-subject EEG classification , 2014, Neural Networks.

[8]  Simon Hanslmayr,et al.  Data-driven re-referencing of intracranial EEG based on independent component analysis (ICA) , 2017, Journal of Neuroscience Methods.

[9]  A. Prasad Vinod,et al.  EEG-based motor imagery classification using subject-specific spatio-spectral features , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[10]  Cuntai Guan,et al.  On the use of convolutional neural networks and augmented CSP features for multi-class motor imagery of EEG signals classification , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[11]  Zhenqi Li,et al.  Emotion Recognition from EEG Using RASM and LSTM , 2017, ICIMCS.

[12]  Wenming Zheng,et al.  EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks , 2020, IEEE Transactions on Affective Computing.

[13]  Ayman AbuBaker,et al.  EEG Mouse:A Machine Learning-Based Brain Computer Interface , 2014 .

[14]  Toshihisa Tanaka,et al.  Active Data Selection for Motor Imagery EEG Classification , 2015, IEEE Transactions on Biomedical Engineering.

[15]  Klaus-Robert Müller,et al.  Interpretable deep neural networks for single-trial EEG classification , 2016, Journal of Neuroscience Methods.

[16]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Aurobinda Routray,et al.  Statistical features extraction for multivariate pattern analysis in meditation EEG using PCA , 2016, 2016 IEEE EMBS International Student Conference (ISC).

[18]  Cong Wang,et al.  DeepMag: Sniffing Mobile Apps in Magnetic Field through Deep Convolutional Neural Networks , 2018, 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[19]  Yong Zhang,et al.  Classification of EEG Signals Based on Autoregressive Model and Wavelet Packet Decomposition , 2016, Neural Processing Letters.

[20]  Yilmaz Kaya,et al.  1D-local binary pattern based feature extraction for classification of epileptic EEG signals , 2014, Appl. Math. Comput..

[21]  Jean Gotman,et al.  Diagnostic utility of invasive EEG for epilepsy surgery: Indications, modalities, and techniques , 2016, Epilepsia.

[22]  Lucas C. Parra,et al.  Recipes for the linear analysis of EEG , 2005, NeuroImage.

[23]  Hardik Meisheri,et al.  Multiclass Common Spatial Pattern for EEG based Brain Computer Interface with Adaptive Learning Classifier , 2018, ArXiv.

[24]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[25]  Jiamin Liu,et al.  Multi-modal Emotion Recognition with Temporal-Band Attention Based on LSTM-RNN , 2017, PCM.

[26]  Lina Yao,et al.  Intent Recognition in Smart Living Through Deep Recurrent Neural Networks , 2017, ICONIP.

[27]  Owen Falzon,et al.  EEG feature extraction using common spatial pattern with spectral graph decomposition , 2017, 2017 International Conference on Computing Networking and Informatics (ICCNI).

[28]  Luca Maria Gambardella,et al.  Convolutional Neural Network Committees for Handwritten Character Classification , 2011, 2011 International Conference on Document Analysis and Recognition.

[29]  Andreas Schulze-Bonhage,et al.  Signal quality of simultaneously recorded invasive and non-invasive EEG , 2009, NeuroImage.

[30]  Danilo P. Mandic,et al.  Augmented Complex Common Spatial Patterns for Classification of Noncircular EEG From Motor Imagery Tasks , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[31]  Michael Bach,et al.  ISCEV standard for clinical visual evoked potentials (2009 update) , 2010, Documenta Ophthalmologica.

[32]  Wolfram Burgard,et al.  Deep transfer learning for error decoding from non-invasive EEG , 2017, 2018 6th International Conference on Brain-Computer Interface (BCI).

[33]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[34]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[35]  Peng Xu,et al.  Autoregressive model in the Lp norm space for EEG analysis , 2015, Journal of Neuroscience Methods.

[36]  Michael Bach,et al.  ISCEV standard for clinical visual evoked potentials: (2016 update) , 2016, Documenta Ophthalmologica.

[37]  U. Rajendra Acharya,et al.  Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals , 2017, Comput. Biol. Medicine.