Convolutional neural networks and genetic algorithm for visual imagery classification

Brain–Computer Interface (BCI) systems establish a channel for direct communication between the brain and the outside world without having to use the peripheral nervous system. While most BCI systems use evoked potentials and motor imagery, in the present work we present a technique that employs visual imagery. Our technique uses neural networks to classify the signals produced in visual imagery. To this end, we have used densely connected neural and convolutional networks, together with a genetic algorithm to find the best parameters for these networks. The results we obtained are a 60% success rate in the classification of four imagined objects (a tree, a dog, an airplane and a house) plus a state of relaxation, thus outperforming the state of the art in visual imagery classification.

[1]  Evgin Göçeri,et al.  Challenges and Recent Solutions for Image Segmentation in the Era of Deep Learning , 2019 .

[2]  Sam Kwong,et al.  Genetic algorithms: concepts and applications [in engineering design] , 1996, IEEE Trans. Ind. Electron..

[3]  Dan Liu,et al.  An effective feature extraction method by power spectral density of EEG signal for 2-class motor imagery-based BCI , 2018, Medical & Biological Engineering & Computing.

[4]  A. Ishai,et al.  Distributed neural systems for the generation of visual images , 2000, NeuroImage.

[5]  Mark W. Greenlee,et al.  Cortical activation evoked by visual mental imagery as measured by functional MRI , 2000 .

[6]  Ehsan Tarkesh Esfahani,et al.  Classification of primitive shapes using brain-computer interfaces , 2012, Comput. Aided Des..

[7]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update , 2018, Journal of neural engineering.

[8]  S. Gielen,et al.  The brain–computer interface cycle , 2009, Journal of neural engineering.

[9]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Kshitij Dwivedi,et al.  End-to-End Deep Image Reconstruction From Human Brain Activity , 2018, bioRxiv.

[11]  Hao Wu,et al.  An effective feature selection method for hyperspectral image classification based on genetic algorithm and support vector machine , 2011, Knowl. Based Syst..

[12]  A. Frolov,et al.  Brain-Computer Interface Based on Generation of Visual Images , 2011, PloS one.

[13]  P. Welch.  The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[14]  R. Malach,et al.  Negative BOLD Differentiates Visual Imagery and Perception , 2005, Neuron.

[15]  Evgin Goceri,et al.  Computer-based segmentation, change detection and quantification for lesions in multiple sclerosis , 2017, 2017 International Conference on Computer Science and Engineering (UBMK).

[16]  Enrique Hortal,et al.  Using a brain-machine interface to control a hybrid upper limb exoskeleton during rehabilitation of patients with neurological conditions , 2015, Journal of NeuroEngineering and Rehabilitation.

[17]  Jaime Gómez Gil,et al.  Brain Computer Interfaces, a Review , 2012, Sensors.

[18]  Evgin Goceri,et al.  Automated detection and extraction of skull from MR head images: Preliminary results , 2017, 2017 International Conference on Computer Science and Engineering (UBMK).

[19]  G. Horváth.  Visual imagination and the narrative image. Parallelisms between art history and neuroscience , 2018, Cortex.

[20]  Ying Sun,et al.  Asynchronous P300 BCI: SSVEP-based control state detection , 2010, 2010 18th European Signal Processing Conference.

[21]  Kip A Ludwig,et al.  Using a common average reference to improve cortical neuron recordings from microelectrode arrays. , 2009, Journal of neurophysiology.

[22]  Yuanqing Li,et al.  A Hybrid Brain Computer Interface to Control the Direction and Speed of a Simulated or Real Wheelchair , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  A. Zeman,et al.  The neural correlates of visual imagery: A co-ordinate-based meta-analysis , 2018, Cortex.

[24]  Joakim Riml,et al.  Changes in short term river flow regulation and hydropeaking in Nordic rivers , 2018, Scientific Reports.

[25]  Fernando Lopez-Lezcano,et al.  Center for Computer Research in Music and Acoustics (CCRMA) , 1994, ICMC.

[26]  Tanja Schultz,et al.  Brain-to-text: decoding spoken phrases from phone representations in the brain , 2015, Front. Neurosci..

[27]  S. Kosslyn,et al.  Brain areas underlying visual mental imagery and visual perception: an fMRI study. , 2004, Brain research. Cognitive brain research.

[28]  M. McHugh.  Interrater reliability: the kappa statistic , 2012, Biochemia medica.

[29]  Guohua Shen,et al.  Deep image reconstruction from human brain activity , 2017, bioRxiv.

[30]  Sam Kwong,et al.  Genetic algorithms: concepts and applications [in engineering design] , 1996, IEEE Trans. Ind. Electron..

[31]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[32]  Andrés Úbeda,et al.  Evaluating Classifiers to Detect Arm Movement Intention from EEG Signals , 2014, Sensors.

[33]  Evgin Goceri,et al.  Analysis of Deep Networks with Residual Blocks and Different Activation Functions: Classification of Skin Diseases , 2019, 2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA).

[34]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[35]  R. VanRullen,et al.  The Phase of Ongoing EEG Oscillations Predicts Visual Perception , 2009, The Journal of Neuroscience.

[36]  R. B. Reilly,et al.  FASTER: Fully Automated Statistical Thresholding for EEG artifact Rejection , 2010, Journal of Neuroscience Methods.

[37]  Abdulhamit Subasi,et al.  Classification of EEG signals using neural network and logistic regression , 2005, Comput. Methods Programs Biomed..

[38]  Evgin Goceri,et al.  Diagnosis of Alzheimer's disease with Sobolev gradient‐based optimization and 3D convolutional neural network , 2019, International journal for numerical methods in biomedical engineering.

[39]  John Onians,et al.  The Eye's mind – Visual imagination, neuroscience and the humanities , 2018, Cortex.

[40]  O. Ozdamar,et al.  Wavelet preprocessing for automated neural network detection of EEG spikes , 1995 .

[41]  E. Vogel,et al.  The visual N1 component as an index of a discrimination process. , 2000, Psychophysiology.

[42]  Lingling Yang,et al.  An online BCI game based on the decoding of users' attention to color stimulus , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[43]  Evgin Goceri,et al.  Skin Disease Diagnosis from Photographs Using Deep Learning , 2019, VipIMAGE 2019.

[44]  Amjed S. Al-Fahoum,et al.  Methods of EEG Signal Features Extraction Using Linear Analysis in Frequency and Time-Frequency Domains , 2014, ISRN neuroscience.

[45]  Leslie G. Ungerleider,et al.  Distributed Neural Systems for the Generation of Visual Images , 2000, Neuron.

[46]  B. Fischer,et al.  Visual field representations and locations of visual areas V1/2/3 in human visual cortex. , 2003, Journal of vision.

[47]  T. Katsuura,et al.  Effects of object color stimuli on human brain activities in perception and attention referred to EEG alpha band response. , 2007, Journal of physiological anthropology.

[48]  D. Heeger,et al.  Decoding and Reconstructing Color from Responses in Human Visual Cortex , 2009, The Journal of Neuroscience.

[49]  Evgin Goceri,et al.  Challenges and Recent Solutions for Image Segmentation in the Era of Deep Learning , 2019, 2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA).

[50]  Melanie Mitchell,et al.  An introduction to genetic algorithms , 1996 .

[51]  Jussi T. Lindgren,et al.  Attending to Visual Stimuli versus Performing Visual Imagery as a Control Strategy for EEG-based Brain-Computer Interfaces , 2018, Scientific Reports.

[52]  Chi Zhang,et al.  Constraint-Free Natural Image Reconstruction From fMRI Signals Based on Convolutional Neural Network , 2018, Front. Hum. Neurosci..

[53]  Mubarak Shah,et al.  Brain2Image: Converting Brain Signals into Images , 2017, ACM Multimedia.

[54]  Arthur W. Wetzel,et al.  Network anatomy and in vivo physiology of visual cortical neurons , 2011, Nature.

[55]  Kyung-shik Shin,et al.  A genetic algorithm application in bankruptcy prediction modeling , 2002, Expert Syst. Appl..

[56]  Elif Derya Übeyli,et al.  Recurrent neural networks employing Lyapunov exponents for EEG signals classification , 2005, Expert Syst. Appl..

[57]  Manuel Schabus,et al.  A shift of visual spatial attention is selectively associated with human EEG alpha activity , 2005, The European journal of neuroscience.

[58]  A. Franklin,et al.  Categorical encoding of color in the brain , 2014, Proceedings of the National Academy of Sciences.

[59]  Catherine Tallon-Baudry,et al.  Neural responses to heartbeats distinguish self from other during imagination , 2019, NeuroImage.

[60]  S. Coyle,et al.  Brain–computer interfaces: a review , 2003 .

[61]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[62]  Norlaili Mat Safri,et al.  EEG based bci using visual imagery task for robot control , 2013 .

[63]  Febo Cincotti,et al.  Out of the frying pan into the fire--the P300-based BCI faces real-world challenges. , 2011, Progress in brain research.

[64]  Dimitrios Pantazis,et al.  Ultra-Rapid serial visual presentation reveals dynamics of feedforward and feedback processes in the ventral visual pathway , 2018, bioRxiv.

[65]  Hasan Ocak,et al.  Optimal classification of epileptic seizures in EEG using wavelet analysis and genetic algorithm , 2008, Signal Process..

[66]  Mohd Nasir Taib,et al.  The Analysis of EEG Spectrogram Image for Brainwave Balancing Application Using ANN , 2011, 2011 UkSim 13th International Conference on Computer Modelling and Simulation.

[67]  Fraser Milton,et al.  The neural correlates of visual imagery vividness – An fMRI study and literature review , 2017, Cortex.