Fusing Frontal and Occipital EEG Features to Detect “Brain Switch” by Utilizing Convolutional Neural Network

Flicker is the most widely used steady-state visual evoked potential (SSVEP) stimulus. In addition, checkerboard can induce steady-state motion visual evoked potential (SSMVEP) in the occipital area. More recently, the action video is proposed to simultaneously elicit SSMVEP and induce sensorimotor area activations via the mirror neuron systems through Action Observation (AO). Integration of AO with brain–computer interface (BCI) is appealing for neural rehabilitation applications. In order to make such a BCI paradigm more feasible in neural rehabilitation, it is essential to discriminate whether a user is actively engaged with the BCI, i.e. intentional control (IC) state, or not engaged, i.e. non-intentional control (NC) state. In this study, the EEG responses to these three types of visual stimuli were compared for the first time and a convolutional neural network (CNN) was proposed to discriminate IC and NC states. A visual gaiting stimulus was designed to realize BCI-based AO. The results showed that the power of alpha rhythms from frontal area decreased more when the participants engaged at the gaiting stimuli than when the participants engaged at the other two types of stimuli. In addition, the correlation coefficient between the EEG from the occipital area and the template signals increased when the participants engaged at the stimuli. The results also clearly demonstrated the proposed CNN method can discriminate the IC and NC states. In addition, the combination of the attention feature from the frontal area and the SSVEP/SSMVEP feature from the occipital area showed significant performance improvement for the gaiting stimulus, but not for the other two types of stimuli.