A Hybrid BCI Approach to Detect Brain Switch in Action Observation by Utilizing Convolution Neural Network

Action observation (AO) is a promising methodology for promoting cortical activations of the sensorimotor area via the mirror neuron systems. Integration of AO with brain-computer interfaces (BCIs) is appealing for neural rehabilitation applications. One of the important issues in BCI-based AO is discriminating when a user is actively engaged with the BCI,i.e. intentional control (IC) state versus when a user is not engaged with the BCI, i.e. non-intentional control (NC) state. In this study, we designed a visual gaiting stimulus, which was in the form gaiting sequence of a human, to elicit the steady-state motion visual evoked potential (SSMVEP) response in visual area. And we proposed a convolutional neural network (CNN) to discriminate IC and NC states. The results illustrated the proposed CNN can discriminate the IC and NC states. Furthermore, combining attention feature from the frontal area and SSMVEP features, i.e. a hybrid approach, showed significant performance improvement over only using SSMVEP features when the time window length was longer than 2s. And the accuracy achieved 91.94±6.20% when utilizing the hybrid approach in IC vs. NC discrimination.