Hybrid deep neural network using transfer learning for EEG motor imagery decoding

Abstract A major challenge in motor imagery (MI) of electroencephalogram (EEG) based brain–computer interfaces (BCIs) is the individual differences for different people. That the classification model should be retrained from scratch for a new subject often leads to unnecessary time consumption. In this paper, a “brain-ID” framework based on the hybrid deep neural network with transfer learning (HDNN-TL) is proposed to deal with individual differences of 4-class MI task. An end-to-end HDNN is developed to learn the common features of MI signal. HDNN consists of convolutional neural network (CNN) and Long Short-Term Memory (LSTM) which are utilized to decode the spatial and temporal features of the MI signal simultaneously. To deal with the EEG individual differences problem, transfer learning technique is implemented to fine-tune the followed fully connected (FC) layer to accommodate new subject with fewer training data. The classification performance on BCI competition IV dataset 2a by the proposed HDNN-TL in terms of kappa value is 0.8. We compared HDNN-TL, HDNN and other state-of-art methods and the experimental results demonstrate that the proposed method can get a satisfying result for new subjects with less time and fewer training data in MI task.