Subject Matching for Cross-Subject EEG-based Recognition of Driver States Related to Situation Awareness.

Situation awareness (SA) has received much attention in recent years because of its importance for operators of dynamic systems. Electroencephalography (EEG) can be used to measure mental states of operators related to SA. However, cross-subject EEG-based SA recognition is a critical challenge, as data distributions of different subjects vary significantly. Subject variability is considered as a domain shift problem. Several attempts have been made to find domain-invariant features among subjects, where subject-specific information is neglected. In this work, we propose a simple but efficient subject matching framework by finding a connection between a target (test) subject and source (training) subjects. Specifically, the framework includes two stages: (1) we train the model with multi-source domain alignment layers to collect source domain statistics. (2) During testing, a distance is computed to perform subject matching in the latent representation space. We use a reciprocal exponential function as a similarity measure to dynamically select similar source subjects. Experiment results show that our framework achieves a state-of-the-art accuracy 74.14% for the Taiwan driving dataset.

[1]  D. Jude Hemanth,et al.  Human emotion recognition using intelligent approaches: A review , 2020, Intell. Decis. Technol..

[2]  Olga Sourina,et al.  Inter-subject transfer learning for EEG-based mental fatigue recognition , 2020, Adv. Eng. Informatics.

[3]  Fabio Maria Carlucci,et al.  Domain Generalization by Solving Jigsaw Puzzles , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Wei Tech Ang,et al.  Mobile EEG-based situation awareness recognition for air traffic controllers , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[5]  Muhammad Ghulam,et al.  Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion , 2019, Future Gener. Comput. Syst..

[6]  Federico Tombari,et al.  Batch Normalization Embeddings for Deep Domain Generalization , 2020, Pattern Recognit..

[7]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  M. Endsley Automation and situation awareness. , 1996 .

[9]  Jiaying Liu,et al.  Adaptive Batch Normalization for practical domain adaptation , 2018, Pattern Recognit..

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[12]  Jun Zhang,et al.  Dynamic frequency feature selection based approach for classification of motor imageries , 2016, Comput. Biol. Medicine.

[13]  Sreenatha G. Anavatti,et al.  Encephalographic Assessment of Situation Awareness in Teleoperation of Human-Swarm Teaming , 2019, ICONIP.

[14]  E. Kapetanios,et al.  Assessing the Effectiveness of Automated Emotion Recognition in Adults and Children for Clinical Investigation , 2020, Frontiers in Human Neuroscience.

[15]  Xi Peng,et al.  Learning to Learn Single Domain Generalization , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Bohyung Han,et al.  Domain-Specific Batch Normalization for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Chiara Knecht,et al.  Situation Awareness Training for General Aviation Pilots using Eye Tracking , 2016 .

[18]  Mica R. Endsley,et al.  Measurement of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[19]  Gernot R. Müller-Putz,et al.  Domain Adaptation Techniques for EEG-Based Emotion Recognition: A Comparative Study on Two Public Datasets , 2019, IEEE Transactions on Cognitive and Developmental Systems.

[20]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[21]  Ryota Kanai,et al.  What contributes to individual differences in brain structure? , 2014, Front. Hum. Neurosci..

[22]  Fei Wang,et al.  A deep multi-source adaptation transfer network for cross-subject electroencephalogram emotion recognition , 2021, Neural Computing and Applications.

[23]  He Li,et al.  Depersonalized Cross-Subject Vigilance Estimation with Adversarial Domain Generalization , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[24]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[25]  Bohyung Han,et al.  Learning to Optimize Domain Specific Normalization for Domain Generalization , 2019, ECCV.

[26]  D. Jude Hemanth,et al.  Brain signal based human emotion analysis by circular back propagation and Deep Kohonen Neural Networks , 2018, Comput. Electr. Eng..

[27]  Yun Luo,et al.  WGAN Domain Adaptation for EEG-Based Emotion Recognition , 2018, ICONIP.

[28]  Barbara Caputo,et al.  Boosting Domain Adaptation by Discovering Latent Domains , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Chee Peng Lim,et al.  A Review of Situation Awareness Assessment Approaches in Aviation Environments , 2018, IEEE Systems Journal.

[30]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[31]  Utku Kose,et al.  Prediction of Electroencephalogram Time Series With Electro-Search Optimization Algorithm Trained Adaptive Neuro-Fuzzy Inference System , 2019, IEEE Access.

[32]  Changde Du,et al.  Domain Adaptation for EEG Emotion Recognition Based on Latent Representation Similarity , 2020, IEEE Transactions on Cognitive and Developmental Systems.

[33]  Chin-Teng Lin,et al.  Multi-channel EEG recordings during a sustained-attention driving task , 2018, Scientific Data.

[34]  Wei Wei,et al.  Reducing Calibration Efforts in RSVP Tasks With Multi-Source Adversarial Domain Adaptation , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  Qun Wang,et al.  An experimental analysis of situation awareness for cockpit display interface evaluation based on flight simulation , 2013 .

[36]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[37]  E. Kapetanios,et al.  Feature extraction techniques for human emotion identification from face images , 2019, ICDP.