Multiclass voluntary facial expression classification based on Filter Bank Common Spatial Pattern

This paper investigates the classification of voluntary facial expressions from electroencephalogram (EEG) and electromyogram (EMG) signals using the Filter Bank Common Spatial Pattern (FBCSP) algorithm. The FBCSP algorithm is an autonomous and effective machine learning approach for classifying two classes of EEG measurements in motor imagery-based Brain Computer Interface (BCI). However, the problem of facial expression recognition typically involves more than just two classes of measurements. Hence, this paper proposes an extension of FBCSP to the multiclass paradigm using a decision threshold-based classifier for classifying facial expressions from EEG and EMG measurements. A study is conducted using the proposed Multiclass FBCSP on 4 subjects who performed 6 different facial expressions. The results show that the Multiclass FBCSP is effective in classifying multiple facial expressions from the EEG and EMG measurements.

[1]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[2]  Klaus-Robert Müller,et al.  The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects , 2007, NeuroImage.

[3]  Ronald Leif Steenerson,et al.  The Effectiveness of Neuromuscular Facial Retraining Combined with Electromyography in Facial Paralysis Rehabilitation , 2003, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[4]  N. O'dwyer,et al.  Physiotherapy Rehabilitation of the Smile after Long-Term Facial Nerve Palsy using Video Self-Modeling and Implementation Intentions , 2006, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[5]  Patrick J Byrne,et al.  Importance of facial expression in facial nerve rehabilitation , 2004, Current opinion in otolaryngology & head and neck surgery.

[6]  R. Morecraft,et al.  The Motor Cortex and Facial Expression:: New Insights From Neuroscience , 2004, The neurologist.

[7]  Anton Nijholt,et al.  BrainGain: BCI for HCI and Games , 2008 .

[8]  Klaus-Robert Müller,et al.  Boosting bit rates in noninvasive EEG single-trial classifications by feature combination and multiclass paradigms , 2004, IEEE Transactions on Biomedical Engineering.

[9]  W. Penfield.  The Cerebral Cortex of Man , 1950 .

[10]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[11]  J. Russell,et al.  Facial and vocal expressions of emotion. , 2003, Annual review of psychology.

[12]  Jeonghun Ku,et al.  The neural mechanism of imagining facial affective expression , 2007, Brain Research.

[13]  Alan J. Dix,et al.  Affective Videogames and Modes of Affective Gaming: Assist Me, Challenge Me, Emote Me (ACE) , 2005, DiGRA Conference.

[14]  Kai Keng Ang,et al.  Rough set-based neuro-fuzzy system , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[15]  Hugo D Critchley,et al.  Imitating expressions: emotion-specific neural substrates in facial mimicry. , 2006, Social cognitive and affective neuroscience.

[16]  G. Pfurtscheller,et al.  Designing optimal spatial filters for single-trial EEG classification in a movement task , 1999, Clinical Neurophysiology.

[17]  U. Dimberg,et al.  Rapid facial reactions to emotional facial expressions. , 1998, Scandinavian journal of psychology.

[18]  L. Rothkrantz,et al.  Toward an affect-sensitive multimodal human-computer interaction , 2003, Proc. IEEE.

[19]  J. Mazziotta,et al.  Neural mechanisms of empathy in humans: A relay from neural systems for imitation to limbic areas , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[20]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[21]  L. Cohen,et al.  Brain–computer interfaces: communication and restoration of movement in paralysis , 2007, The Journal of physiology.

[22]  John J. B. Allen,et al.  Voluntary facial expression and hemispheric asymmetry over the frontal cortex. , 2001, Psychophysiology.

[23]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).