Automated and objective action coding of facial expressions in patients with acute facial palsy

Aim of the present observational single center study was to objectively assess facial function in patients with idiopathic facial palsy with a new computer-based system that automatically recognizes action units (AUs) defined by the Facial Action Coding System (FACS). Still photographs using posed facial expressions of 28 healthy subjects and of 299 patients with acute facial palsy were automatically analyzed for bilateral AU expression profiles. All palsies were graded with the House–Brackmann (HB) grading system and with the Stennert Index (SI). Changes of the AU profiles during follow-up were analyzed for 77 patients. The initial HB grading of all patients was 3.3 ± 1.2. SI at rest was 1.86 ± 1.3 and during motion 3.79 ± 4.3. Healthy subjects showed a significant AU asymmetry score of 21 ± 11 % and there was no significant difference to patients (p = 0.128). At initial examination of patients, the number of activated AUs was significantly lower on the paralyzed side than on the healthy side (p < 0.0001). The final examination for patients took place 4 ± 6 months post baseline. The number of activated AUs and the ratio between affected and healthy side increased significantly between baseline and final examination (both p < 0.0001). The asymmetry score decreased between baseline and final examination (p < 0.0001). The number of activated AUs on the healthy side did not change significantly (p = 0.779). Radical rethinking in facial grading is worthwhile: automated FACS delivers fast and objective global and regional data on facial motor function for use in clinical routine and clinical trials.

[1]  John J. Soraghan,et al.  Quantitative Analysis of Facial Paralysis Using Local Binary Patterns in Biomedical Videos , 2009, IEEE Transactions on Biomedical Engineering.

[2]  K. Schmidt,et al.  Muscle Activity in the Partially Paralyzed Face After Placement of a Fascial Sling: A Preliminary Report , 2005, Annals of plastic surgery.

[3]  Naoaki Yanagihara,et al.  Yanagihara Facial Nerve Grading System as a Prognostic Tool in Bell’s Palsy , 2014, Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology.

[4]  Videomimicography: the standards of normal revised. , 2003, Archives of otolaryngology--head & neck surgery.

[5]  T. Hadlock,et al.  Toward a universal, automated facial measurement tool in facial reanimation. , 2012, Archives of facial plastic surgery.

[6]  U. Fisch,et al.  Objective scaling of facial nerve function based on area analysis (OSCAR) , 1998 .

[7]  P. Ekman.  Facial expression and emotion. , 1993, The American psychologist.

[8]  James T. Heaton,et al.  Determining the threshold for asymmetry detection in facial expressions , 2014, The Laryngoscope.

[9]  R. Croft.  A prospective cohort study comes of age. , 2000, Leprosy review.

[10]  James T Heaton,et al.  Detection and perceptual impact of side-to-side facial movement asymmetry. , 2013, JAMA facial plastic surgery.

[11]  A. Verma.  Early Treatment with Prednisolone or Acyclovir in Bell's Palsy , 2008 .

[12]  J. Cohn,et al.  Automated Tracking of Facial Features in Patients with Facial Neuromuscular Dysfunction , 2001, Plastic and reconstructive surgery.

[13]  J. Borod,et al.  Neurobiological and cultural aspects of facial asymmetry. , 1990, Journal of communication disorders.

[14]  T. Hadlock,et al.  A Comparison of Facial Nerve Grading Systems , 2012, Annals of plastic surgery.

[15]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Tsuhan Chen,et al.  The painful face - Pain expression recognition using active appearance models , 2009, Image Vis. Comput..

[17]  Orlando Guntinas-Lichius,et al.  Facial muscle activation patterns in healthy male humans: A multi-channel surface EMG study , 2010, Journal of Neuroscience Methods.

[18]  G. Gronseth,et al.  Evidence-based guideline update: Steroids and antivirals for Bell palsy , 2012, Neurology.

[19]  Naohito Hato,et al.  Objective assessment of the severity of unilateral facial palsy using OKAO Vision® facial image analysis software , 2012, Acta oto-laryngologica.

[20]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[21]  A. Jenny,et al.  Development of a New Documentation System for Facial Movements as a Basis for the International Registry for Neuromuscular Reconstruction in the Face , 1994, Plastic and reconstructive surgery.

[22]  J Byers,et al.  Computerized quantitative dynamic analysis of facial motion in the paralyzed and synkinetic face. , 1992, The American journal of otology.

[23]  J. Reichenbach,et al.  Quantitative Magnetic Resonance Imaging Volumetry of Facial Muscles in Healthy Patients with Facial Palsy , 2014, Plastic and reconstructive surgery. Global open.

[24]  V F Ferrario,et al.  Anatomy of emotion: a 3D study of facial mimicry. , 2007, European journal of histochemistry : EJH.

[25]  S. Schwartz,et al.  Clinical Practice Guideline: Bell’s Palsy , 2013 .

[26]  Maria W G Nijhuis-van der Sanden,et al.  Frequency and Location of Synkineses in Patients With Peripheral Facial Nerve Paresis , 2010, Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology.

[27]  J. W. House,et al.  Facial Nerve Grading System , 1985, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[28]  Jeffrey F. Cohn,et al.  Automated Facial Image Analysis: Detecting Improvement in Abnormal Facial Movement After Treatment With Botulinum Toxin A , 2007, Annals of plastic surgery.

[29]  M. Engström,et al.  The effect of study design and analysis methods on recovery rates in Bell's palsy , 2009, The Laryngoscope.

[30]  P. Ekman,et al.  The asymmetry of facial actions is inconsistent with models of hemispheric specialization. , 1985, Psychophysiology.

[31]  Hamid R. Djalilian,et al.  Facial Nerve Grading System 2.0 , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[32]  C. Nduka,et al.  Validation of a new photogrammetric technique to monitor the treatment effect of Botulinum toxin in synkinesis , 2013, Eye.

[33]  O. Guntinas-Lichius,et al.  Reference values for dynamic facial muscle ultrasonography in adults , 2014, Muscle & nerve.

[34]  Jessie M VanSwearingen,et al.  Puckering and Blowing Facial Expressions in People With Facial Movement Disorders , 2008, Physical Therapy.

[35]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[36]  John J Soraghan,et al.  Objective method of assessing and presenting the House‐Brackmann and regional grades of facial palsy by production of a facogram , 2009, Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology.

[37]  J. Nedzelski,et al.  Development of a sensitive clinical facial grading system. , 1996, European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery.

[38]  M. Kanerva,et al.  Prednisolone and valaciclovir in Bell's palsy: a randomised, double-blind, placebo-controlled, multicentre trial , 2008, The Lancet Neurology.

[39]  E. Stennert,et al.  [An index for paresis and defective healing--an easily applied method for objectively determining therapeutic results in facial paresis (author's transl)]. , 1977, HNO.

[40]  J. Cohn,et al.  Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. , 1999, Psychophysiology.

[41]  O. Witte,et al.  Prognostication of recovery time after acute peripheral facial palsy: a prospective cohort study , 2013, BMJ Open.

[42]  R. Gur,et al.  Automated Facial Action Coding System for dynamic analysis of facial expressions in neuropsychiatric disorders , 2011, Journal of Neuroscience Methods.

[43]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[44]  C. Linstrom.  Objective Facial Motion Analysis in Patients With Facial Nerve Dysfunction , 2002, The Laryngoscope.

[45]  O. Guntinas-Lichius,et al.  Prognostic Value of Electroneurography and Electromyography in Facial Palsy , 2008, The Laryngoscope.

[46]  R. Gibson,et al.  What the Face Reveals , 2002 .

[47]  Ali Sehizadeh,et al.  Computerized Objective Measurement of Facial Motion: Normal Variation and Test-Retest Reliability , 2010, Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology.

[48]  K. Heilman,et al.  Digitizing the moving face during dynamic displays of emotion , 2000, Neuropsychologia.