Noninvasive estimation of cognitive status in mild traumatic brain injury using speech production and facial expression

In civilian and military populations, there is strong need for objective, noninvasive technologies in monitoring cognitive status associated with mild traumatic brain injury (mTBI). Previous work has shown that monitoring technologies based upon motor control characteristics in speech production provide sensitive indication of cognitive impairments resulting from neurotraumatic injury. Here, this approach is generalized to biomarkers from both speech and facial expression during speaking and preliminary analysis is presented for noninvasively estimating cognitive status associated with mTBI. High-quality audio and video recordings were obtained from 20 subjects in conjunction with cognitive task performance, measured as Processing Speed Index (PSI). Of the 20 subjects, five had a documented history of mild traumatic brain injury, and 15 were control subjects. Models were trained on the control subjects to estimate PSI, and then used to estimate PSI scores from the mTBI cases. Pearson's correlation coefficient between the estimates and the recorded PSI scores revealed r = 0.98 for the speech features and r = 0.92 for the facial features. These results demonstrate the promise of cognitive assessment technologies based on motor timing and coordination underlying vocal and facial expression during speaking in the context of mTBI.

[1]  J. Ghajar,et al.  Predictive visual tracking: specificity in mild traumatic brain injury and sleep deprivation. , 2014, Military medicine.

[2]  Thomas F. Quatieri,et al.  Detecting Depression using Vocal, Facial and Semantic Communication Cues , 2016, AVEC@ACM Multimedia.

[3]  A Alonte,et al.  Relationship of Scores on the Berg Balance Scale to Results of the Limits of Stability Portion of the Smart BalanceMaster® Suite of Tests , 1995 .

[4]  Thomas F. Quatieri,et al.  Prediction of cognitive performance in an animal fluency task based on rate and articulatory markers , 2014, INTERSPEECH.

[5]  J. Langlois,et al.  Traumatic brain injury in the United States; emergency department visits, hospitalizations, and deaths , 2006 .

[6]  Satrajit S. Ghosh,et al.  Segment-dependent dynamics in predicting parkinson's disease , 2015, INTERSPEECH.

[7]  Thomas F. Quatieri,et al.  Vocal biomarkers to discriminate cognitive load in a working memory task , 2015, INTERSPEECH.

[8]  John DeLuca,et al.  Cognitive reserve protects against cognitive dysfunction in multiple sclerosis , 2009, Journal of clinical and experimental neuropsychology.

[9]  William C Walker,et al.  Objectively assessing balance deficits after TBI: Role of computerized posturography. , 2007, Journal of rehabilitation research and development.

[10]  James R. Williamson,et al.  Multi-modal Biomarkers to Discriminate Cognitive State , 2015 .

[11]  D Kushner,et al.  Mild traumatic brain injury: toward understanding manifestations and treatment. , 1998, Archives of internal medicine.

[12]  Thomas F. Quatieri,et al.  Vocal and Facial Biomarkers of Depression based on Motor Incoordination and Timing , 2014, AVEC '14.

[13]  Thomas F. Quatieri,et al.  Articulatory dynamics and coordination in classifying cognitive change with preclinical mTBI , 2014, INTERSPEECH.

[14]  P. Wolfe,et al.  Kalman-based autoregressive moving average modeling and inference for formant and antiformant tracking a ) , 2011 .

[15]  Christian Poellabauer,et al.  Challenges in Concussion Detection Using Vocal Acoustic Biomarkers , 2015, IEEE Access.

[16]  Erin D Bigler,et al.  Premorbid Intellectual Functioning, Education, and Brain Size in Traumatic Brain Injury: An Investigation of the Cognitive Reserve Hypothesis , 2003, Applied neuropsychology.

[17]  E Brooke Lerner,et al.  Frequency of factors that complicate the identification of mild traumatic brain injury in level I trauma center patients , 2015, Concussion.

[18]  D. Kondziolka,et al.  Eye Tracking Detects Disconjugate Eye Movements Associated with Structural Traumatic Brain Injury and Concussion , 2015, Journal of neurotrauma.

[19]  M. Wald,et al.  Traumatic brain injury in the United States: national estimates of prevalence and incidence, 2002–2006 , 2010, Injury Prevention.

[20]  L A Fos,et al.  Paced Visual Serial Addition Test: An Alternative Measure of Information Processing Speed , 1999, Applied neuropsychology.

[21]  M. Wald,et al.  Traumatic brain injury in the United States; emergency department visits, hospitalizations, and deaths, 2002-2006 , 2010 .

[22]  Yaakov Stern,et al.  Exploring the neural basis of cognitive reserve in aging. , 2012, Biochimica et biophysica acta.

[23]  Gwen Littlewort,et al.  The computer expression recognition toolbox (CERT) , 2011, Face and Gesture 2011.

[24]  Maureen Dennis,et al.  Brain reserve capacity, cognitive reserve capacity, and age-based functional plasticity after congenital and acquired brain injury in children , 2013 .

[25]  Y. Stern.  What is cognitive reserve? Theory and research application of the reserve concept , 2002, Journal of the International Neuropsychological Society.

[26]  Thomas F. Quatieri,et al.  Vocal biomarkers of depression based on motor incoordination , 2013, AVEC@ACM Multimedia.

[27]  Thomas F. Quatieri,et al.  On the relative importance of vocal source, system, and prosody in human depression , 2013, 2013 IEEE International Conference on Body Sensor Networks.