Vocal biomarkers to discriminate cognitive load in a working memory task

Early, accurate detection of cognitive load can help reduce risk of accidents and injuries, and inform intervention and rehabilitation in recovery. Thus, simple noninvasive biomarkers are desired for determining cognitive load under cognitively complex tasks. In this study, a novel set of vocal biomarkers are introduced for detecting different cognitive load conditions. Our vocal biomarkers use phonemeand pseudosyllable-based measures, and articulatory and source coordination derived from cross-correlation and temporal coherence of formant and creakiness measures. A ~2-hour protocol was designed to induce cognitive load by stressing auditory working memory. This was done by repeatedly requiring the subject to recall a sentence while holding a number of digits in memory. We demonstrate the power of our speech features to discriminate between high and low load conditions. Using a database consisting of audio from 13 subjects, we apply classification models of cognitive load, showing a ~7% detection equal-error rate from features derived from 40 sentence utterances (~4 minutes of audio).

[1]  Fang Chen,et al.  Towards Automatic Cognitive Load Measurement from Speech Analysis , 2007, HCI.

[2]  Thomas F. Quatieri,et al.  Vocal and Facial Biomarkers of Depression based on Motor Incoordination and Timing , 2014, AVEC '14.

[3]  Thomas F. Quatieri,et al.  Articulatory dynamics and coordination in classifying cognitive change with preclinical mTBI , 2014, INTERSPEECH.

[4]  D B Pisoni,et al.  Effects of cognitive workload on speech production: acoustic analyses and perceptual consequences. , 1993, The Journal of the Acoustical Society of America.

[5]  P. Wolfe,et al.  Kalman-based autoregressive moving average modeling and inference for formant and antiformant tracking a ) , 2011 .

[6]  Jody Kreiman,et al.  Toward a taxonomy of nonmodal phonation , 2001, J. Phonetics.

[7]  James R. Williamson,et al.  Seizure prediction using EEG spatiotemporal correlation structure , 2012, Epilepsy & Behavior.

[8]  Thomas F. Quatieri,et al.  Prediction of cognitive performance in an animal fluency task based on rate and articulatory markers , 2014, INTERSPEECH.

[9]  John H. L. Hansen,et al.  Analysis and detection of cognitive load and frustration in drivers' speech , 2010, INTERSPEECH.

[10]  D. Pisoni,et al.  PRESTO: Perceptually robust English sentence test: Open‐set—Design, philosophy, and preliminary findings. , 2010 .

[11]  P. Boersma.  ACCURATE SHORT-TERM ANALYSIS OF THE FUNDAMENTAL FREQUENCY AND THE HARMONICS-TO-NOISE RATIO OF A SAMPLED SOUND , 1993 .

[12]  Thomas F. Quatieri,et al.  Phonologically-based biomarkers for major depressive disorder , 2011, EURASIP J. Adv. Signal Process..

[13]  Eliathamby Ambikairajah,et al.  A Study of Voice Source and Vocal Tract Filter Based Features in Cognitive Load Classification , 2010, 2010 20th International Conference on Pattern Recognition.

[14]  Fang Chen,et al.  Speech-based cognitive load monitoring system , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  Richard Wright,et al.  A new method for eliciting three speaking styles in the laboratory , 2008, Speech Commun..

[17]  John Kane,et al.  Improved automatic detection of creak , 2013, Comput. Speech Lang..

[18]  Hiroshi Ishiguro,et al.  A Method for Automatic Detection of Vocal Fry , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  Jean-Luc Rouas.  Automatic Prosodic Variations Modeling for Language and Dialect Discrimination , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[20]  H. Levitt.  Transformed up-down methods in psychoacoustics. , 1971, The Journal of the Acoustical Society of America.

[21]  Michael Frankfurter,et al.  Applied Longitudinal Data Analysis Modeling Change And Event Occurrence , 2016 .

[22]  Timothy J. Hazen,et al.  A comparison of query-by-example methods for spoken term detection , 2009, INTERSPEECH.

[23]  Eliathamby Ambikairajah,et al.  A non-uniform subband approach to speech-based cognitive load classification , 2009, 2009 7th International Conference on Information, Communications and Signal Processing (ICICS).

[24]  Fang Chen,et al.  Think before you talk: an empirical study of relationship between speech pauses and cognitive load , 2008, OZCHI.