Do speech features for detecting cognitive load depend on specific languages?

Speech-based cognitive load modeling recently proposed in English have enabled objective, quantitative and unobtrusive evaluation of cognitive load without extra equipment. However, no evidence indicates that these techniques could be applied to speech data in other languages without modification. In this study, a modified Stroop Test and a Reading Span Task were conducted to collect speech data in English and Chinese respectively, from which twenty non-linguistic features were extracted to investigate whether they were language dependent. Some discriminating speech features were observed language dependent, which serves as an evidence that there is a necessity to adapt speech-based cognitive load detection techniques to diverse language contexts for a higher performance.

[1]  John H. L. Hansen,et al.  Analysis and detection of cognitive load and frustration in drivers' speech , 2010, INTERSPEECH.

[2]  P. Poupart,et al.  Automatic Speech Feature Extraction for Cognitive Load Classification , 2010 .

[3]  Fang Chen,et al.  Investigating speech features and automatic measurement of cognitive load , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[4]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[5]  Fang Chen,et al.  Measuring Cognitive Load Using Linguistic Features: Implications for Usability Evaluation and Adaptive Interaction Design , 2014, Int. J. Hum. Comput. Interact..

[6]  Andrew Sears,et al.  Automated stress detection using keystroke and linguistic features: An exploratory study , 2009, Int. J. Hum. Comput. Stud..

[7]  Christian A. Müller,et al.  Assessment of a User's Time Pressure and Cognitive Load on the Basis of Features of Speech , 2011, Resource-Adaptive Cognitive Processes.

[8]  Anthony Jameson,et al.  Interpreting symptoms of cognitive load in speech input , 1999 .

[9]  Martha E. Crosby,et al.  Assessing Cognitive Load with Physiological Sensors , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[10]  Eliathamby Ambikairajah,et al.  Formant Frequencies under Cognitive Load: Effects and Classification , 2011, EURASIP J. Adv. Signal Process..

[11]  P. Chandler,et al.  Cognitive Load Theory and the Format of Instruction , 1991 .

[12]  Fang Chen,et al.  Potential speech features for cognitive load measurement , 2007, OZCHI '07.

[13]  Eric G. Hansen,et al.  Analysis of mrate, shimmer, jitter, and F/sub 0/ contour features across stress and speaking style in the SUSAS database , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[14]  Hongbing Hu,et al.  A spectral/temporal method for robust fundamental frequency tracking. , 2008, The Journal of the Acoustical Society of America.

[15]  S. Helal,et al.  Proceedings of the 33rd Annual Hawaii International Conference on System Sciences , 2000 .

[16]  Yu Chen,et al.  Automatic cognitive load evaluation using writing features: An exploratory study , 2013 .

[17]  Estelle Campione,et al.  A large-scale multilingual study of silent pause duration , 2002, Speech Prosody 2002.

[18]  Fang Chen,et al.  Automatic cognitive load detection from speech features , 2007, OZCHI '07.

[19]  Léon J. M. Rothkrantz,et al.  Voice Stress Analysis , 2004, TSD.

[20]  Christian A. Müller,et al.  Recognizing Time Pressure and Cognitive Load on the Basis of Speech: An Experimental Study , 2001, User Modeling.

[21]  Fang Chen,et al.  Using language complexity to measure cognitive load for adaptive interaction design , 2010, IUI '10.

[22]  Wei-Ping Zhu,et al.  A New Technique for the Estimation of Jitter and Shimmer of Voiced Speech Signal , 2006, 2006 Canadian Conference on Electrical and Computer Engineering.

[23]  Fang Chen,et al.  Towards Automatic Cognitive Load Measurement from Speech Analysis , 2007, HCI.

[24]  Fang Chen,et al.  Cognitive skills learning: pen input patterns in computer-based athlete training , 2010, ICMI-MLMI '10.

[25]  Tuomo K Leino,et al.  Effect of cognitive load on articulation rate and formant frequencies during simulator flights. , 2011, The Journal of the Acoustical Society of America.

[26]  Eliathamby Ambikairajah,et al.  Glottal features for speech-based cognitive load classification , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[27]  Fang Chen,et al.  Speech-based cognitive load monitoring system , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[28]  Fang Chen,et al.  Cognitive Load Measurement from User's Linguistic Speech Features for Adaptive Interaction Design , 2009, INTERACT.

[29]  Vidhyasaharan Sethu,et al.  Investigation of spectral centroid features for cognitive load classification , 2011, Speech Commun..

[30]  Julian Togelius,et al.  Modeling Player Experience for Content Creation , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[31]  Richard P. Heitz,et al.  An automated version of the operation span task , 2005, Behavior research methods.