A speech database for stress monitoring in the cockpit

This article presents a new database of speech produced under cognitive load for the purpose of non-invasive psychological stress monitoring. The voices and the heart rates of eight airline pilots were recorded while completing an advanced flight simulation programme in a level D full flight simulator. Focusing on real-world applicability, the experiments were designed to yield the maximum degree of realism possible. Evaluation of physiological reference measures in pilots demonstrates that several heart rate variability parameters correlate with speech features derived from the recorded data. The article discusses the evolution of speech monitoring in aviation and proposes that application-orientated research methods can be useful in designing a system for real-world monitoring.

[1]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[2]  Chris Baber,et al.  Towards a definition and working model of stress and its effects on speech , 1996, Speech Commun..

[3]  Bin Yang,et al.  Robust Estimation of Voice Quality Parameters Under Realworld Disturbances , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[4]  John H. L. Hansen,et al.  Getting started with SUSAS: a speech under simulated and actual stress database , 1997, EUROSPEECH.

[5]  Roddy Cowie,et al.  Emotional speech: Towards a new generation of databases , 2003, Speech Commun..

[6]  John H. L. Hansen,et al.  UTDrive: Emotion and Cognitive Load Classification for In-Vehicle Scenarios , 2011 .

[7]  P van Leeuwen,et al.  Musical rhythms in heart period dynamics: a cross-cultural and interdisciplinary approach to cardiac rhythms. , 1999, American journal of physiology. Heart and circulatory physiology.

[8]  Björn W. Schuller,et al.  Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge , 2011, Speech Commun..

[9]  John H. L. Hansen,et al.  CU-Move: Advanced In-Vehicle Speech Systems for Route Navigation , 2005 .

[10]  K. Stevens,et al.  Classification of glottal vibration from acoustic measurements , 1995 .

[11]  F. T. Eggemeier,et al.  Recommendations for Mental Workload Measurement in a Test and Evaluation Environment , 1993 .

[12]  Constantine Kotropoulos,et al.  Emotional speech recognition: Resources, features, and methods , 2006, Speech Commun..

[13]  I. Altman,et al.  Social and psychological factors in stress. , 1970 .

[14]  John H. L. Hansen,et al.  Speech Under Stress: Analysis, Modeling and Recognition , 2007, Speaker Classification.

[15]  Johannes Luig,et al.  A Robust Feature Set for Classification of Speech under Stress , 2009 .

[16]  Marek Correspondence,et al.  Guidelines Heart rate variability Standards of measurement , physiological interpretation , and clinical use Task Force of The European Society of Cardiology and The North American Society of Pacing and Electrophysiology ( Membership of the Task Force , 2005 .

[17]  Eero Väyrynen,et al.  Effect of cognitive load on speech prosody in aviation: Evidence from military simulator flights. , 2011, Applied ergonomics.

[18]  Johannes Luig.  Speech Features for Stress Recognition , 2010 .

[19]  Graham Wilson,et al.  Psychophysiological assessment of workload in multi-task environments , 2020 .

[20]  J.H.L. Hansen,et al.  UT-Scope: Speech under Lombard Effect and Cognitive Stress , 2007, 2007 IEEE Aerospace Conference.