Workload and the use of automatic speech recognition: The effects of time and resource demands

Abstract Previous research has indicated that workload can have an adverse effect on the use of speech recognition systems. In this paper, the relationship between workload and speech is discussed, and two studies are reported. In the first study, time-stress is considered. In the second study, dual-task performance is considered. Both studies show workload to significantly reduce recognition accuracy and user performance. The nature of the impairment is shown to differ between individuals and types of workload. Furthermore, it appears that workload affects the selection of words to use, the articulation of the words and the relationship between speaking to ASR and performing other tasks. It is proposed that speaking to ASR is, in itself, demanding and that as workload increases so the ability to perform the task within the limits required by ASR suffers.

[1]  A. W. Ellis.  Normality and pathology in cognitive functions , 1982 .

[2]  C. A. Simpson.  Speech variability effects on recognition accuracy associated with concurrent task performance by pilots , 1986 .

[3]  Christopher D. Wickens,et al.  The Effects of Stress on Pilot Judgment in a MIDIS Simulator , 1993 .

[4]  G. R. Doddington,et al.  Computers: Speech recognition: Turning theory to practice: New ICs have brought the requisite computer power to speech technology; an evaluation of equipment shows where it stands today , 1981, IEEE Spectrum.

[5]  Gary K. Poock,et al.  Effect of task duration on voice recognition system performance , 1981 .

[6]  Steven D. Harris,et al.  A system for the assessment of human performance in concurrent verbal and manual control tasks , 1978 .

[7]  Chris Baber,et al.  Feedback Requirements for Automatic Speech Recognition in the Process Control Room , 1992, Int. J. Man Mach. Stud..

[8]  Christopher Baber,et al.  Speech technology in control room systems : a human factors perspective , 1991 .

[9]  Mansour Rahimi,et al.  Evaluation of 16 Measures of Mental Workload using a Simulated Flight Task Emphasizing Mediational Activity , 1985 .

[10]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[11]  Daniel G Bobrow,et al.  On data-limited and resource-limited processes , 1975, Cognitive Psychology.

[12]  George E. Cooper,et al.  The use of pilot rating in the evaluation of aircraft handling qualities , 1969 .

[13]  Jan Noyes,et al.  Sources of Human Error in Data Entry Tasks Using Speech Input , 1990 .

[14]  Chris Baber,et al.  Instructions and demonstration as media for training new users of Automatic Speech Recognition Devices , 1990 .

[15]  Christopher D. Wickens,et al.  The Structure of Attentional Resources , 1980 .

[16]  P. Fitts,et al.  S-R compatibility: spatial characteristics of stimulus and response codes. , 1953, Journal of experimental psychology.

[17]  Jm Noyes,et al.  Automatic speech recognition systems: Effects of environmental stressors , 1995 .

[18]  C. A. Simpson,et al.  System Design for Speech Recognition and Generation , 1985, Human factors.

[19]  Jan Noyes,et al.  Automatic Speech Recognition in Adverse Environments , 1996, Hum. Factors.

[20]  G S Dell,et al.  A spreading-activation theory of retrieval in sentence production. , 1986, Psychological review.

[21]  C D Wickens,et al.  Principles of S-C-R Compatibility with Spatial and Verbal Tasks: The Role of Display-Control Location and Voice-Interactive Display-Control Interfacing , 1984, Human factors.

[22]  D. Bouwhuis,et al.  The Structure of Multimodal Dialogue , 1989 .

[23]  J. C. Spearandio.  Variation of operator's strategies and regulating effects on workload. , 1971, Ergonomics.

[24]  Christopher D. Wickens,et al.  Workload Assessment and Prediction , 1990 .

[25]  Mansour Rahimi,et al.  Techniques in mental workload assessment. , 1995 .

[26]  K. Hapeshi,et al.  Concurrent manual tracking and speaking: implications for automatic speech recognition , 1989 .

[27]  C D Wickens,et al.  Compatibility and Resource Competition between Modalities of Input, Central Processing, and Output , 1983, Human factors.

[28]  S. T. Klapp,et al.  Temporal compatibility in dual motor tasks II: Simultaneous articulation and hand movements , 1981, Memory & cognition.

[29]  D. G. MacKay.  The Problems of Flexibility, Fluency, and Speed-Accuracy Trade-Off in Skilled Behavior. , 1982 .

[30]  G. Hitch.  The role of short-term working memory in mental arithmetic , 1978, Cognitive Psychology.

[31]  E. Poulton,et al.  MEASURING THE SPARE ‘MENTAL CAPACITY ’OF CAR DRIVERS BY A SUBSIDIARY TASK , 1961 .

[32]  J. C. Sperandio,et al.  Variation of Operator's Strategies and Regulating Effects on Workload , 1971 .

[33]  Charlotte Linde,et al.  Field Study of Communication and Workload in Police Helicopters: Implications for AI Cockpit Design , 1988 .

[34]  C D Wickens,et al.  Codes and Modalities in Multiple Resources: A Success and a Qualification , 1988, Human factors.

[35]  D. Damos.  Multiple-task performance , 2020 .

[36]  A. Sanders.  Towards a model of stress and human performance. , 1983, Acta psychologica.