The Roles of Immediacy and Redundancy Relative Subjective Workload Assessment

Subjective workload assessment is one of the most frequently used tools for system evaluation. However, little is known about which factors contribute to the sensitivity, validity, and reliability of subjective workload ratings. One major classification of subjective workload assessment tools is based on relative judgments of the workload of different task conditions. The present research evaluated different approaches to relative subjective workload assessment in terms of their sensitivity to demand manipulations, concurrent validity with performance, and test-retest reliability. Results indicated that performing relative judgments retrospectively after having experienced all task conditions was superior to rating each task condition immediately after performing it. Further, redundant relative comparisons may produce more sensitive ratings than do relative comparisons to a single reference task. Overall, the results support the use of retrospective relative workload judgments as an evaluation tool.

[1]  James P. Stevens,et al.  Intermediate Statistics: A Modern Approach , 1990 .

[2]  Sally A. Seven,et al.  Selecting Performance Measures: "Objective" versus "Subjective" Measurement , 1992 .

[3]  Mansour Rahimi,et al.  Evaluation of 16 Measures of Mental Workload using a Simulated Flight Task Emphasizing Mediational Activity , 1985 .

[4]  A. O. Dick,et al.  Operator Workload: Comprehensive Review and Evaluation of Operator Workload Methodologies , 1989 .

[5]  Pamela S. Tsang,et al.  Time-Sharing Visual and Auditory Tracking Tasks , 1987 .

[6]  Pamela S. Tsang,et al.  Absolute Magnitude Estimation and Relative Judgement Approaches to Subjective Workload Assessment , 1987 .

[7]  F. Thomas Eggemeier,et al.  Workload assessment methodology. , 1986 .

[8]  W H Levison,et al.  A Methodology for Quantifying the Effects of Aging on Perceptual-Motor Capability , 1981, Human factors.

[9]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[10]  James C. Schueren,et al.  Using the Subjective Workload Dominance (SWORD) Technique for Projective Workload Assessment , 1991 .

[11]  D. H. Dodd,et al.  Computational procedures for estimating magnitude of effect for some analysis of variance designs. , 1973 .

[12]  Thomas E. Nygren,et al.  The Subjective Workload Assessment Technique: A Scaling Procedure for Measuring Mental Workload , 1988 .

[13]  William B. Rouse,et al.  Advances in man-machine systems research : a research annual , 1984 .

[14]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[15]  D. Gopher,et al.  On the Psychophysics of Workload: Why Bother with Subjective Measures? , 1984 .

[16]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[17]  M. A. Vidulich,et al.  The Use of Judgment Matrices in Subjective Workload Assessment: The Subjective Workload Dominance (SWORD) Technique , 1989 .

[18]  J. C. Byers,et al.  Comparison of Four Subjective Workload Rating Scales , 1992 .

[19]  N. Clayton Silver,et al.  A FORTRAN 77 program for averaging correlation coefficients , 1989 .