Using Time Windows to Evaluate Operator Performance

A proper understanding of human performance characteristics is a prerequisite for designers of complex systems. Although human factors texts provide some insights into basic performance issues, the emergence of highly automated computing systems have fundamentally altered the way humans work. The purpose of this article is to present a research approach to quantify and analyze human performance within a complex, time-critical system. The approach is centered on a measurement construct, called a time window, which enables a functional relation between constraints on operator activities and time availability. A blackboard model is developed as the mechanism to generate, maintain, and complete time windows. Moreover, an object-oriented methodology is described that implements the blackboard model within a realistic task context. To demonstrate the utility of time windows, an existing implementation in a real-time human-in-the-loop simulation is also described. Using time window outcomes, some cursory analyse...

[1]  John A. Swets,et al.  Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers , 1996 .

[2]  Raanan Lipshitz,et al.  Why classical decision theory is an inappropriate standard for evaluating and aiding most human decision making. , 1993 .

[3]  Christopher D. Wickens,et al.  Strategic workload management and decision biases in aviation , 1994 .

[4]  H. Penny Nii,et al.  Blackboard systems: the blackboard model of problem solving and the evolution of blackboard architectures , 1995 .

[5]  Dianne E. Howie,et al.  Measures of operator performance in complex, dynamic microworlds: advancing the state of the art. , 1998, Ergonomics.

[6]  Mladen Stanojević,et al.  Blackboard metaphor in tactical decision making , 1992 .

[7]  J D Caldeira.  Parametric Assumptions of Some "Nonparametric" Measures of Sensory Efficiency , 1980, Human factors.

[8]  Berndt Brehmer,et al.  Distributed decision making. Cognitive models for cooperative work , 1991 .

[9]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[10]  K J Vicente,et al.  On putting the cart before the horse: Taking perception seriously in unified theories of cognition , 1992, Behavioral and Brain Sciences.

[11]  Penelope M Sanderson,et al.  State-space and verbal protocol methods for studying the human operator in process control. , 1989, Ergonomics.

[12]  Christine M. Mitchell,et al.  A methodology for human-machine systems research: knowledge engineering, modeling, and simulation , 1995, IEEE Trans. Syst. Man Cybern..

[13]  A. Kirlik.  Requirements for psychological models to support design: Towards ecological task analysis , 1991 .

[14]  Lucy Suchman.  Plans and situated actions: the problem of human-machine communication , 1987 .

[15]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[16]  Ravi S. Adapathya,et al.  Strategic Behavior, Workload, and Performance in Task Scheduling , 1991 .

[17]  Kim J. Vicente,et al.  COMMENTARY Heeding the Legacy of Meister, Brunswik, & Gibson: Toward a Broader View of Human Factors Research , 1997, Hum. Factors.

[18]  A Craig,et al.  Nonparametric Measures of Sensory Efficiency for Sustained Monitoring Tasks , 1979, Human factors.

[19]  Christine M. Mitchell,et al.  OFMspert: inference of operator intentions in supervisory control using a blackboard architecture , 1988, IEEE Trans. Syst. Man Cybern..

[20]  Raymond B. Cattell,et al.  Handbook of multivariate experimental psychology , 1968 .

[21]  I V Laudeman,et al.  Quantitative measurement of observed workload in the analysis of aircrew performance. , 1995, The International journal of aviation psychology.

[22]  Lucy A. Suchman,et al.  Plans and Situated Actions: The Problem of Human-Machine Communication (Learning in Doing: Social, , 1987 .

[23]  Victor R. Lesser,et al.  The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty , 1980, CSUR.

[24]  Christopher D. Wickens,et al.  The Effects of Participatory Mode and Task Workload on the Detection of Dynamic System Failures , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[25]  R. Hutton,et al.  Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands. , 1998, Ergonomics.

[26]  J. Greeno.  Gibson's affordances. , 1994, Psychological review.

[27]  H. Penny Nii,et al.  Blackboard Systems, Part One: The Blackboard Model of Problem Solving and the Evolution of Blackboard Architectures , 1986, AI Mag..

[28]  C. Wickens.  Engineering psychology and human performance, 2nd ed. , 1992 .

[29]  Ivar Jacobson,et al.  The Unified Modeling Language User Guide , 1998, J. Database Manag..

[30]  K. J. Vicente,et al.  Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work , 1999 .

[31]  Kenneth R. Hammond.  Generalization in Operational Contexts: What Does It Mean? Can It be Done? , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[32]  Hojjat Adeli,et al.  An integrated computing environment for solution of complex engineering problems using the object-oriented programming paradigm and a blackboard architecture , 1995 .

[33]  D. Gabbay,et al.  Temporal Logic Mathematical Foundations and Computational Aspects , 1994 .

[34]  James G. Greeno,et al.  Situativity and Symbols: Response to Vera and Simon , 1993, Cogn. Sci..

[35]  J. Greeno.  THE SITUATIVITY OF KNOWING, LEARNING, AND RESEARCH , 1998 .

[36]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .