Integrating Verification and Validation with the Design of Complex Man-Machine Systems

Human factors evaluation is a critical process for ensuring the success of man-machine systems. Meister (1987a, b) describes more traditional methods for systems design, development and testing, and system effectiveness testing. However, recent advances in man-machine interface system (M-MIS) technology (i.e., sensors, processors, displays and controls, and the overall control room configuration) and the man-machine systems that they control have created the need for more sophisticated approaches to evaluation. In particular, special attention is needed to address (a) cognitive issues related to how operators develop and maintain awareness of the state of the controlled system and (b) crew coordination issues related to the ways that individuals interact with each other and the rest of the man-machine system to accomplish operational goals. These issues may be difficult and expensive to resolve if evaluation is left until late in the design process and tested using a production prototype or a full-scope, full-scale simulator. This paper proposes an alternative approach in which human performance issues are evaluated earlier in the design process using lower fidelity testbeds (e.g., part-task simulators) in addition to more traditional means. This approach requires a systematic evaluation framework for defining issues and specifying required attributes of testbed fidelity.

[1]  E.M. Roth,et al.  Human factors evaluation issues for advanced control rooms: a research agenda , 1992, Conference Record for 1992 Fifth Conference on Human Factors and Power Plants.

[2]  Edwin Hutchins,et al.  The technology of team navigation , 1990 .

[3]  David C. Nagel,et al.  Human factors in aviation , 1988 .

[4]  Emilie M. Roth,et al.  Navigating through Large Display Networks in Dynamic Control Applications , 1990 .

[5]  Randall J. Mumaw,et al.  Evaluation Issues for Computer-Based Control Rooms , 1991 .

[6]  L. P. Goodstein,et al.  Information Technology and Work , 1988 .

[7]  Robert L. Helmreich,et al.  Group interaction and flight crew performance , 1988 .

[8]  John M. O'Hara,et al.  Advanced Control Room Evaluation: General Approach and Rationale , 1991 .

[9]  Earl L. Wiener,et al.  Human factors of advanced technology (glass cockpit) transport aircraft , 1989 .

[10]  James Wilson,et al.  Rapid Prototyping for User Interface Design , 1988 .

[11]  E.M. Roth,et al.  An analytic technique for framing control room evaluation issues , 1991, Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics.

[12]  David Woods,et al.  Significance Messages: An Integrated Display Concept , 1988 .

[13]  Austin Henderson,et al.  Rooms: the use of multiple virtual workspaces to reduce space contention in a window-based graphical user interface , 1986, TOGS.

[14]  William F. Stubler.  When is a Naval Outer Air Battle like a Nuclear Power Plant? , 1990 .

[15]  David D. Woods,et al.  Visual Momentum: A Concept to Improve the Cognitive Coupling of Person and Computer , 1984, Int. J. Man Mach. Stud..

[16]  Leo Beltracchi.  A Direct Manipulation Interface for Water-Based Rankine Cycle Heat Engines , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[17]  Mica R. Endsley,et al.  Design and Evaluation for Situation Awareness Enhancement , 1988 .

[18]  Gavriel Salvendy,et al.  Handbook of human factors. , 1987 .

[19]  Jens Rasmussen,et al.  Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering , 1986 .

[20]  T A Stinnett.  HUMAN FACTORS IN THE SUPER COCKPIT , 1989 .

[21]  David Woods,et al.  Human factors challenges in process control: The case of nuclear power plants. , 1987 .

[22]  H. S. Osborne,et al.  The international electrotechnical commission , 1953, Electrical Engineering.

[23]  D. Woods.  Coping with complexity: the psychology of human behaviour in complex systems , 1988 .