The Design of a HSMM-Based Operator State Modeling Display

This paper presents the findings from the design, development, and evaluation of a hidden semi-Markov model based operator state monitoring display. This operator state monitoring display is designed to function as a decision support tool for the supervisor of a small team of operators (2-4) that are each monitoring and controlling multiple highly autonomous, heterogeneous unmanned vehicles. After a cognitive task analysis was conducted to determine the functional and information requirements of the decision support tool, a human subject experiment was conducted to validate the design. Preliminary results show that the decision support tool improves team supervisor performance in terms of increased decision accuracy and decreased non-necessary interventions in single alert scenarios. In scenarios where the supervisor faced multiple simultaneous alerts, the decision support tool was shown to decrease non-necessary interventions and have no affect on supervisor decision accuracy. Additionally, the decision support tool was not shown to effect mental workload as measured by a secondary task. These initial results suggest that hidden semi-Markov operator state modeling can be effectively utilized in a real-time decision support tool for team supervisors.

[1]  Y. Guédon.  Estimating Hidden Semi-Markov Chains From Discrete Sequences , 2003 .

[2]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[3]  Gary Klein,et al.  Sources of Power: How People Make Decisions , 2017 .

[4]  Allen M. Waxman,et al.  Automated activity pattern learning and monitoring provide decision support to supervisors of busy environments , 2009, Intell. Decis. Technol..

[5]  Stephanie Guerlain,et al.  User-Initiated Notification: A Concept for Aiding the Monitoring Activities of Process Control Operators , 1996 .

[6]  R. Yerkes,et al.  The relation of strength of stimulus to rapidity of habit‐formation , 1908 .

[7]  S. Fiske,et al.  The Handbook of Social Psychology , 1935 .

[8]  Chandler Stolp,et al.  The Visual Display of Quantitative Information , 1983 .

[9]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[10]  A. Tversky,et al.  Judgment under Uncertainty , 1982 .

[11]  Carl Gutwin,et al.  Task analysis for groupware usability evaluation: Modeling shared-workspace tasks with the mechanics of collaboration , 2003, TCHI.

[12]  Jessica Townsend,et al.  Is There a Problem , 1992 .

[13]  Peter O. Passenier,et al.  The Effect of a Priori Probability and Complexity on Decision Making in a Supervisory Control Task , 1996, Hum. Factors.

[14]  J. Hackman,et al.  Leading Teams: Setting the Stage for Great Performances , 2002 .

[15]  G. Robert J. Hockey,et al.  State, Stress, and Time Pressure , 1993 .

[16]  Arthur D. Fisk,et al.  Modeling and analysis of a dynamic judgment task using a lens model approach , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[17]  Edward J. Sondik,et al.  The optimal control of par-tially observable Markov processes , 1971 .

[18]  Carl Nehme,et al.  Modeling human supervisory control in heterogeneous unmanned vehicle systems , 2009 .

[19]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[20]  Valerie L. Shalin,et al.  Cognitive task analysis , 2000 .

[21]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[22]  A. Pentland,et al.  Toward augmented control systems , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[23]  D. Kumaran,et al.  Frames, Biases, and Rational Decision-Making in the Human Brain , 2006, Science.

[24]  Hank Hsin Han Huang,et al.  Developing an abstraction layer for the visualization of HSMM-based predictive decision support , 2009 .

[25]  Karl R. Gegenfurtner,et al.  Color Vision: From Genes to Perception , 1999 .

[26]  James D. Herbsleb,et al.  Characterizing the sequential structure of interactive behaviors through statistical and grammatical techniques , 1994 .

[27]  E. Miller.  Handbook of Social Psychology , 1946, Mental Health.

[28]  D. Broadbent.  Perception and communication , 1958 .

[29]  John S. Gardenier.  Ship Navigational Failure Detection and Diagnosis , 1981 .

[30]  G Gigerenzer,et al.  Reasoning the fast and frugal way: models of bounded rationality. , 1996, Psychological review.

[31]  Cathleen Wharton,et al.  The cognitive walkthrough method: a practitioner's guide , 1994 .

[32]  Stacey D. Scott,et al.  Investigating supervisory-level activity awareness displays for command and control operations , 2009 .

[33]  Elizabeth C. Hirschman,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[34]  Thomas B. Sheridan,et al.  HUMAN SUPERVISORY CONTROL ISSUES IN NETWORK CENTRIC WARFARE , 2004 .

[35]  N. Moray,et al.  Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. , 1996, Ergonomics.

[36]  Neil Brewer,et al.  Effects of supervisory monitoring on productivity and quality of performance. , 1998 .

[37]  Mary L. Cummings,et al.  Developing Operator Capacity Estimates for Supervisory Control of Autonomous Vehicles , 2007, Hum. Factors.

[38]  Jacob W. Crandall,et al.  The Impact of Heterogeneity on Operator Performance in Future Unmanned Vehicle Systems , 2008 .

[39]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[40]  Hervé Debar,et al.  Aggregation and Correlation of Intrusion-Detection Alerts , 2001, Recent Advances in Intrusion Detection.

[41]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[42]  W. Klein,et al.  Heuristics and Biases: Resistance of Personal Risk Perceptions to Debiasing Interventions , 2002 .

[43]  A. Welford.  THE ‘PSYCHOLOGICAL REFRACTORY PERIOD’ AND THE TIMING OF HIGH‐SPEED PERFORMANCE—A REVIEW AND A THEORY , 1952 .

[44]  Regina A. Pomranky,et al.  The role of trust in automation reliance , 2003, Int. J. Hum. Comput. Stud..

[45]  FisherCarolanne,et al.  Exploratory sequential data analysis , 1993 .

[46]  Christopher A. Higgins,et al.  Computerized performance monitoring systems: use and abuse , 1986, CACM.

[47]  E. Salas,et al.  Toward an understanding of team performance and training. , 1992 .

[48]  Nancy G Leveson,et al.  Software safety: why, what, and how , 1986, CSUR.

[49]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[50]  S. Greenberg,et al.  The Importance of Awareness for Team Cognition in Distributed Collaboration , 2001 .

[51]  Stacey D. Scott,et al.  Aiding Team Supervision in Command and Control Operations with Large-Screen Displays , 2007 .

[52]  K. Preston White,et al.  Systems engineering models of human-machine interaction , 1981, Proceedings of the IEEE.

[53]  Paul Scerri,et al.  Using Dynamic Neural Network to Model Team Performance for Coordination Algorithm Configuration and Reconfiguration of Large Multi-Agent Teams , 2006 .

[54]  Mary L. Cummings,et al.  Automation Architecture for Single Operator, Multiple UAV Command and Control, , 2007 .

[55]  Mary L. Cummings,et al.  Automation Bias in Intelligent Time Critical Decision Support Systems , 2004 .

[56]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[57]  Nicholas Roy,et al.  Comparing Learning Techniques for Hidden Markov Models of Human Supervisory Control Behavior , 2009 .

[59]  Leo Maurice Hurvich,et al.  Color vision , 1981 .

[60]  Gary Klein,et al.  Working Minds: A Practitioner's Guide to Cognitive Task Analysis , 2006 .