Effects of Agent Transparency on Multi-Robot Management Effectiveness

Abstract : The objective of the study was to investigate the effects of agent transparency on operator performance in the context of joint human-agent decision making in multi-robot management. The agent display configurations were based on the 3 levels of the situation awareness-based agent transparency model (basic-information, reasoning, and projections/uncertainty). Results showed that participants calibrated their trust in the agent more effectively (proper reliance and correct rejections) and reported higher levels of trust when they were provided with the agent's reasoning and uncertainty information. No speed-accuracy trade-offs were observed. Nor did the participants report higher levels of workload when agent transparency increased. Working memory capacity was found to be a significant predictor of participants' trust in the agent. Individual differences in spatial ability accounted for variations in ocular indices of workload across display configurations.

[1]  Lu Wang,et al.  Trust and Reliance on an Automated Combat Identification System , 2009, Hum. Factors.

[2]  Wu,et al.  A Playbook Approach to Variable Autonomy Control: Application for Control of Multiple, Heterogeneous Unmanned Air Vehicles , 2004 .

[3]  Christopher D. Wickens,et al.  The benefits of imperfect diagnostic automation: a synthesis of the literature , 2007 .

[4]  Sheue-Ling Hwang,et al.  Predicting Work Performance in Nuclear Power Plants , 2008 .

[5]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[6]  D. Derryberry,et al.  Anxiety-related attentional biases and their regulation by attentional control. , 2002, Journal of abnormal psychology.

[7]  Nadine B. Sarter,et al.  How in the World Did We Ever Get into That Mode? Mode Error and Awareness in Supervisory Control , 1995, Hum. Factors.

[8]  Justin G. Hollands,et al.  Beyond Identity: Incorporating System Reliability Information Into an Automated Combat Identification System , 2011, Hum. Factors.

[9]  Thomas Fincannon,et al.  The influence of camouflage, obstruction, familiarity and spatial ability on target identification from an unmanned ground vehicle , 2013, Ergonomics.

[10]  Makoto Itoh,et al.  Human’s Overtrust in and Overreliance on Advanced Driver Assistance Systems: A Theoretical Framework , 2013 .

[11]  Mark R. Lehto,et al.  Foundations for an Empirically Determined Scale of Trust in Automated Systems , 2000 .

[12]  Miles MacLeod,et al.  Usability measurement in context , 1994, Behav. Inf. Technol..

[13]  L. Ball,et al.  Working Memory, Metacognitive Uncertainty, and Belief Bias in Syllogistic Reasoning , 2000, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[14]  B. Cain.  A Review of the Mental Workload Literature , 2007 .

[15]  Eric N. Wiebe,et al.  The Effects of Automated Decision Algorithm Modality and Transparency on Reported Trust and Task Performance , 2008 .

[16]  E. Salas,et al.  Human Factors : The Journal of the Human Factors and Ergonomics Society , 2007 .

[17]  Ann M. Bisantz,et al.  Utilizing Graphical Formats to Convey Uncertainty in a Decision Making Task , 2000 .

[18]  Michael W. Boyce,et al.  Situation Awareness-Based Agent Transparency , 2014 .

[19]  Jean Scholtz,et al.  Toward a Framework for Evaluating Ubiquitous Computing Applications , 2004, IEEE Pervasive Comput..

[20]  Corinna E. Lathan,et al.  The Effects of Operator Spatial Perception and Sensory Feedback on Human-Robot Teleoperation Performance , 2002, Presence: Teleoperators & Virtual Environments.

[21]  Tyler H. Shaw,et al.  Modeling Human-Automation Team Performance in Networked Systems: Individual Differences in Working Memory Count , 2010 .

[22]  Linda G. Pierce,et al.  Automation Usage Decisions: Controlling Intent and Appraisal Errors in a Target Detection Task , 2007, Hum. Factors.

[23]  M. Loeb,et al.  The Psychology of Vigilance , 1982 .

[24]  M. Hegarty,et al.  A dissociation between mental rotation and perspective-taking spatial abilities , 2004 .

[25]  Kait Clark,et al.  Enhanced change detection performance reveals improved strategy use in avid action video game players. , 2011, Acta psychologica.

[26]  J. Zaichkowsky.  Measuring the Involvement Construct , 1985 .

[27]  Anand S. Rao,et al.  BDI Agents: From Theory to Practice , 1995, ICMAS.

[28]  Yoshiaki Takeuchi,et al.  Individual differences in wayfinding strategies , 2003 .

[29]  Gaia Scerif,et al.  Using developmental cognitive neuroscience to study behavioral and attentional control. , 2009, Developmental psychobiology.

[30]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[31]  Paul R. Havig,et al.  Transparency in a Human-Machine Context: Approaches for Fostering Shared Awareness/Intent , 2014, HCI.

[32]  Maia B. Cook,et al.  Human Factors of the Confirmation Bias in Intelligence Analysis: Decision Support From Graphical Evidence Landscapes , 2008, Hum. Factors.

[33]  F. Paas,et al.  Instructional control of cognitive load in the training of complex cognitive tasks , 1994 .

[34]  Göran Falkman,et al.  Transparency of Automated Combat Classification , 2014, HCI.

[35]  Jessie Y C Chen,et al.  Supervisory control of multiple robots in dynamic tasking environments , 2012, Ergonomics.

[36]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[37]  V. Groom,et al.  Can robots be teammates?: Benchmarks in human–robot teams , 2007 .

[38]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[39]  Raja Parasuraman,et al.  Performance Consequences of Automation-Induced 'Complacency' , 1993 .

[40]  Michael F. Bunting,et al.  Working memory span tasks: A methodological review and user’s guide , 2005, Psychonomic bulletin & review.

[41]  T. Levine,et al.  Eta Squared, Partial Eta Squared, and Misreporting of Effect Size in Communication Research , 2002 .

[42]  Michael J. Barnes,et al.  The Effects of Level of Autonomy on Human-Agent Teaming for Multi-Robot Control and Local Security Maintenance , 2013 .

[43]  L. Gugerty,et al.  Reference-frame misalignment and cardinal direction judgments: group differences and strategies. , 2004, Journal of experimental psychology. Applied.

[44]  Paula J. Durlach,et al.  Human–Robot Interaction in the Context of Simulated Route Reconnaissance Missions , 2008 .

[45]  E. Williams.  Experimental Designs Balanced for the Estimation of Residual Effects of Treatments , 1949 .

[46]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[47]  Nigel Bevan,et al.  Extending Quality in Use to Provide a Framework for Usability Measurement , 2009, HCI.

[48]  Donald A. Norman,et al.  A non-parametric analysis of recognition experiments , 1964 .

[49]  Regina A. Pomranky,et al.  The role of trust in automation reliance , 2003, Int. J. Hum. Comput. Stud..

[50]  Kenneth Holmqvist,et al.  Eye tracking: a comprehensive guide to methods and measures , 2011 .

[51]  John D. Lee,et al.  Human-Automation Collaboration in Dynamic Mission Planning: A Challenge Requiring an Ecological Approach , 2006 .

[52]  Ewart de Visser,et al.  Measurement of trust in human-robot collaboration , 2007, 2007 International Symposium on Collaborative Technologies and Systems.

[53]  Fang Xie,et al.  Analysis on Eye Movement Indexes Based on Simulated Flight Task , 2014, HCI.

[54]  Lindsay O. Long,et al.  Visual Spatial Abilities in Uninhabited Ground Vehicle Task Performance During Teleoperation and Direct Line of Sight , 2011, PRESENCE: Teleoperators and Virtual Environments.

[55]  Peter Norvig,et al.  Artificial intelligence - a modern approach, 2nd Edition , 2003, Prentice Hall series in artificial intelligence.

[56]  Mary L. Cummings,et al.  Automation and Accountability in Decision Support System Interface Design , 2006 .

[57]  Ruth B. Ekstrom,et al.  Manual for kit of factor-referenced cognitive tests , 1976 .