A study measuring the impact of shared decision making in a human-robot team

This paper presents the results of a user study in which the impact of sharing decision making in a human-robot team was measured. In the experiments outlined here, a human and robot play a game together in which the robot searches an arena for items, with input from the human, and the human-robot team earns points for finding and correctly identifying the items. The user study reported here involved 60 human subjects. Each subject interacted with two different robots. With one robot, the human acted as a supervisor: the human issued commands and the robot obeyed. With the other robot, the human acted as a collaborator: the human and robot shared decisions and were required to reach agreement about the robot’s actions in the arena before any actions were taken, facilitated using computational argumentation. Objective performance metrics were collected and analyzed for both types of human-robot team, as well subjective feedback from human subjects regarding attitudes toward working with a robot. The objective results showed significant improvement in performance metrics with the human-as-collaborator pairs versus the human-as-supervisor pairs. Subjective results demonstrated significant differences across many subjective measures and indicated a distinct preference for the human-as-collaborator mode. The primary contribution of this work lies in the demonstration and evaluation of a computational argumentation approach to human-robot interaction, particularly in proving the efficacy of this approach over a less autonomous mode of interaction.

[1]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[2]  Elizabeth Sklar,et al.  HRTeam: a framework to support research on human/multi-robot interaction , 2013, AAMAS.

[3]  Anh Nguyen,et al.  A BDI agent architecture for dialogue modelling and coordination in a smart personal assistant , 2005, IEEE/WIC/ACM International Conference on Intelligent Agent Technology.

[4]  Brett Browning,et al.  Dynamically formed heterogeneous robot teams performing tightly-coordinated tasks , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[5]  Richard T. Vaughan,et al.  The Player/Stage Project: Tools for Multi-Robot and Distributed Sensor Systems , 2003 .

[6]  E. Sklar,et al.  An Argumentation Engine : ArgTrust , 2012 .

[7]  Sung-Bae Cho,et al.  Mixed-Initiative Human–Robot Interaction Using Hierarchical Bayesian Networks , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[8]  Elizabeth Sklar,et al.  Evaluation of a trust-modulated argumentation-based interactive decision-making tool , 2015, Autonomous Agents and Multi-Agent Systems.

[9]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[10]  Elizabeth Sklar,et al.  Towards the application of argumentation-based dialogues for education , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[11]  Peter McBurney,et al.  Argumentation-Based Dialogues for Agent Co-Ordination , 2003 .

[12]  Julie A. Shah,et al.  Decision-making authority, team efficiency and human worker satisfaction in mixed human–robot teams , 2015, Auton. Robots.

[13]  Peter McBurney,et al.  Dialogue Games for Agent Argumentation , 2009, Argumentation in Artificial Intelligence.

[14]  Marc-Philippe Huget,et al.  Communication in Multiagent Systems , 2003 .

[15]  Ross A. Knepper,et al.  Recovering from failure by asking for help , 2015, Auton. Robots.

[16]  Andrea Lockerd Thomaz,et al.  Timed Petri nets for fluent turn-taking over multimodal interaction resources in human-robot collaboration , 2016, Int. J. Robotics Res..

[17]  Matthias Scheutz,et al.  Toward Humanlike Task-Based Dialogue Processing for Human Robot Interaction , 2011, AI Mag..

[18]  Holly A. Yanco,et al.  A Taxonomy for Human-Robot Interaction , 2002 .

[19]  Stephanie Rosenthal,et al.  An effective personal mobile robot agent through symbiotic human-robot interaction , 2010, AAMAS.

[20]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[21]  Trevor J. M. Bench-Capon,et al.  Argumentation in artificial intelligence , 2007, Artif. Intell..

[22]  Peter McBurney,et al.  A denotational semantics for deliberation dialogues , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[23]  V. Groom,et al.  Can robots be teammates?: Benchmarks in human–robot teams , 2007 .

[24]  Leila Takayama,et al.  Help me help you: Interfaces for personal robots , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[25]  Elizabeth Sklar,et al.  Enabling Human-Robot Collaboration via Argumentation (Extended Abstract) , 2013 .

[26]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[27]  Michael Wooldridge,et al.  Properties and Complexity of Some Formal Inter-agent Dialogues , 2003, J. Log. Comput..

[28]  Abhizna Butchibabu,et al.  Anticipatory communication strategies for human robot team coordination , 2016 .

[29]  Matthias Scheutz,et al.  Learning to Recognize Novel Objects in One Shot through Human-Robot Interactions in Natural Language Dialogues , 2014, AAAI.

[30]  D. Walton,et al.  Commitment in Dialogue: Basic Concepts of Interpersonal Reasoning , 1995 .

[31]  Susan R. Fussell,et al.  Effects of adaptive robot dialogue on information exchange and social relations , 2006, HRI '06.

[32]  Katia Sycara,et al.  Developing a Testbed for Studying Human-Robot Interaction in Urban Search and Rescue , 2003 .

[33]  Elizabeth Sklar,et al.  Argumentation-based dialogue games for shared control in human-robot systems , 2015, HRI 2015.

[34]  Simon Parsons,et al.  A Generalization of Dung's Abstract Framework for Argumentation: Arguing with Sets of Attacking Arguments , 2006, ArgMAS.

[35]  Iyad Rahwan,et al.  Agreeing on plans through iterated disputes , 2010, AAMAS.

[36]  International Foundation for Autonomous Agents and MultiAgent Systems ( IFAAMAS ) , 2007 .

[37]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[38]  Jean Scholtz,et al.  Theory and evaluation of human robot interactions , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[39]  Peter McBurney,et al.  Dialogue Game Protocols , 2003, Communication in Multiagent Systems.

[40]  Sarvapali D. Ramchurn,et al.  Argumentation-based negotiation , 2003, The Knowledge Engineering Review.

[41]  Anthony Hunter,et al.  An inquiry dialogue system , 2008, Autonomous Agents and Multi-Agent Systems.

[42]  Chris Reed,et al.  Argumentation Schemes , 2008 .

[43]  Theory and Applications of Formal Argumentation , 2013, Lecture Notes in Computer Science.

[44]  Adam Cheyer,et al.  The Open Agent Architecture , 1997, Autonomous Agents and Multi-Agent Systems.

[45]  Elizabeth Black,et al.  Choosing persuasive arguments for action , 2011, AAMAS.

[46]  Brett Browning,et al.  Believable Robot Characters , 2011, AI Mag..

[47]  Terrence Fong,et al.  The human-robot interaction operating system , 2006, HRI '06.

[48]  Peter McBurney,et al.  What kind of argument are we going to have today? , 2005, AAMAS '05.

[49]  Alexander I. Rudnicky,et al.  Exploring Spoken Dialog Interaction in Human-Robot Teams , 2009 .

[50]  E. Krabbe.  Games that agents play: A formal framework for dialogues between autonomous agents , 2001 .

[51]  Illah R. Nourbakhsh,et al.  Educational Results of the Personal Exploration Rover Museum Exhibit , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[52]  Simon Parsons,et al.  A Model for Integrating Dialogue and the Execution of Joint Plans , 2009, ArgMAS.

[53]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[54]  Angel Rolando Medellin Gasque,et al.  Argumentation-based dialogues over cooperative plans , 2013 .

[55]  Roderic A. Girle,et al.  Commands in Dialogue Logic , 1996, FAPR.

[56]  Bilge Mutlu,et al.  Designing Embodied Cues for Dialog with Robots , 2011, AI Mag..

[57]  Elizabeth Sklar,et al.  Using argumentation to reason about trust and belief , 2012, J. Log. Comput..

[58]  Peter McBurney,et al.  Representing Epistemic Uncertainty by Means of Dialectical Argumentation , 2001, Annals of Mathematics and Artificial Intelligence.

[59]  Elizabeth Sklar,et al.  Designing the HRTeam framework: lessons learned from a rough-and-ready human/multi-robot team , 2011, AAMAS'11.

[60]  Wolfram Burgard,et al.  Probabilistic Algorithms and the Interactive Museum Tour-Guide Robot Minerva , 2000, Int. J. Robotics Res..

[61]  Andrea Lockerd Thomaz,et al.  Turn-Taking Based on Information Flow for Fluent Human-Robot Interaction , 2011, AI Mag..

[62]  Cynthia Breazeal,et al.  Collaboration in Human-Robot Teams , 2004, AIAA 1st Intelligent Systems Technical Conference.

[63]  Henry Prakken,et al.  DOI: 10.1017/S000000000000000 Printed in the United Kingdom Formal systems for persuasion dialogue , 2022 .

[64]  Elizabeth Black.  A generative framework for argumentation-based inquiry dialogues , 2007 .

[65]  Arvid Kappas,et al.  Towards Empathic Virtual and Robotic Tutors , 2013, AIED.

[66]  Britta Wrede,et al.  Pamini: A framework for assembling mixed-initiative human-robot interaction from generic interaction patterns , 2010, SIGDIAL Conference.

[67]  G. Glass,et al.  Consequences of Failure to Meet Assumptions Underlying the Fixed Effects Analyses of Variance and Covariance , 1972 .

[68]  Alan J. Dix.  Human-Computer Interaction , 2018, Encyclopedia of Database Systems.

[69]  Kristinn R. Thórisson,et al.  Learning Smooth, Human-Like Turntaking in Realtime Dialogue , 2008, IVA.

[70]  Joris Hulstijn,et al.  Dialogue Models for Inquiry and Transaction , 2000 .

[71]  Jean Scholtz,et al.  The Peer-to-Peer Human-Robot Interaction Project , 2005 .

[72]  Trevor J. M. Bench-Capon,et al.  Arguments over Co-operative Plans , 2011, TAFA.

[73]  Elizabeth Sklar,et al.  Analysis of empirical results on argumentation-based dialogue to support shared decision making in a human-robot team , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[74]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[75]  Simon Parsons,et al.  Argumentation-based dialogues for deliberation , 2005, AAMAS '05.

[76]  Stephanie Rosenthal,et al.  Designing robots for long-term social interaction , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[77]  Elizabeth Sklar,et al.  A prototype system for argumentation-based reasoning about trust , 2011 .

[78]  J. Shaffer.  Multiple Hypothesis Testing , 1995 .

[79]  Mani B. Srivastava,et al.  Enabling Intelligence Analysis through Agent-Support: the CISpaces Toolkit , 2015, AAMAS.

[80]  Mohammad Q. Azhar,et al.  Toward an argumentation-based dialogue framework for human-robot collaboration , 2012, ICMI '12.

[81]  Mani B. Srivastava,et al.  Supporting Reasoning with Different Types of Evidence in Intelligence Analysis , 2015, AAMAS.

[82]  Terrence Fong,et al.  Collaboration, Dialogue, and Human-Robot Interaction , 2001 .

[83]  E. Sklar,et al.  A system of argumentation for reasoning about trust , 2010 .

[84]  Peter McBurney,et al.  Chance Discovery Using Dialectical Argumentation , 2001, JSAI Workshops.

[85]  Jaime R Carbonell,et al.  Mixed-initiative man-computer instructional dialogues , 1970 .

[86]  Rocco J. Perla,et al.  Ten Common Misunderstandings, Misconceptions, Persistent Myths and Urban Legends about Likert Scales and Likert Response Formats and their Antidotes , 2007 .

[87]  Eric Horvitz,et al.  Principles of mixed-initiative user interfaces , 1999, CHI '99.

[88]  Kerstin Fischer,et al.  How People Talk with Robots: Designing Dialog to Reduce User Uncertainty , 2011, AI Mag..

[89]  John Fox,et al.  The Knowledge Engineering Review , 1984, The Knowledge Engineering Review.

[90]  Maya Cakmak,et al.  Teaching People How to Teach Robots: The Effect of Instructional Materials and Dialog Design , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[91]  Martin Caminada,et al.  Truth, Lies and Bullshit; distinguishing classes of dishonesty , 2009, IJCAI 2009.