Language-Capable Robots may Inadvertently Weaken Human Moral Norms

Previous research in moral psychology and human-robot interaction has shown that technology shapes human morality, and research in human-robot interaction has shown that humans naturally perceive robots as moral agents. Accordingly, we propose that language-capable autonomous robots are uniquely positioned among technologies to significantly impact human morality. We therefore argue that it is imperative that language-capable robots behave according to human moral norms and communicate in such a way that their intention to adhere to those norms is clear. Unfortunately, the design of current natural language oriented robot architectures enables certain architectural components to circumvent or preempt those architectures' moral reasoning capabilities. In this paper, we show how this may occur, using clarification request generation in current dialog systems as a motivating example. Furthermore, we present experimental evidence that the types of behavior exhibited by current approaches to clarification request generation can cause robots to (1) miscommunicate their moral intentions and (2) weaken humans' perceptions of moral norms within the current context. This work strengthens previous preliminary findings, and does so within an experimental paradigm that provides increased external and ecological validity over earlier approaches.

[1]  Matthias Scheutz,et al.  Going Beyond Literal Command-Based Instructions: Extending Robotic Natural Language Interaction Capabilities , 2015, AAAI.

[2]  Matthias Scheutz,et al.  "Thank You for Sharing that Interesting Fact!": Effects of Capability and Context on Indirect Speech Act Use in Task-Based Human-Robot Dialogue , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[3]  Jessica B. Hamrick,et al.  psiTurk: An open-source framework for conducting replicable behavioral experiments online , 2016, Behavior research methods.

[4]  Matthew Purver The Theory and Use of Clarification Requests in Dialogue , 2004 .

[5]  Matthias Scheutz,et al.  "Sorry, I Can't Do That": Developing Mechanisms to Appropriately Reject Directives in Human-Robot Interactions , 2015, AAAI Fall Symposia.

[6]  Evan A. Krause,et al.  Novel Mechanisms for Natural Human-Robot Interactions in the DIARC Architecture , 2013 .

[7]  Pamela J. Hinds,et al.  Using Robots to Moderate Team Conflict: The Case of Repairing Violations , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[8]  Takayuki Kanda,et al.  Do people hold a humanoid robot morally accountable for the harm it causes? , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[9]  Kazuaki Tanaka,et al.  Comparing Video, Avatar, and Robot Mediated Communication: Pros and Cons of Embodiment , 2014, CollabTech.

[10]  Matthias Scheutz,et al.  Towards morally sensitive action selection for autonomous social robots , 2015, 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[11]  Alexander I. Rudnicky,et al.  Miscommunication Recovery in Physically Situated Dialogue , 2015, SIGDIAL Conference.

[12]  E. Wagenmakers A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[13]  Todd M. Gureckis,et al.  CUNY Academic , 2016 .

[14]  Ronald C. Arkin Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture - Part 2: Formalization for Ethical Control , 2008, AGI.

[15]  Ronald C. Arkin,et al.  Governing lethal behavior: Embedding ethics in a hybrid deliberative/reactive robot architecture part I: Motivation and philosophy , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[16]  Matthias Scheutz,et al.  The Need for Moral Competency in Autonomous Agent Architectures , 2013, PT-AI.

[17]  Ewart de Visser,et al.  Measurement of trust in human-robot collaboration , 2007, 2007 International Symposium on Collaborative Technologies and Systems.

[18]  Nikolaos Mavridis,et al.  A review of verbal and non-verbal human-robot interactive communication , 2014, Robotics Auton. Syst..

[19]  Eric-Jan Wagenmakers,et al.  Bayesian tests to quantify the result of a replication attempt. , 2014, Journal of experimental psychology. General.

[20]  F. Eyssel,et al.  Social categorization of social robots: anthropomorphism as a function of robot group membership. , 2012, The British journal of social psychology.

[21]  L. M. M.-T. Theory of Probability , 1929, Nature.

[22]  Cynthia Matuszek,et al.  Grounded Language Learning: Where Robotics and NLP Meet , 2018, IJCAI.

[23]  B. Scassellati,et al.  Robots for use in autism research. , 2012, Annual review of biomedical engineering.

[24]  J. Berger,et al.  Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence , 1987 .

[25]  Ross A. Knepper On the Communicative Aspect of Human-Robot Joint Action * , 2016 .

[26]  Matthias Scheutz,et al.  How Robots Can Affect Human Behavior: Investigating the Effects of Robotic Displays of Protest and Distress , 2014, Int. J. Soc. Robotics.

[27]  Matthias Scheutz,et al.  Dempster-Shafer theoretic resolution of referential ambiguity , 2018, Auton. Robots.

[28]  Matthias Scheutz,et al.  First steps toward natural human-like HRI , 2007, Auton. Robots.

[29]  Matthias Scheutz,et al.  An Overview of the Distributed Integrated Cognition Affect and Reflection DIARC Architecture , 2018, Intelligent Systems, Control and Automation: Science and Engineering.

[30]  Benjamin Bergen,et al.  A Theoretical Model of Indirect Request Comprehension , 2017, AAAI Fall Symposia.

[31]  Tony Belpaeme,et al.  Children Comply with a Robot’s Indirect Requests , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[32]  R. Mclean,et al.  Using a repeated measures ANOVA to analyze the data from a pretest-posttest design : a potentially confusing task , 1975 .

[33]  N. Sharkey,et al.  The crying shame of robot nannies: An ethical appraisal , 2010 .

[34]  Dan Klein,et al.  Unified Pragmatic Models for Generating and Following Instructions , 2017, NAACL.

[35]  Matthias Scheutz,et al.  Sacrifice One For the Good of Many? People Apply Different Moral Norms to Human and Robot Agents , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[36]  Arindam Dey,et al.  He who hesitates is lost (...in thoughts over a robot) , 2018, APAScience.

[37]  Francesca Gino,et al.  Understanding ordinary unethical behavior: why people who value morality act immorally , 2015, Current Opinion in Behavioral Sciences.

[38]  Siobhan Chapman Logic and Conversation , 2005 .

[39]  Matthias Scheutz,et al.  Enabling robots to understand indirect speech acts in task-based interactions , 2017, HRI 2017.

[40]  Stefanie Tellex,et al.  Toward Information Theoretic Human-Robot Dialog , 2012, Robotics: Science and Systems.

[41]  Takanori Shibata,et al.  Living With Seal Robots—Its Sociopsychological and Physiological Influences on the Elderly at a Care House , 2007, IEEE Transactions on Robotics.

[42]  Tatsuya Nomura,et al.  Why Do Children Abuse Robots? , 2015, HRI.

[43]  Jonathan A C Sterne,et al.  Sifting the evidence—what's wrong with significance tests? , 2001, BMJ : British Medical Journal.

[44]  Maartje M. A. de Graaf,et al.  Sharing a life with Harvey: Exploring the acceptance of and relationship-building with a social robot , 2015, Comput. Hum. Behav..

[45]  Kent Bach,et al.  The top 10 misconceptions about implicature , 2006 .

[46]  Clifford Nass,et al.  Computers are social actors , 1994, CHI '94.

[47]  Daniel B Wright,et al.  Comparing groups in a before-after design: when t test and ANCOVA produce different results. , 2006, The British journal of educational psychology.

[48]  Peter-Paul Verbeek,et al.  Moralizing Technology: Understanding and Designing the Morality of Things , 2011 .

[49]  Jie W Weiss,et al.  Bayesian Statistical Inference for Psychological Research , 2008 .

[50]  Leif D. Nelson,et al.  False-Positive Psychology , 2011, Psychological science.

[51]  Sean Trott,et al.  Recognizing Intention from Natural Language : Clarification Dialog and Construction Grammar , 2016 .

[52]  S. Levinson Presumptive Meanings: The theory of generalized conversational implicature , 2001 .

[53]  Matthias Scheutz,et al.  Resolution of Referential Ambiguity in Human-Robot Dialogue Using Dempster-Shafer Theoretic Pragmatics , 2017, Robotics: Science and Systems.

[54]  Eric-Jan Wagenmakers,et al.  Replication Bayes factors from evidence updating , 2018, Behavior Research Methods.

[55]  Tom Williams,et al.  A Bayesian Analysis of Moral Norm Malleability during Clarification Dialogues , 2018, CogSci.

[56]  Jennifer Wiley,et al.  What Are the Odds? A Practical Guide to Computing and Reporting Bayes Factors , 2014, J. Probl. Solving.

[57]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[58]  Jamy Li,et al.  The benefit of being physically present: A survey of experimental works comparing copresent robots, telepresent robots and virtual agents , 2015, Int. J. Hum. Comput. Stud..

[59]  Brian Scassellati,et al.  The Benefits of Interactions with Physically Present Robots over Video-Displayed Agents , 2011, Int. J. Soc. Robotics.

[60]  Alexander I. Rudnicky,et al.  Sorry and I Didn’t Catch That! - An Investigation of Non-understanding Errors and Recovery Strategies , 2005, SIGDIAL.

[61]  Luciana Benotti,et al.  Polite Interactions with Robots , 2016, Robophilosophy/TRANSOR.

[62]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[63]  D. Dimitrov,et al.  Pretest-posttest designs and measurement of change. , 2003, Work.

[64]  Gregory Ward,et al.  Drawing the boundaries of meaning : neo-Gricean studies in pragmatics and semantics in honor of Laurence R. Horn , 2006 .

[65]  Brett Browning,et al.  Believable Robot Characters , 2011, AI Mag..

[66]  Johanna Seibt,et al.  What Social Robots Can and Should Do - Proceedings of Robophilosophy 2016 / TRANSOR 2016, Aarhus, Denmark, October 17-21, 2016 , 2016, Robophilosophy/TRANSOR.

[67]  Kerstin Fischer,et al.  Levels of embodiment: Linguistic analyses of factors influencing HRI , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[68]  Jesse J. Chandler,et al.  Crowdsourcing Samples in Cognitive Science , 2017, Trends in Cognitive Sciences.

[69]  Patrick Lin,et al.  Autonomous Military Robotics: Risk, Ethics, and Design , 2008 .

[70]  Matthias Scheutz,et al.  The Pragmatic Parliament: A Framework for Socially-Appropriate Utterance Selection in Artificial Agents , 2017, CogSci.