DualKeepon: a human–robot interaction testbed to study linguistic features of speech

In this paper, we present a novel dual-robot testbed called DualKeepon for carrying out pairwise comparisons of linguistic features of speech in human–robot interactions. Our solution, using a modified version of the MyKeepon robotic toy developed by Beatbots, is a portable open-source system for researchers to set up experiments quickly, and in an intuitive way. We provide an online tutorial with all required materials to replicate the system. We present two human–robot interaction studies to demonstrate the testbed. The first study investigates the perception of robots using filled pauses. The second study investigates how social roles, realized by different prosodic and lexical speaking profiles, affect trust. Results show that the proposed testbed is a helpful tool for linguistic studies. In addition to the basic setup, advanced users of the system have the ability to connect the system to different robot platforms, i.e., NAO, Pepper.

[1]  Brian Scassellati,et al.  Personalizing Robot Tutors to Individuals’ Learning Differences , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[2]  Kerstin Fischer,et al.  Levels of embodiment: Linguistic analyses of factors influencing HRI , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[3]  Bram Vanderborght,et al.  Probolino: A Portable Low-Cost Social Device for Home-Based Autism Therapy , 2015, ICSR.

[4]  Matthias Scheutz,et al.  Let Me Tell You! : Investigating the Effects of Robot Communication Strategies in Advice-giving Situations based on Robot Appearance, Interaction Modality and Distance , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[5]  D. Pisoni,et al.  The Handbook of Speech Perception , 2004 .

[6]  Pierre Dillenbourg,et al.  When Children Teach a Robot to Write: An Autonomous Teachable Humanoid Which Uses Simulated Handwriting , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[7]  Y. Mohammad,et al.  Embodiment of knowledge into the interaction and physical domains using robots , 2007, 2007 International Conference on Control, Automation and Systems.

[8]  Tony Belpaeme,et al.  From characterising three years of HRI to methodology and reporting recommendations , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[9]  Julia Hirschberg,et al.  Charisma perception from text and speech , 2009, Speech Commun..

[10]  Sonia Chernova,et al.  The robot management system: a framework for conducting human-robot interaction studies through crowdsourcing , 2014, HRI 2014.

[11]  Emily J. Ashurst,et al.  Robot education peers in a situated primary school study: Personalisation promotes child learning , 2017, PloS one.

[12]  Blair MacIntyre,et al.  Wizard of Oz support throughout an iterative design process , 2005, IEEE Pervasive Computing.

[13]  Aaron Wolfgang.  Nonverbal behavior : applications and cultural implications , 1980 .

[14]  Solís Obiols.  The Matched Guise Technique: a Critical Approximation to a Classic Test for Formal Measurement of Language Attitudes , 2002 .

[15]  Karen E. Johnson,et al.  Peer Review Negotiations: Revision Activities in ESL Writing Instruction , 1994 .

[16]  Robin R. Murphy,et al.  Review of Human Studies Methods in HRI and Recommendations , 2010, Int. J. Soc. Robotics.

[17]  Sean Andrist,et al.  Effects of Culture on the Credibility of Robot Speech: A Comparison between English and Arabic , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[18]  B. Robins,et al.  Robots as Isolators or Mediators for Children with Autism A Cautionary Tale , 2005 .

[19]  Hanafiah Yussof,et al.  HRI observation with My Keepon robot using Kansei Engineering approach , 2016, 2016 2nd IEEE International Symposium on Robotics and Manufacturing Automation (ROMA).

[20]  M. Bresnahan,et al.  Attitudinal and affective response toward accented English , 2002 .

[21]  Cristina A. Costescu,et al.  Robot-Enhanced CBT for dysfunctional emotions in social situations for children with ASD , 2017 .

[22]  Steve Whittaker,et al.  The role of vision in face-to-face and mediated communication. , 1997 .

[23]  Kerstin Dautenhahn,et al.  Practical and methodological challenges in designing and conducting human-robot interaction studies , 2005 .

[24]  Cristina A. Costescu,et al.  Reversal Learning Task in Children with Autism Spectrum Disorder: A Robot-Based Approach , 2015, Journal of autism and developmental disorders.

[25]  Vanessa Evers,et al.  The Effect of a Robot's Social Character on Children's Task Engagement: Peer Versus Tutor , 2015, ICSR.

[26]  Kerstin Fischer,et al.  From Cognitive Semantics to Lexical Pragmatics: The Functional Polysemy of Discourse Particles , 2000 .

[27]  Rolf Pfeifer,et al.  Morphological Computation - Connecting Brain, Body, and Environment , 2006, Australian Conference on Artificial Intelligence.

[28]  Oliver Niebuhr,et al.  Influence of Robot Gender and Speaker Gender on Prosodic Entrainment in HRI , 2017 .

[29]  Caroline Henton.  Text-to-Speech Synthesis Development , 2012 .

[30]  Juan Fasola,et al.  A socially assistive robot exercise coach for the elderly , 2013, J. Hum. Robot Interact..

[31]  Kerstin Fischer,et al.  Robots as Confederates: How Robots Can and Should Support Research in the Humanities , 2016, Robophilosophy/TRANSOR.

[32]  Ayanna M. Howard,et al.  Engaging children in math education using a socially interactive humanoid robot , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[33]  Pradipta Biswas,et al.  Eye Gaze Controlled Projected Display in Automotive and Military Aviation Environments , 2018, Multimodal Technol. Interact..

[34]  Brian Scassellati,et al.  The Benefits of Interactions with Physically Present Robots over Video-Displayed Agents , 2011, Int. J. Soc. Robotics.

[35]  Anna K. Kuhlen,et al.  Language in Dialogue: When Confederates Might Be Hazardous to Your Data , 2022 .

[36]  S. Fillenbaum,et al.  Evaluational reactions to spoken languages. , 1960, Journal of abnormal and social psychology.

[37]  Bram Vanderborght,et al.  Enhancing My Keepon robot: A simple and low-cost solution for robot platform in Human-Robot Interaction studies , 2014, The 23rd IEEE International Symposium on Robot and Human Interactive Communication.

[38]  Petra Wagner,et al.  Interactive Hesitation Synthesis: Modelling and Evaluation , 2018 .

[39]  Leila Takayama,et al.  Help Me Please: Robot Politeness Strategies for Soliciting Help From Humans , 2016, CHI.

[40]  Petra Wagner,et al.  The Hesitating Robot - Implementation and First Impressions , 2018, HRI.

[41]  Susan R. Fussell,et al.  How a robot should give advice , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[42]  John M. Murphy,et al.  Peer Response Groups: Do L2 Writers Use Peer Comments in Revising Their Drafts? , 1993 .

[43]  Cynthia Breazeal,et al.  Young Children Treat Robots as Informants , 2016, Top. Cogn. Sci..

[44]  Sean Andrist,et al.  Rhetorical robots: Making robots more effective speakers using linguistic cues of expertise , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[45]  Bram Vanderborght,et al.  Do infants perceive the social robot Keepon as a communicative partner? , 2016, Infant behavior & development.

[46]  V. Manusov,et al.  When a Small Thing Means so Much: Nonverbal Cues as Turning Points in Relationships , 2013 .

[47]  Tom Ziemke,et al.  Robot-assisted therapy for autism spectrum disorders with (partially) autonomous control: Challenges and outlook , 2012, Paladyn J. Behav. Robotics.

[48]  Jean E. Fox Tree,et al.  Interpreting Pauses and Ums at Turn Exchanges , 2002 .

[49]  Strahlt die negative Reputation von Häsitationsmarkern auf ihre Sprecher aus , 2017 .

[50]  Pamela J. Hinds,et al.  Whose job is it anyway? a study of human-robot interaction in a collaborative task , 2004 .

[51]  Joachim de Greeff,et al.  How to improve human-robot interaction with Conversational Fillers , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[52]  Wafa Johal,et al.  Keep on moving! Exploring anthropomorphic effects of motion during idle moments , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[53]  Marko Dragojevic,et al.  Communication Accommodation Theory , 2015 .

[54]  Laurel D. Riek,et al.  Wizard of Oz studies in HRI , 2012, J. Hum. Robot Interact..