Approaching human-like spatial awareness in social robotics: an investigation of spatial interaction strategies with a receptionist robot

This doctoral thesis investigates the influence of social signals in the spatial domain that aim to raise a robot’s awareness towards its human interlocutor. A concept of spatial awareness thereby extends the robot’s possibilities for expressing its knowledge about the situation as well as its own capabilities. As a result, especially untrained users can build up more appropriate expectations about the current situation which supposedly leads to a minimization of misunderstandings and thereby an enhancement of user experience. On the background of research that investigates communication among humans, relations are drawn in order to utilize gained insights for developing a robot that is capable of acting socially intelligent with regard to human-like treatment of spatial configurations and signals. In a study-driven approach, an integrated concept of spatial awareness is therefore proposed. An important aspect of that concept, which is founded in its spatial extent, lies in its aspiration to cover a holistic encounter between human and robot with the goal to improve user experience from the first sight until the end of reciprocal awareness. It describes how spatial configurations and signals can be perceived and interpreted in a social robot. Furthermore, it also presents signals and behavioral properties for such a robot that target at influencing said configurations and enhancing robot verbosity. In order to approve the concept’s validity in realistic settings, an interactive scenario is presented in the form of a receptionist robot to which it is applied. In the context of this setup, a comprehensive user study is conducted that verifies the implementation of spatial awareness to be beneficial for an interaction with humans that are naive to the subject. Furthermore, the importance of addressing an entire encounter in human-robot interaction is confirmed as well as a strong interdependency of a robot’s social signals among each other.

[1]  O. Watson.  Conflicts and Directions in Proxemic Research , 1972 .

[2]  L. Wheaton,et al.  I give you a cup, I get a cup: A kinematic study on social intention , 2014, Neuropsychologia.

[3]  Henrik I. Christensen,et al.  Evaluation of Passing Distance for Social Robots , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[4]  Gerhard Sagerer,et al.  Understanding Social Robots , 2009, 2009 Second International Conferences on Advances in Computer-Human Interactions.

[5]  Britta Wrede,et al.  The social robot ‘Flobi’: Key concepts of industrial design , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[6]  Mohan S. Kankanhalli,et al.  Temporal encoded F-formation system for social interaction detection , 2013, ACM Multimedia.

[7]  Kerstin Dautenhahn,et al.  A personalized robot companion? - The role of individual differences on spatial preferences in HRI scenarios , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[8]  T. M. Ciolek,et al.  Environment and the Spatial Arrangement of Conversational Encounters , 1980 .

[9]  Brett Browning,et al.  Believable Robot Characters , 2011, AI Mag..

[10]  Gernot A. Fink.  Developing HMM-Based Recognizers with ESMERALDA , 1999, TSD.

[11]  Maha Salem,et al.  Effects of Politeness and Interaction Context on Perception and Experience of HRI , 2013, ICSR.

[12]  Sven Wachsmuth,et al.  Active peripersonal space for more intuitive HRI , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[13]  Michael K Tanenhaus,et al.  Priming and alignment: Mechanism or consequence? , 2004, The Behavioral and brain sciences.

[14]  Marc Schröder,et al.  The German Text-to-Speech Synthesis System MARY: A Tool for Research, Development and Teaching , 2003, Int. J. Speech Technol..

[15]  Nikolaos Mavridis,et al.  A review of verbal and non-verbal human-robot interactive communication , 2014, Robotics Auton. Syst..

[16]  Terrence Fong,et al.  Collaboration, Dialogue, and Human-Robot Interaction , 2001 .

[17]  Claudia Wagner,et al.  Anthropomorphic inferences from emotional nonverbal cues: A case study , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[18]  Henrik I. Christensen,et al.  What's in the gap? Interaction transitions that make HRI work , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[19]  Cynthia Breazeal,et al.  Function meets style: insights from emotion theory applied to HRI , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[20]  Kerstin Dautenhahn,et al.  Living with Robots: Investigating the Habituation Effect in Participants' Preferences During a Longitudinal Human-Robot Interaction Study , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[21]  Nikolaos G. Tsagarakis,et al.  iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research , 2007, Adv. Robotics.

[22]  Ingo Lütkebohle,et al.  Facilitating Re-Use by Design: A Filtering, Transformation, and Selection Architecture for Robotic Software Systems , 2009 .

[23]  Takayuki Kanda,et al.  Interactive Robots as Social Partners and Peer Tutors for Children: A Field Trial , 2004, Hum. Comput. Interact..

[24]  Andrea Lockerd Thomaz,et al.  Effects of nonverbal communication on efficiency and robustness in human-robot teamwork , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  H. Kuzuoka,et al.  Reconfiguring spatial formation arrangement by robot body orientation , 2010, HRI 2010.

[26]  Hirotake Yamazoe,et al.  Gaze-communicative behavior of stuffed-toy robot with joint attention and eye contact based on ambient gaze-tracking , 2007, ICMI '07.

[27]  Roman Rädle,et al.  Proxemics in Human-Computer Interaction , 2014 .

[28]  C. Spence,et al.  Extending or projecting peripersonal space with tools? Multisensory interactions highlight only the distal and proximal ends of tools , 2004, Neuroscience Letters.

[29]  T. V. Oosterhout,et al.  A visual method for robot proxemics measurements , 2008 .

[30]  Christian Bauckhage,et al.  An XML based framework for cognitive vision architectures , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[31]  Yuichiro Yoshikawa,et al.  The effects of responsive eye movement and blinking behavior in a communication robot , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Kerstin Dautenhahn,et al.  Preferences and perceptions of robot appearance and embodiment in human-robot interaction trials , 2009 .

[33]  Sal Restivo,et al.  Bringing up and booting up: social theory and the emergence of socially intelligent robots , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[34]  Britta Wrede,et al.  A computational model of emotional alignment , 2011 .

[35]  D. Guitton,et al.  Gaze control in humans: eye-head coordination during orienting movements to targets within and beyond the oculomotor range. , 1987, Journal of neurophysiology.

[36]  M. Matarić,et al.  The use of socially assistive robots in the design of intelligent cognitive therapies for people with dementia , 2009, 2009 IEEE International Conference on Rehabilitation Robotics.

[37]  T. B. Sheridan.  Eight ultimate challenges of human-robot communication , 1997, Proceedings 6th IEEE International Workshop on Robot and Human Communication. RO-MAN'97 SENDAI.

[38]  J. Allwood.  BODILY COMMUNICATION DIMENSIONS OF EXPRESSION AND CONTENT , 2002 .

[39]  Takayuki Kanda,et al.  A larger audience, please! — Encouraging people to listen to a guide robot , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[40]  Giorgio Metta,et al.  Towards long-lived robot genes , 2008, Robotics Auton. Syst..

[41]  P. Mundy,et al.  Attention, Joint Attention, and Social Cognition , 2007, Current directions in psychological science.

[42]  Britta Wrede,et al.  Social facilitation with social robots? , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[43]  Marco Antonelli,et al.  Implicit Sensorimotor Mapping of the Peripersonal Space by Gazing and Reaching , 2011, IEEE Transactions on Autonomous Mental Development.

[44]  Jennifer S. Pardo,et al.  Is alignment always the result of automatic priming? , 2004, Behavioral and Brain Sciences.

[45]  Dariu Gavrila,et al.  The Development and Real-World Deployment of FROG, the Fun Robotic Outdoor Guide , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[46]  M. Patterson.  A sequential functional model of nonverbal exchange. , 1982 .

[47]  Jodi Forlizzi,et al.  Receptionist or information kiosk: how do people talk with a robot? , 2010, CSCW '10.

[48]  Brian Scassellati,et al.  Humanoid Robots: A New Kind of Tool , 2000, IEEE Intell. Syst..

[49]  F. Pavani,et al.  Action-specific remapping of peripersonal space , 2010, Neuropsychologia.

[50]  H. H. Clark,et al.  Referring as a collaborative process , 1986, Cognition.

[51]  Nicholas P. Holmes,et al.  The body schema and multisensory representation(s) of peripersonal space , 2004, Cognitive Processing.

[52]  Takanori Shibata,et al.  Living With Seal Robots—Its Sociopsychological and Physiological Influences on the Elderly at a Care House , 2007, IEEE Transactions on Robotics.

[53]  Sven Wachsmuth,et al.  A Computational Model for the Alignment of Hierarchical Scene Representations in Human-Robot Interaction , 2009, IJCAI.

[54]  Ingo Lütkebohle,et al.  Generic middleware support for coordinating robot software components: The Task-State-Pattern , 2011 .

[55]  Christian Bauckhage,et al.  An active memory as a model for information fusion , 2004 .

[56]  G. Pellegrino,et al.  Social Modulation of Peripersonal Space Boundaries , 2013, Current Biology.

[57]  R. Dolan,et al.  Psychology: Reward value of attractiveness and gaze , 2001, Nature.

[58]  Stephanie Rosenthal,et al.  Designing robots for long-term social interaction , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[59]  H. Hüttenrauch,et al.  Making a Case for Spatial Prompting in Human-Robot Communication , 2006 .

[60]  Sarah Brown-Schmidt,et al.  Real-Time Investigation of Referential Domains in Unscripted Conversation: A Targeted Language Game Approach , 2008, Cogn. Sci..

[61]  T. Jones,et al.  Interpersonal distance, body orientation, and touch: effects of culture, gender, and age. , 1995, The Journal of social psychology.

[62]  Stefan Kopp,et al.  Synthesizing multimodal utterances for conversational agents , 2004, Comput. Animat. Virtual Worlds.

[63]  K. M. Lee,et al.  Can robots manifest personality? : An empirical test of personality recognition, social responses, and social presence in human-robot interaction , 2006 .

[64]  E. Hall,et al.  The Hidden Dimension , 1970 .

[65]  A. Kendon.  Conducting Interaction: Patterns of Behavior in Focused Encounters , 1990 .

[66]  Katharina J. Rohlfing,et al.  Towards Learning by Interacting , 2009, Creating Brain-Like Intelligence.

[67]  Ipke Wachsmuth,et al.  From body space to interaction space: modeling spatial cooperation for virtual humans , 2011, AAMAS.

[68]  Ana Paiva,et al.  Social Robots for Long-Term Interaction: A Survey , 2013, International Journal of Social Robotics.

[69]  Leila Takayama,et al.  Judging a bot by its cover: An experiment on expectation setting for personal robots , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[70]  Elena Torta,et al.  Design of Robust Robotic Proxemic Behaviour , 2011, ICSR.

[71]  Takayuki Kanda,et al.  Guidelines for Contextual Motion Design of a Humanoid Robot , 2013, Int. J. Soc. Robotics.

[72]  Kerstin Fischer,et al.  How People Talk with Robots: Designing Dialog to Reduce User Uncertainty , 2011, AI Mag..

[73]  Yvonne Rogers,et al.  Using F-formations to analyse spatial patterns of interaction in physical environments , 2011, CSCW.

[74]  Manja Lohse,et al.  The role of expectations and situations in human-robot interaction , 2011, HRI 2011.

[75]  Brian T. Gill,et al.  Validating interaction patterns in HRI , 2010, IEEE/ACM International Conference on Human-Robot Interaction.

[76]  Jannik Fritsch,et al.  Humanoid robot platform suitable for studying embodied interaction , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[77]  Giulio Sandini,et al.  The iCub humanoid robot: An open-systems platform for research in cognitive development , 2010, Neural Networks.

[78]  Mitsuharu Morisawa,et al.  Humanoid robot HRP-4 - Humanoid robotics platform with lightweight and slim body , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[79]  Edward T. Hall,et al.  A System for the Notation of Proxemic Behavior1 , 1963 .

[80]  Alexander H. Waibel,et al.  Behavior models for learning and receptionist dialogs , 2007, INTERSPEECH.

[81]  Cheryl Albas.  Proxemic Behavior: A Study of Extrusion , 1991 .

[82]  R. Simmons,et al.  COMPANION: A Constraint-Optimizing Method for Person-Acceptable Navigation , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[83]  Jennifer S. Pardo,et al.  On phonetic convergence during conversational interaction. , 2006, The Journal of the Acoustical Society of America.

[84]  Ira B. Perelle,et al.  An international study of human handedness: The data , 1994, Behavior genetics.

[85]  Hideaki Kuzuoka,et al.  “The first five seconds”: Contingent stepwise entry into an interaction as a means to secure sustained engagement in HRI , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[86]  Sonja Caraian,et al.  Influence of robot-issued joint attention cues on gaze and preference , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[87]  Sebastian Wrede,et al.  Attitude of German museum visitors towards an interactive art guide robot , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[88]  Marek P. Michalowski,et al.  A spatial model of engagement for a social robot , 2006, 9th IEEE International Workshop on Advanced Motion Control, 2006..

[89]  M. Knapp,et al.  Nonverbal communication in human interaction , 1972 .

[90]  N. Triplett,et al.  The Dynamogenic Factors in Pacemaking and Competition , 1898 .

[91]  Annika Peters,et al.  Spatial coordination - human and robotic communicative whole-body motions in narrow passages , 2012 .

[92]  Hartwig Holzapfel,et al.  Coding interaction patterns between human and receptionist robot , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[93]  Matthias Scheutz,et al.  Investigating multimodal real-time patterns of joint attention in an hri word learning task , 2010, HRI 2010.

[94]  Cynthia Breazeal,et al.  Social Robots that Interact with People , 2008, Springer Handbook of Robotics.

[95]  Cynthia Breazeal,et al.  Toward sociable robots , 2003, Robotics Auton. Syst..

[96]  Emanuel A. Schegloff.  Opening sequencing , 2002 .

[97]  Sven Wachsmuth,et al.  What can I do for you? Appearance and Application of Robots , 2007 .

[98]  Sven Wachsmuth,et al.  On Grounding Natural Kind Terms in Human-Robot Communication , 2013, KI - Künstliche Intelligenz.

[99]  F. Binkofski,et al.  Can Machines Think? Interaction and Perspective Taking with Robots Investigated via fMRI , 2008, PloS one.

[100]  W. Kruskal,et al.  Use of Ranks in One-Criterion Variance Analysis , 1952 .

[101]  Brian R. Duffy,et al.  Anthropomorphism and the social robot , 2003, Robotics Auton. Syst..

[102]  Xun Wang,et al.  Interpreting Robot Pointing Behavior , 2013, ICSR.

[103]  M. Tanenhaus,et al.  Context, syntactic priming, and referential form in an interactive dialogue task: implications for models of alignment , 2010 .

[104]  Peter Robinson,et al.  Cooperative gestures: effective signaling for humanoid robots , 2010, HRI 2010.

[105]  M. Argyle,et al.  EYE-CONTACT, DISTANCE AND AFFILIATION. , 1965, Sociometry.

[106]  Giulio Sandini,et al.  An experimental evaluation of a novel minimum-jerk cartesian controller for humanoid robots , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[107]  Cynthia Breazeal,et al.  Effects of anticipatory action on human-robot teamwork: Efficiency, fluency, and perception of team , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[108]  Rodrigo Ventura,et al.  People Detection and Tracking in a , 2007 .

[109]  Frank Lömker,et al.  Building Modular Vision Systems with a Graphical Plugin Environment , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[110]  Pamela J. Hinds,et al.  Autonomy and Common Ground in Human-Robot Interaction: A Field Study , 2007, IEEE Intelligent Systems.

[111]  Britta Wrede,et al.  Pamini: A framework for assembling mixed-initiative human-robot interaction from generic interaction patterns , 2010, SIGDIAL Conference.

[112]  Ehud Sharlin,et al.  Designing Social Greetings and Proxemics in Human Robot Interaction , 2013 .

[113]  E. Sundstrom,et al.  Interpersonal relationships and personal space: Research review and theoretical model , 1976 .

[114]  Russ Tedrake,et al.  Efficient Bipedal Robots Based on Passive-Dynamic Walkers , 2005, Science.

[115]  Horst-Michael Groß,et al.  TOOMAS: Interactive Shopping Guide robots in everyday use - final implementation and experiences from long-term field trials , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[116]  A. Kendon.  Some functions of gaze-direction in social interaction. , 1967, Acta psychologica.

[117]  Stefan Kopp,et al.  Appropriate feedback in asymmetric interactions , 2010 .

[118]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[119]  Anna-Lisa Vollmer,et al.  Robot feedback shapes the tutor’s presentation: How a robot’s online gaze strategies lead to micro-adaptation of the human’s conduct , 2013 .

[120]  Stefan Kopp,et al.  Closing the Loop: Towards Tightly Synchronized Robot Gesture and Speech , 2013, ICSR.

[121]  Nicole C. Krämer,et al.  Social Communicative Effects of a Virtual Program Guide , 2005, IVA.

[122]  Sven Wachsmuth,et al.  The Receptionist Robot* , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[123]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[124]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[125]  T. M. Ciolek.  The proxemics lexicon: A first approximation , 1983 .

[126]  Heiko Wersing,et al.  Peripersonal space and object recognition for humanoids , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[127]  Reid G. Simmons,et al.  Expressing ethnicity through behaviors of a robot character , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[128]  Daniel C. Richardson,et al.  The Art of Conversation Is Coordination , 2007, Psychological science.

[130]  Stefan Kopp,et al.  Alignment in communication : towards a new theory of communication , 2013 .

[131]  L. Mondada.  Emergent focused interactions in public places: A systematic analysis of the multimodal achievement of a common interactional space , 2009 .

[132]  Martin J. Pickering,et al.  Toward a neural basis of interactive alignment in conversation , 2012, Front. Hum. Neurosci..

[133]  Alois Knoll,et al.  Integrating Language, Vision and Action for Human Robot Dialog Systems , 2007, HCI.

[134]  H. Kuzuoka,et al.  A Trial Attempt by a Museum Guide Robot to Engage and Disengage the Audience on Time , 2013 .

[135]  Kerstin Fischer,et al.  Partner Orientation in Asymmetric Communication: Evidence from contingent robot response , 2014 .

[136]  Kerstin Dautenhahn,et al.  Methodology & Themes of Human-Robot Interaction: A Growing Research Field , 2007 .

[137]  Ingo Lütkebohle,et al.  Mixed Initiative in Interactive Robotic Learning , 2009 .

[138]  Sven Wachsmuth,et al.  Using articulated scene models for dynamic 3d scene analysis in vista spaces , 2010 .

[139]  Ipke Wachsmuth,et al.  Spatial References with Gaze and Pointing in Shared Space of Humans and Robots , 2014, Spatial Cognition.

[140]  Annika Peters,et al.  Small movements as communicational cues in HRI , 2011 .

[141]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[142]  Hideaki Kuzuoka,et al.  An Ethnomethodological Study of a Museum Guide Robot's Attempt at Engagement and Disengagement , 2014, J. Robotics.

[143]  Ingo Lütkebohle,et al.  The curious robot - Structuring interactive robot learning , 2009, 2009 IEEE International Conference on Robotics and Automation.

[144]  J. Cacioppo,et al.  On seeing human: a three-factor theory of anthropomorphism. , 2007, Psychological review.

[145]  Leila Takayama,et al.  Influences on proxemic behaviors in human-robot interaction , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[146]  M. Pickering,et al.  Toward a mechanistic psychology of dialogue , 2004, Behavioral and Brain Sciences.

[147]  Takayuki Kanda,et al.  A Model of Proximity Control for Information-Presenting Robots , 2010, IEEE Transactions on Robotics.

[148]  Kiyohiro Shikano,et al.  ASKA: receptionist robot with speech dialogue system , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[149]  Illah R. Nourbakhsh,et al.  A survey of socially interactive robots , 2003, Robotics Auton. Syst..

[150]  D. Lloyd,et al.  The space between us: A neurophilosophical framework for the investigation of human interpersonal space , 2009, Neuroscience & Biobehavioral Reviews.

[151]  Kerstin Dautenhahn,et al.  Sharing Spaces with Robots in a Home Scenario - Anthropomorphic Attributions and their Effect on Proxemic Expectations and Evaluations in a Live HRI Trial , 2008, AAAI Fall Symposium: AI in Eldercare: New Solutions to Old Problems.

[152]  C. Moore,et al.  Joint attention : its origins and role in development , 1995 .

[153]  J. Donath.  Signals, cues and meaning , 2007 .

[154]  Toshiaki Tsuji,et al.  Realization and Evaluation of Realistic Nod with Receptionist robot SAYA , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[155]  H. R. Beech,et al.  Social Interaction , 1970, Encyclopedia of Social Network Analysis and Mining.

[156]  Britta Wrede,et al.  Towards a typology of meaningful signals and cues in social robotics , 2011, 2011 RO-MAN.