User Localization During Human-Robot Interaction

This paper presents a user localization system based on the fusion of visual information and sound source localization, implemented on a social robot called Maggie. One of the main requisites to obtain a natural interaction between human-human and human-robot is an adequate spatial situation between the interlocutors, that is, to be orientated and situated at the right distance during the conversation in order to have a satisfactory communicative process. Our social robot uses a complete multimodal dialog system which manages the user-robot interaction during the communicative process. One of its main components is the presented user localization system. To determine the most suitable allocation of the robot in relation to the user, a proxemic study of the human-robot interaction is required, which is described in this paper. The study has been made with two groups of users: children, aged between 8 and 17, and adults. Finally, at the end of the paper, experimental results with the proposed multimodal dialog system are presented.

[1]  Hiroaki Kitano,et al.  Applying scattering theory to robot audition system: robust sound source localization and extraction , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[2]  H. S. Wolff,et al.  iRun: Horizontal and Vertical Shape of a Region-Based Graph Compression , 2022, Sensors.

[3]  Anders Green,et al.  Investigating Spatial Relationships in Human-Robot Interaction , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  E. Hall,et al.  The Hidden Dimension , 1970 .

[5]  Edin Arnautovic,et al.  Using communicative acts in interaction design specifications for automated synthesis of user interfaces , 2006, 21st IEEE/ACM International Conference on Automated Software Engineering (ASE'06).

[6]  Jean Rouat,et al.  Robust sound source localization using a microphone array on a mobile robot , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[7]  Robin R. Murphy,et al.  Towards a computational method of scaling a robot's behavior via proxemics , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[8]  Michael S. Brandstein,et al.  Microphone Arrays - Signal Processing Techniques and Applications , 2001, Microphone Arrays.

[9]  Fernando Alonso-Martín,et al.  INTEGRATION OF A VOICE RECOGNITION SYSTEM IN A SOCIAL ROBOT , 2011, Cybern. Syst..

[10]  Andrew Rae,et al.  Particle filtering for bearing-only audio-visual speaker detection and tracking , 2009, 2009 3rd International Conference on Signals, Circuits and Systems (SCS).

[11]  Donn Byrne,et al.  Too close for comfort : Sex differences in response to invasions of personal space , 1975 .

[12]  Ashutosh Saxena,et al.  Learning sound location from a single microphone , 2009, 2009 IEEE International Conference on Robotics and Automation.

[13]  M. Argyle,et al.  EYE-CONTACT, DISTANCE AND AFFILIATION. , 1965, Sociometry.

[14]  A. A. Handzel,et al.  Biomimetic sound-source localization , 2002 .

[15]  Tetsuya Ogata,et al.  Improvement in listening capability for humanoid robot HRP-2 , 2010, 2010 IEEE International Conference on Robotics and Automation.

[16]  A. Hudspeth.  The Hair Cells of the Inner Ear , 1983 .

[17]  R. Hoy,et al.  The cyclopean ear: a new sense for the praying mantis. , 1986, Science.

[18]  François Michaud,et al.  Embedded and Integrated Audition for a Mobile Robot , 2006, AAAI Fall Symposium: Aurally Informed Performance.

[19]  Judee K. Burgoon,et al.  TOWARD A THEORY OF PERSONAL SPACE EXPECTATIONS AND THEIR VIOLATIONS , 1976 .

[20]  B. Morrongiello.  Infants' monaural localization of sounds: effects of unilateral ear infection. , 1989, The Journal of the Acoustical Society of America.

[21]  Arkady B. Zaslavsky,et al.  Communicative acts of Elvin-enhanced mobile agents , 2003, IEEE/WIC International Conference on Intelligent Agent Technology, 2003. IAT 2003..

[22]  Brian Scassellati,et al.  Infant-like Social Interactions between a Robot and a Human Caregiver , 2000, Adapt. Behav..

[23]  R. Barber,et al.  Maggie: A Robotic Platform for Human-Robot Social Interaction , 2006, 2006 IEEE Conference on Robotics, Automation and Mechatronics.

[24]  P. S. Krishnaprasad,et al.  Robot phonotaxis with dynamic sound-source localization , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[25]  Hiroaki Kitano,et al.  Real-time sound source localization and separation for robot audition , 2002, INTERSPEECH.

[26]  A C Brown.  The Sense of Rotation and the Anatomy and Physiology of the Semicircular Canals of the Internal Ear. , Journal of anatomy and physiology.

[27]  Chen-Yu Chan,et al.  Simultaneous localization of mobile robot and multiple sound sources using microphone array , 2009, 2009 IEEE International Conference on Robotics and Automation.

[28]  Clifford Nass,et al.  Computers are social actors , 1994, CHI '94.

[29]  B. Wright,et al.  Different patterns of human discrimination learning for two interaural cues to sound-source location , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Ramon Barber,et al.  A New Human Based Architecture for Intelligent Autonomous Robots , 2001 .

[31]  François Michaud,et al.  Embedded auditory system for small mobile robots , 2008, 2008 IEEE International Conference on Robotics and Automation.

[32]  Boaz Rafaely,et al.  Microphone Array Signal Processing , 2008 .

[33]  Brown Ac,et al.  The Sense of Rotation and the Anatomy and Physiology of the Semicircular Canals of the Internal Ear. , 1874 .

[34]  María Malfaz,et al.  Multimodal Human-Robot Interaction Framework for a Personal Robot , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[35]  Chrystopher L. Nehaniv,et al.  An empirical framework for human-robot proxemics , 2009 .

[36]  Kerstin Dautenhahn,et al.  Living with Robots: Investigating the Habituation Effect in Participants' Preferences During a Longitudinal Human-Robot Interaction Study , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[37]  Jean Rouat,et al.  Robust 3D Localization and Tracking of Sound Sources Using Beamforming and Particle Filtering , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[38]  Jean Rouat,et al.  Robust localization and tracking of simultaneous moving sound sources using beamforming and particle filtering , 2007, Robotics Auton. Syst..

[39]  R. J. Ross,et al.  Detection of substrate vibrations by salamanders: inner ear sense organ activity , 1978 .

[40]  Fernando Alonso-Martín,et al.  Maggie: A Social Robot as a Gaming Platform , 2011, Int. J. Soc. Robotics.

[41]  Leila Takayama,et al.  Influences on proxemic behaviors in human-robot interaction , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[42]  Fernando Alonso-Martín,et al.  Using a Social Robot as a Gaming Platform , 2010, ICSR.

[43]  John L. Williams.  Personal space and its relation to extraversion-introversion. , 1971 .

[44]  Hiroshi G. Okuno,et al.  An open source software system for robot audition HARK and its evaluation , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[45]  L. Hayduk,et al.  Personal Space: An Evaluative and Orienting Overview. , 1978 .

[46]  Hermann Kaindl,et al.  Fully-automatic generation of user interfaces for multiple devices from a high-level model based on communicative acts , 2007, 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07).

[47]  Hearing in Birds and Reptiles: An Overview , 2000 .