iSocioBot: A Multimodal Interactive Social Robot

We present one way of constructing a social robot, such that it is able to interact with humans using multiple modalities. The robotic system is able to direct attention towards the dominant speaker using sound source localization and face detection, it is capable of identifying persons using face recognition and speaker identification and the system is able to communicate and engage in a dialog with humans by using speech recognition, speech synthesis and different facial expressions. The software is built upon the open-source robot operating system framework and our software is made publicly available. Furthermore, the electrical parts (sensors, laptop, base platform, etc.) are standard components, thus allowing for replicating the system. The design of the robot is unique and we justify why this design is suitable for our robot and the intended use. By making software, hardware and design accessible to everyone, we make research in social robotics available to a broader audience. To evaluate the properties and the appearance of the robot we invited users to interact with it in pairs (active interaction partner/observer) and collected their responses via an extended version of the Godspeed Questionnaire. Results suggest an overall positive impression of the robot and interaction experience, as well as significant differences in responses based on type of interaction and gender.

[1]  J. Broekens,et al.  Assistive social robots in elderly care: a review , 2009 .

[2]  Zheng-Hua Tan,et al.  Designing and Implementing an Interactive Social Robot from Off-the-shelf Components , 2015 .

[3]  F. Eyssel,et al.  Social categorization of social robots: anthropomorphism as a function of robot group membership. , 2012, The British journal of social psychology.

[4]  Elizabeth Ann Jochum,et al.  Head Orientation Behavior of Users and Durations in Playful Open-Ended Interactions with an Android Robot , 2015, CR@RO-MAN.

[5]  John S. Seiter,et al.  Persuasion: Social Inflence and Compliance Gaining , 2015 .

[6]  Cynthia Breazeal,et al.  Social interactions in HRI: the robot view , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[7]  Matti Pietikäinen,et al.  Computer Vision Using Local Binary Patterns , 2011, Computational Imaging and Vision.

[8]  R. Cialdini.  Influence: The Psychology of Persuasion , 1993 .

[9]  Friederike Eyssel,et al.  Learning with Educational Companion Robots? Toward Attitudes on Education Robots, Predictors of Attitudes, and Application Potentials for Education Robots , 2015, Int. J. Soc. Robotics.

[10]  P. Boersma.  ACCURATE SHORT-TERM ANALYSIS OF THE FUNDAMENTAL FREQUENCY AND THE HARMONICS-TO-NOISE RATIO OF A SAMPLED SOUND , 1993 .

[11]  Ville Ojansivu,et al.  Blur Insensitive Texture Classification Using Local Phase Quantization , 2008, ICISP.

[12]  Lakhmi C. Jain,et al.  Introduction to Local Binary Patterns: New Variants and Applications , 2013, Local Binary Patterns.

[13]  Wan Ling Chang,et al.  Interaction Expands Function: Social Shaping of the Therapeutic Robot PARO in a Nursing Home , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[14]  Takayuki Kanda,et al.  Interaction design for an enjoyable play interaction with a small humanoid robot , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[15]  Zheng-Hua Tan,et al.  Local feature learning for face recognition under varying poses , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[16]  Alexander H. Waibel,et al.  Enabling Multimodal Human–Robot Interaction for the Karlsruhe Humanoid Robot , 2007, IEEE Transactions on Robotics.

[17]  Marek P. Michalowski,et al.  A spatial model of engagement for a social robot , 2006, 9th IEEE International Workshop on Advanced Motion Control, 2006..

[18]  Kerstin Sophie Haring,et al.  Perception of a humanoid robot: A cross-cultural comparison , 2015, 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[19]  Danilo De Rossi,et al.  HEFES: An Hybrid Engine for Facial Expressions Synthesis to control human-like androids and avatars , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[20]  Justin Dauwels,et al.  Perception of Humanoid Social Mediator in Two-Person Dialogs , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[21]  P. Ekman.  An argument for basic emotions , 1992 .

[22]  J. Movellan,et al.  Ruby: A Robotic Platform for Real-time Social Interaction , 2004 .

[23]  Tetsuo Ono,et al.  Development and evaluation of interactive humanoid robots , 2004, Proceedings of the IEEE.

[24]  Patrick Kenny,et al.  Front-End Factor Analysis for Speaker Verification , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[25]  Evgenios Vlachos,et al.  An open-ended approach to evaluating Android faces , 2015, 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[26]  Hiroaki Kitano,et al.  Social Interaction of Humanoid RobotBased on Audio-Visual Tracking , 2002, IEA/AIE.

[27]  Jie Zhu,et al.  Head orientation and gaze direction in meetings , 2002, CHI Extended Abstracts.

[28]  Jonathan G. Fiscus,et al.  Darpa Timit Acoustic-Phonetic Continuous Speech Corpus CD-ROM {TIMIT} | NIST , 1993 .

[29]  Cynthia Breazeal,et al.  Persuasive Robotics: The influence of robot gender on human behavior , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  D. Feil-Seifer,et al.  Defining socially assistive robotics , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[31]  Giulio Sandini,et al.  The iCub humanoid robot: an open platform for research in embodied cognition , 2008, PerMIS.

[32]  Hiroaki Kitano,et al.  Human-robot interaction through real-time auditory and visual multiple-talker tracking , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[33]  Alexander H. Waibel,et al.  Natural human-robot interaction using speech, head pose and gestures , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[34]  Karl F. MacDorman,et al.  Revisiting the uncanny valley theory: Developing and validating an alternative to the Godspeed indices , 2010, Comput. Hum. Behav..

[35]  Gentiane Venture,et al.  Social vs. Useful HRI: Experiencing the Familiar, Perceiving the Robot as a Sociable Partner and Responding to Its Actions , 2013, Int. J. Soc. Robotics.

[36]  Elizabeth Ann Jochum,et al.  Using Theatre to Study Interaction with Care Robots , 2016, Int. J. Soc. Robotics.

[37]  Patrick Kenny,et al.  Eigenvoice modeling with sparse training data , 2005, IEEE Transactions on Speech and Audio Processing.

[38]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[39]  Evgenios Vlachos,et al.  Android Emotions Revealed , 2012, ICSR.

[40]  Bruce A. MacDonald,et al.  The Role of Healthcare Robots for Older People at Home: A Review , 2014, Int. J. Soc. Robotics.

[41]  Elizabeth S. Kim,et al.  Social Robots as Embedded Reinforcers of Social Behavior in Children with Autism , 2012, Journal of Autism and Developmental Disorders.

[42]  Dong-Wook Lee,et al.  Difference of Efficiency in Human-Robot Interaction According to Condition of Experimental Environment , 2012, ICSR.

[43]  Raquel Frizera Vassallo,et al.  Human–Robot Interaction and Cooperation Through People Detection and Gesture Recognition , 2013, Journal of Control, Automation and Electrical Systems.

[44]  R. Barber,et al.  Maggie: A Robotic Platform for Human-Robot Social Interaction , 2006, 2006 IEEE Conference on Robotics, Automation and Mechatronics.

[45]  K. Dautenhahn,et al.  A long-term study of children with autism playing with a robotic pet: Taking inspirations from non-directive play therapy to encourage children’s proactivity and initiative-taking , 2009 .

[46]  Jacob W. Crandall,et al.  Programming Robots to Express Emotions: Interaction Paradigms, Communication Modalities, and Context , 2014, IEEE Transactions on Human-Machine Systems.

[47]  Jonathan G. Fiscus,et al.  DARPA TIMIT:: acoustic-phonetic continuous speech corpus CD-ROM, NIST speech disc 1-1.1 , 1993 .

[48]  Dana Kulic,et al.  Measurement Instruments for the Anthropomorphism, Animacy, Likeability, Perceived Intelligence, and Perceived Safety of Robots , 2009, Int. J. Soc. Robotics.

[49]  Michael S. Brandstein,et al.  Robust Localization in Reverberant Rooms , 2001, Microphone Arrays.

[50]  Lakhmi C. Jain,et al.  Local Binary Patterns: New Variants and Applications , 2013, Local Binary Patterns.

[51]  Joseph H. DiBiase.  A High-Accuracy, Low-Latency Technique for Talker Localization in Reverberant Environments Using Microphone Arrays , 2000 .

[52]  Zheng-Hua Tan,et al.  Low-Complexity Variable Frame Rate Analysis for Speech Recognition and Voice Activity Detection , 2010, IEEE Journal of Selected Topics in Signal Processing.

[53]  T. Tsuji,et al.  Development of the Face Robot SAYA for Rich Facial Expressions , 2006, 2006 SICE-ICASE International Joint Conference.