Humanoid robots communication with participants using sign language: An interaction based sign language game

This work presents the preliminary results of an ongoing project which aims to use humanoid robots as sign language tutors. The study mainly focuses on children who have some problems when they are communicating with other individuals such as hearing-impaired or autistic children. In this paper, an interactive game, which is based on sign language, between a humanoid robot and a human participant is introduced. The game consists of an imitation based learning phase where the signs are taught in the first step and they are tested in the second step within the frame of an interaction game. The goal of the interactive game is to reinforce the semantic meaning of the signs in a motivating and engaging way, as well as to test the learning performance of the participants. We aim to design a comfortable learning environment by using the humanoid robot as an educational medium. The game also improves the participants' imitation and turn-taking skills and teaches the semantic meanings of the signs.

[1]  Masayuki Inaba,et al.  Design and development of research platform for perception-action integration in humanoid robot: H6 , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[2]  Hatice Kose-Bagci,et al.  Gesture recognition for humanoid assisted interactive sign language tutoring , 2013, 2013 21st Signal Processing and Communications Applications Conference (SIU).

[3]  R. Barber,et al.  Maggie: A Robotic Platform for Human-Robot Social Interaction , 2006, 2006 IEEE Conference on Robotics, Automation and Mechatronics.

[4]  Hainan Zhao,et al.  Avatars based Chinese Sign Language Synthesis System , 2008, 2008 IEEE International Conference on Systems, Man and Cybernetics.

[5]  James Ohene-Djan,et al.  An adaptive WWW-based system to teach British sign language , 2005, Fifth IEEE International Conference on Advanced Learning Technologies (ICALT'05).

[6]  Samir I. Shaheen,et al.  Sign language recognition using a combination of new vision based features , 2011, Pattern Recognit. Lett..

[7]  Toshikazu Kawasaki,et al.  Design of prototype humanoid robotics platform for HRP , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Kerstin Dautenhahn,et al.  Robots as social mediators for children with Autism - A preliminary analysis comparing two different robotic platforms , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[9]  Andrea Lockerd Thomaz,et al.  Simon plays Simon says: The timing of turn-taking in an imitation game , 2011, 2011 RO-MAN.

[10]  Kikuo Fujimura,et al.  The intelligent ASIMO: system overview and integration , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Jannik Fritsch,et al.  Humanoid robot platform suitable for studying embodied interaction , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Ho-Sub Yoon,et al.  Visual Processing of Rock, Scissors, Paper Game for Human Robot Interaction , 2006, 2006 SICE-ICASE International Joint Conference.

[13]  T. Röfer,et al.  A Robust Closed-Loop Gait for the Standard Platform League Humanoid , 2009 .

[14]  Surendra Ranganath,et al.  Facial expressions in American sign language: Tracking and recognition , 2012, Pattern Recognit..

[15]  Pierre Blazevic,et al.  Mechatronic design of NAO humanoid , 2009, 2009 IEEE International Conference on Robotics and Automation.

[16]  Changchun Liu,et al.  Online Affect Detection and Robot Behavior Adaptation for Intervention of Children With Autism , 2008, IEEE Transactions on Robotics.

[17]  Liya Ding,et al.  Modelling and recognition of the linguistic components in American Sign Language , 2009, Image Vis. Comput..

[18]  Mohamed Jemni,et al.  Using ICT to Teach Sign Language , 2008, 2008 Eighth IEEE International Conference on Advanced Learning Technologies.

[19]  Hatice Kose-Bagci,et al.  Ispy-usign humanoid assisted interactive sign language tutoring games , 2013, 2013 IEEE RO-MAN.

[20]  S. Spitzer,et al.  14 – Play in Children with Autism: Structure and Experience , 2008 .

[21]  Rajkumar Palaniappan,et al.  A phoneme based sign language recognition system using 2D moment invariant interleaving feature and Neural Network , 2011, 2011 IEEE Student Conference on Research and Development.

[22]  Heung-Il Suk,et al.  Hand gesture recognition based on dynamic Bayesian network framework , 2010, Pattern Recognit..

[23]  Nicoletta Adamo-Villani,et al.  The MathSigner: an interactive learning tool for American sign language , 2004, Proceedings. Eighth International Conference on Information Visualisation, 2004. IV 2004..

[24]  Nikolaos G. Tsagarakis,et al.  Lower Body Design of the `iCub' a Human-baby like Crawling Robot , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.