Understanding a child's play for robot interaction by sequencing play primitives using Hidden Markov Models

In this paper, we discuss a methodology to build a system for a robot playmate that extracts and sequences low-level play primitives during a robot-child interaction scenario. The motivation is to provide a robot with basic knowledge of how to manipulate toys in an equivalent manner as a human does - as a first step in engaging children in cooperative play. Our approach involves the extraction of play primitives based on observation of motion gradient vectors computed from the image sequence. Hidden Markov Models (HMMs) are then used to recognize 14 different play primitives during play. Experimental results from a data set of 100 play scenarios including child subjects demonstrate 86.88% accuracy recognizing and sequencing the play primitives.

[1]  Ramesh C. Jain,et al.  Recursive identification of gesture inputs using hidden Markov models , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[2]  Junji Yamato,et al.  Recognizing human action in time-sequential images using hidden Markov model , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Yasushi Yagi,et al.  Gesture recognition using colored gloves , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[4]  Tracie Pasold,et al.  Violence exposure in real-life, video games, television, movies, and the internet: is there desensitization? , 2004, Journal of adolescence.

[5]  M. Meindl,et al.  PlayROB - robot-assisted playing for children with severe physical disabilities , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[6]  Yangsheng Xu,et al.  Online, interactive learning of gestures for human/robot interfaces , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[7]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  J. Piaget.  Play, dreams and imitation in childhood , 1951 .

[9]  François Michaud,et al.  Exploring the use of a mobile robot as an imitation agent with children with low-functioning autism , 2008, Auton. Robots.

[10]  Masayuki Inaba,et al.  Learning by watching: extracting reusable task knowledge from visual observation of human performance , 1994, IEEE Trans. Robotics Autom..

[11]  K. Ginsburg.  The Importance of Play in Promoting Healthy Child Development and Maintaining Strong Parent-Child Bonds , 2007, Pediatrics.

[12]  G. Baranek,et al.  Object play in infants with autism: methodological issues in retrospective video analysis. , 2005, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[13]  Ayanna M. Howard,et al.  Extracting play primitives for a robot playmate by sequencing low-level motion behaviors , 2008, RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication.

[14]  J. Moor.  Playing, Laughing and Learning with Children on the Autism Spectrum: A Practical Resource of Play Ideas for Parents and Carers , 2002 .

[15]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[16]  L. R. Huesmann,et al.  The role of media violence in violent behavior. , 2006, Annual review of public health.

[17]  Brian Scassellati,et al.  How Social Robots Will Help Us to Diagnose, Treat, and Understand Autism , 2005, ISRR.

[18]  Alex Pentland,et al.  Space-time gestures , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[20]  A. Pellegrini,et al.  Physical activity play: the nature and function of a neglected aspect of playing. , 1998, Child development.

[21]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[22]  Jessica K. Hodgins,et al.  Generalizing Demonstrated Manipulation Tasks , 2002, WAFR.

[23]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[24]  E. Klugman,et al.  Building a Play Research Agenda: What Do We Know About Play? What New Questions Do We Need to Ask? , 2007 .

[25]  Katsushi Ikeuchi,et al.  Toward automatic robot instruction from perception-mapping human grasps to manipulator grasps , 1997, IEEE Trans. Robotics Autom..

[26]  A. Billard,et al.  Building Robota, a Mini-Humanoid Robot for the Rehabilitation of Children With Autism , 2007, Assistive technology : the official journal of RESNA.