Enriched manipulation action semantics for robot execution of time constrained tasks

This paper contributes to semantic representation of human demonstrated actions for robot execution of time constrained tasks. We propose a semantic action encoding method based on interactions between the subject and objects in the scene. Our semantic framework is enriched with a descriptive spatial reasoning method which leads to accurate segmentation and recognition of unique action primitives. The proposed framework can classify the segmented action primitives as periodic or discrete to allow robots to autonomously decide how to imitate observed actions at different temporal scales. We evaluated our framework on our new large manipulation action dataset which involves in total 70 demonstrations from 8 different action types. We conducted various experiments with our humanoid robot to evaluate the proposed framework.

[1]  Minija Tamosiunaite,et al.  Joining Movement Sequences: Modified Dynamic Movement Primitives for Robotics Applications Exemplified on Handwriting , 2012, IEEE Transactions on Robotics.

[2]  Yiannis Aloimonos,et al.  Learning the spatial semantics of manipulation actions through preposition grounding , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Wei Liang,et al.  What Is Where: Inferring Containment Relations from Videos , 2016, IJCAI.

[4]  Tobias Luksch,et al.  A Dynamical Systems Approach to Adaptive Sequencing of Movement Primitives , 2012, ROBOTIK.

[5]  Eren Erdal Aksoy,et al.  Semantic parsing of human manipulation activities using on-line learned models for robot imitation , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Dana Kulic,et al.  Scaffolding on-line segmentation of full body human motion patterns , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Eren Erdal Aksoy,et al.  Using structural bootstrapping for object substitution in robotic executions of human-like manipulation tasks , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Jernej Barbic,et al.  Segmenting Motion Capture Data into Distinct Behaviors , 2004, Graphics Interface.

[9]  Maja J. Mataric,et al.  Automated Derivation of Primitives for Movement Classification , 2000, Auton. Robots.

[10]  Anthony G. Cohn,et al.  Learning Functional Object-Categories from a Relational Spatio-Temporal Representation , 2008, ECAI.

[11]  Eren Erdal Aksoy,et al.  Learning the semantics of object–action relations by observation , 2011, Int. J. Robotics Res..

[12]  Yoshihiko Nakamura,et al.  Stochastic Model of Imitating a New Observed Motion Based on the Acquired Motion Primitives , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[14]  Tamim Asfour,et al.  Unifying Representations and Large-Scale Whole-Body Motion Databases for Studying Human Motion , 2016, IEEE Transactions on Robotics.

[15]  Byoung-Tak Zhang,et al.  Enhancing human action recognition through spatio-temporal feature learning and semantic rules , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[16]  C. V. Jawahar,et al.  Learning support order for manipulation in clutter , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Stefan Ulbrich,et al.  Master Motor Map (MMM) — Framework and toolkit for capturing, representing, and reproducing human motion on humanoid robots , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[18]  Benjamin Z. Yao,et al.  Learning and parsing video events with goal and intent prediction , 2013, Comput. Vis. Image Underst..

[19]  Stefan Schaal,et al.  Movement segmentation using a primitive library , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Tamim Asfour,et al.  ARMAR-III: An Integrated Humanoid Platform for Sensory-Motor Control , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[21]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Germain Forestier,et al.  Unsupervised Trajectory Segmentation for Surgical Gesture Recognition in Robotic Training , 2016, IEEE Transactions on Biomedical Engineering.

[23]  Eamonn J. Keogh,et al.  Exact indexing of dynamic time warping , 2002, Knowledge and Information Systems.

[24]  Ales Ude,et al.  Trajectory generation from noisy positions of object features for teaching robot paths , 1993, Robotics Auton. Syst..