Using structural bootstrapping for object substitution in robotic executions of human-like manipulation tasks

In this work we address the problem of finding replacements of missing objects that are needed for the execution of human-like manipulation tasks. This is a usual problem that is easily solved by humans provided their natural knowledge to find object substitutions: using a knife as a screwdriver or a book as a cutting board. On the other hand, in robotic applications, objects required in the task should be included in advance in the problem definition. If any of these objects is missing from the scenario, the conventional approach is to manually redefine the problem according to the available objects in the scene. In this work we propose an automatic way of finding object substitutions for the execution of manipulation tasks. The approach uses a logic-based planner to generate a plan from a prototypical problem definition and searches for replacements in the scene when some of the objects involved in the plan are missing. This is done by means of a repository of objects and attributes with roles, which is used to identify the affordances of the unknown objects in the scene. Planning actions are grounded using a novel approach that encodes the semantic structure of manipulation actions. The system was evaluated in a KUKA arm platform for the task of preparing a salad with successful results.

[1]  Tobias Luksch,et al.  A Dynamical Systems Approach to Adaptive Sequencing of Movement Primitives , 2012, ROBOTIK.

[2]  Stefan Edelkamp,et al.  Automated Planning: Theory and Practice , 2007, Künstliche Intell..

[3]  Florentin Wörgötter,et al.  Fast self-supervised on-line training for object recognition specifically for robotic applications , 2015, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[4]  Justus H. Piater,et al.  Knowledge propagation and relation learning for predicting action effects , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Christopher W. Geib,et al.  Structural bootstrapping at the sensorimotor level for the fast acquisition of action knowledge for cognitive robots , 2013, 2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[6]  Fulvio Mastrogiovanni,et al.  Learning symbolic representations of actions from human demonstrations , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Carme Torras,et al.  Learning weakly correlated cause-effects for gardening with a cognitive system , 2014, Eng. Appl. Artif. Intell..

[8]  Minija Tamosiunaite,et al.  Joining Movement Sequences: Modified Dynamic Movement Primitives for Robotics Applications Exemplified on Handwriting , 2012, IEEE Transactions on Robotics.

[9]  Danica Kragic,et al.  Learning Task Models from Multiple Human Demonstrations , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[10]  Mark Chignell,et al.  Intelligent databases - object-oriented deductive hypermedia technologies , 1989 .

[11]  Justus H. Piater,et al.  Homogeneity analysis for object-action relation reasoning in kitchen scenarios , 2013, MLIS '13.

[12]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[13]  Eren Erdal Aksoy,et al.  Model-free incremental learning of the semantics of manipulation actions , 2015, Robotics Auton. Syst..

[14]  Eren Erdal Aksoy,et al.  Learning the semantics of object–action relations by observation , 2011, Int. J. Robotics Res..

[15]  Ales Ude,et al.  Trajectory generation from noisy positions of object features for teaching robot paths , 1993, Robotics Auton. Syst..

[16]  Paolo Traverso,et al.  Automated Planning: Theory & Practice , 2004 .

[17]  Fahiem Bacchus,et al.  A Knowledge-Based Approach to Planning with Incomplete Information and Sensing , 2002, AIPS.

[18]  Tae-Kyun Kim,et al.  A syntactic approach to robot imitation learning using probabilistic activity grammars , 2013, Robotics Auton. Syst..

[19]  Ales Ude,et al.  Toward a library of manipulation actions based on semantic object-action relations , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Gesellschaft für Klassifikation. Jahrestagung,et al.  Data Analysis, Machine Learning and Knowledge Discovery - Proceedings of the 36th Annual Conference of the Gesellschaft für Klassifikation e. V., Hildesheim, Germany, August 2012 , 2014, GfKl.

[21]  Rüdiger Dillmann,et al.  Incremental Learning of Tasks From User Demonstrations, Past Experiences, and Vocal Comments , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Yoshihiko Nakamura,et al.  Stochastic Model of Imitating a New Observed Motion Based on the Acquired Motion Primitives , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Hema Swetha Koppula,et al.  RoboBrain: Large-Scale Knowledge Engine for Robots , 2014, ArXiv.

[24]  Carme Torras,et al.  Efficient interactive decision-making framework for robotic applications , 2017, Artif. Intell..

[25]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Rüdiger Dillmann,et al.  Advances in Robot Programming by Demonstration , 2010, KI - Künstliche Intelligenz.