On the manipulation of articulated objects in human-robot cooperation scenarios

Articulated and flexible objects constitute a challenge for robot manipulation tasks but are present in different real-world settings, including home and industrial environments. Current approaches to the manipulation of articulated and flexible objects employ ad hoc strategies to sequence and perform actions on them depending on a number of physical or geometrical characteristics related to those objects, as well as on an a priori classification of target object configurations. In this paper, we propose an action planning and execution framework, which (i) considers abstract representations of articulated or flexible objects, (ii) integrates action planning to reason upon such configurations and to sequence an appropriate set of actions with the aim of obtaining a target configuration provided as a goal, and (iii) is able to cooperate with humans to collaboratively carry out the plan. On the one hand, we show that a trade-off exists between the way articulated or flexible objects are perceived and how the system represents them. Such a trade-off greatly impacts on the complexity of the planning process. On the other hand, we demonstrate the system's capabilities in allowing humans to interrupt robot action execution, and - in general - to contribute to the whole manipulation process. Results related to planning performance are discussed, and examples of a Baxter dual-arm manipulator performing actions collaboratively with humans are shown.

[1]  Pieter Abbeel,et al.  Combined task and motion planning through an extensible planner-independent interface layer , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Yu Zhang,et al.  AI Challenges in Human-Robot Cognitive Teaming , 2017, ArXiv.

[3]  Fulvio Mastrogiovanni,et al.  Semantic-Aware Real-Time Scheduling in Robotics , 2013, IEEE Transactions on Robotics.

[4]  Mozafar Saadat,et al.  Industrial applications of automatic manipulation of flexible materials , 2002 .

[5]  Alexander Verl,et al.  Cooperation of human and machines in assembly lines , 2009 .

[6]  Brian Scassellati,et al.  Transparent role assignment and task allocation in human robot collaboration , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Rachid Alami,et al.  Dealing with On-line Human-Robot Negotiations in Hierarchical Agent-Based Task Planner , 2017, ICAPS.

[8]  Helge J. Ritter,et al.  Bi-manual robotic paper manipulation based on real-time marker tracking and physical modelling , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Diego Calvanese,et al.  The Description Logic Handbook: Theory, Implementation, and Applications , 2003, Description Logic Handbook.

[10]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[11]  Ian Horrocks,et al.  A Description Logic Primer , 2012, ArXiv.

[12]  Julie A. Shah,et al.  Fast Scheduling of Multi-Robot Teams with Temporospatial Constraints , 2013, Robotics: Science and Systems.

[13]  Mariano Alcañiz Raya,et al.  Real-time deformable models for surgery simulation: a survey , 2005, Comput. Methods Programs Biomed..

[14]  Il Hong Suh,et al.  Planning of proactive behaviors for human-robot cooperative tasks under uncertainty , 2014, Knowl. Based Syst..

[15]  Craig A. Knoblock,et al.  PDDL-the planning domain definition language , 1998 .

[16]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[17]  Sotiris Makris,et al.  Human–robot interaction review and challenges on task planning and programming , 2016, Int. J. Comput. Integr. Manuf..

[18]  Danica Kragic,et al.  Dual arm manipulation - A survey , 2012, Robotics Auton. Syst..

[19]  Minoru Asada,et al.  Initiative in robot assistance during collaborative task execution , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[20]  George A. Bekey,et al.  Recursive learning for deformable object manipulation , 1997, 1997 8th International Conference on Advanced Robotics. Proceedings. ICAR'97.

[21]  Julie A. Shah,et al.  Decision-making authority, team efficiency and human worker satisfaction in mixed human–robot teams , 2015, Auton. Robots.

[22]  Natàlia Hurtós,et al.  ROSPlan: Planning in the Robot Operating System , 2015, ICAPS.

[23]  Dmitry Berenson,et al.  Manipulation of deformable objects without modeling and simulating deformation , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  J. Lygeros,et al.  Decision Making I , 2014 .

[25]  Fulvio Mastrogiovanni,et al.  Learning symbolic representations of actions from human demonstrations , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Sami Haddadin,et al.  Physical Human-Robot Interaction , 2016, Springer Handbook of Robotics, 2nd Ed..

[27]  Anca D. Dragan,et al.  Goal Inference Improves Objective and Perceived Performance in Human-Robot Collaboration , 2016, AAMAS.

[28]  Manuel Lopes,et al.  Impact of Robot Initiative on Human-Robot Collaboration , 2017, HRI.

[29]  Fulvio Mastrogiovanni,et al.  Automated Planning Techniques for Robot Manipulation Tasks Involving Articulated Objects , 2017, AI*IA.

[30]  Carme Torras,et al.  Integrating Task Planning and Interactive Learning for Robots to Work in Human Environments , 2011, IJCAI.

[31]  Hidefumi Wakamatsu,et al.  Knotting/Unknotting Manipulation of Deformable Linear Objects , 2006, Int. J. Robotics Res..

[32]  Heinz Wörn,et al.  Robot Manipulation of Deformable Objects: Advanced Manufacturing , 2000 .

[33]  Pieter Abbeel,et al.  Learning from Demonstrations Through the Use of Non-rigid Registration , 2013, ISRR.

[34]  Lihui Wang,et al.  Gesture recognition for human-robot collaboration: A review , 2017, International Journal of Industrial Ergonomics.

[35]  Maria Fox,et al.  Validating Plans in the Context of Processes and Exogenous Events , 2005, AAAI.

[36]  Erez Karpas,et al.  Robust Execution of Plans for Human-Robot Teams , 2015, ICAPS.

[37]  Kerstin Dautenhahn,et al.  Socially intelligent robots: dimensions of human–robot interaction , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[38]  SaffiottiAlessandro,et al.  Human-aware task planning , 2010 .

[39]  Ian Horrocks,et al.  OWL rules: A proposal and prototype implementation , 2005, J. Web Semant..

[40]  Alberto Finzi,et al.  Flexible Task Execution and Attentional Regulations in Human-Robot Interaction , 2017, IEEE Transactions on Cognitive and Developmental Systems.

[41]  Hema Swetha Koppula,et al.  Anticipatory Planning for Human-Robot Teams , 2014, ISER.

[42]  Reinhard Koch,et al.  An Adaptable Robot Vision System Performing Manipulation Actions With Flexible Objects , 2014, IEEE Transactions on Automation Science and Engineering.

[43]  Fulvio Mastrogiovanni,et al.  Towards the creation of tactile maps for robots and their use in robot contact motion control , 2015, Robotics Auton. Syst..

[44]  F. Mastrogiovanni,et al.  A System for Hierarchical Planning in Service Mobile Robotics , 2004 .

[45]  Diego Calvanese,et al.  The description logic handbook: theory , 2003 .

[46]  Jitendra Malik,et al.  Combining self-supervised learning and imitation for vision-based rope manipulation , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[47]  Maria Fox,et al.  Modelling Mixed Discrete-Continuous Domains for Planning , 2006, J. Artif. Intell. Res..

[48]  Hector Geffner,et al.  Searching for Plans with Carefully Designed Probes , 2011, ICAPS.

[49]  Trevor Darrell,et al.  A geometric approach to robotic laundry folding , 2012, Int. J. Robotics Res..

[50]  Helge J. Ritter,et al.  Folding paper with anthropomorphic robot hands using real-time physics-based modeling , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[51]  Sami Haddadin,et al.  A Hierarchical Human-Robot Interaction-Planning Framework for Task Allocation in Collaborative Industrial Assembly Processes , 2017, IEEE Robotics and Automation Letters.

[52]  I. ClintHeyer Human-Robot Interaction and Future Industrial Robotics Applications , 2010 .

[53]  Sami Haddadin,et al.  A Hierarchical Human-Robot Interaction-Planning Framework for Task Allocation in Collaborative Industrial Assembly Processes , 2017, ICRA 2017.

[54]  Fulvio Mastrogiovanni,et al.  Flexible human-robot cooperation models for assisted shop-floor tasks , 2017, ArXiv.

[55]  Stefanos Nikolaidis,et al.  Optimization of Temporal Dynamics for Adaptive Human-Robot Interaction in Assembly Manufacturing , 2012, Robotics: Science and Systems.

[56]  P. Jiménez,et al.  Survey on model-based manipulation planning of deformable objects , 2012 .

[57]  Maja J. Mataric,et al.  How Robot Verbal Feedback Can Improve Team Performance in Human-Robot Task Collaborations , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[58]  Heinz Wörn,et al.  Robot Manipulation of Deformable Objects: Advanced Manufacturing , 2000 .

[59]  Julie A. Shah,et al.  Coordination of Human-Robot Teaming with Human Task Preferences , 2015, AAAI Fall Symposia.

[60]  Linda R. Elliott,et al.  Managing workload in human-robot interaction: A review of empirical studies , 2010, Comput. Hum. Behav..

[61]  Wolfram Burgard,et al.  Learning the elasticity parameters of deformable objects with a manipulation robot , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[62]  Masatoshi Ishikawa,et al.  Dynamic High-Speed Knotting of a Rope by a Manipulator , 2013 .

[63]  J. Rintanen Madagascar : Scalable Planning with SAT , 2014 .

[64]  Maria Fox,et al.  PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains , 2003, J. Artif. Intell. Res..