Towards an Intelligent Collaborative Robotic System for Mixed Case Palletizing

In this paper, a novel human-robot collaborative framework for mixed case palletizing is presented. The framework addresses several challenges associated with the detection and localisation of boxes and pallets through visual perception algorithms, high-level optimisation of the collaborative effort through effective role-allocation principles, and maximisation of packing density. A graphical user interface (GUI) is additionally developed to ensure an intuitive allocation of roles and the optimal placement of the boxes on target pallets. The framework is evaluated in two conditions where humans operate with and without the support of a Mobile COllaborative robotic Assistant (MOCA). The results show that the optimised placement can improve up to the 20% with respect to a manual execution of the same task, and reveal the high potential of MOCA in increasing the performance of collaborative palletizing tasks.

[1]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[2]  Christian Ott,et al.  Cartesian Impedance Control of Redundant and Flexible-Joint Robots , 2008, Springer Tracts in Advanced Robotics.

[3]  Daniele Vigo,et al.  The Three-Dimensional Bin Packing Problem , 2000, Oper. Res..

[4]  Oussama Khatib,et al.  Vehicle/arm coordination and multiple mobile manipulator decentralized cooperation , 1996, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS '96.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Fei Zhao,et al.  A Teleoperation Interface for Loco-Manipulation Control of Mobile Collaborative Robotic Assistant , 2019, IEEE Robotics and Automation Letters.

[7]  Christian Wurll.  Mixed Case Palletizing with Industrial Robots , 2016 .

[8]  Elena De Momi,et al.  An Intuitive Augmented Reality Interface for Task Scheduling, Monitoring, and Work Performance Improvement in Human-Robot Collaboration , 2019, 2019 IEEE International Work Conference on Bioinspired Intelligence (IWOBI).

[9]  Francisco José Madrid-Cuevas,et al.  Generation of fiducial marker dictionaries using Mixed Integer Linear Programming , 2016, Pattern Recognit..

[10]  Nikos G. Tsagarakis,et al.  Towards Robot Interaction Autonomy: Explore, Identify, and Interact , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[11]  Dmitry Berenson,et al.  An optimization approach to planning for mobile manipulation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[12]  Kazuhiro Kosuge,et al.  Progress and prospects of the human–robot collaboration , 2017, Autonomous Robots.

[13]  Nikolaos G. Tsagarakis,et al.  Multi-Priority Cartesian Impedance Control Based on Quadratic Programming Optimization , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Arash Ajoudani,et al.  A Capability-Aware Role Allocation Approach to Industrial Assembly Tasks , 2019, IEEE Robotics and Automation Letters.

[15]  Antonio Bicchi,et al.  On making robots understand safety: Embedding injury knowledge into control , 2012, Int. J. Robotics Res..

[16]  Rafael Muñoz-Salinas,et al.  Speeded up detection of squared fiducial markers , 2018, Image Vis. Comput..

[17]  Raffaello D'Andrea,et al.  Coordinating Hundreds of Cooperative, Autonomous Vehicles in Warehouses , 2007, AI Mag..

[18]  Johannes Fottner,et al.  Real-time Pallet Localization with 3D Camera Technology for Forklifts in Logistic Environments , 2018, 2018 IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI).

[19]  Debjit Roy,et al.  Robotized and Automated Warehouse Systems: Review and Recent Developments , 2019, Transp. Sci..

[20]  O. Brock,et al.  Elastic Strips: A Framework for Motion Generation in Human Environments , 2002, Int. J. Robotics Res..

[21]  Jan Riehme,et al.  An efficient approach for the multi-pallet loading problem , 2000, Eur. J. Oper. Res..

[22]  Vincent Lepetit,et al.  Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes , 2011, 2011 International Conference on Computer Vision.

[23]  Oussama Khatib,et al.  Inertial Properties in Robotic Manipulation: An Object-Level Framework , 1995, Int. J. Robotics Res..