A Model Predictive Control Approach for Vision-Based Object Grasping via Mobile Manipulator

This paper presents the design of a vision-based object grasping and motion control architecture for a mobile manipulator system. The optimal grasping areas of the object are estimated using the partial point cloud acquired from an onboard RGB-D sensor system. The reach-to-grasp motion of the mobile manipulator is handled via a Nonlinear Model Predictive Control scheme. The controller is formulated accordingly in order to allow the system to operate in a constrained workspace with static obstacles. The goal of the proposed scheme is to guide the robot's end-effector towards the optimal grasping regions with guaranteed input and state constraints such as occlusion and obstacle avoidance, workspace boundaries and field of view constraints. The performance of the proposed strategy is experimentally verified using an 8 Degrees of Freedom KUKA Youbot in different reach-to-grasp scenarios.

[1]  Ying Wang,et al.  A Hybrid Visual Servo Controller for Robust Grasping by Wheeled Mobile Robots , 2010, IEEE/ASME Transactions on Mechatronics.

[2]  Antonio Bicchi,et al.  On the Closure Properties of Robotic Grasping , 1995, Int. J. Robotics Res..

[3]  Tatsuo Arai,et al.  Real-time trajectory planning for mobile manipulator using model predictive control with constraints , 2011, 2011 8th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[4]  Rüdiger Dillmann,et al.  Visual servoing for humanoid grasping and manipulation tasks , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[5]  John F. Canny,et al.  Easily computable optimum grasps in 2-D and 3-D , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[6]  Andrea Maria Zanchettin,et al.  Constraint-based Model Predictive Control for holonomic mobile manipulators , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Andrea Maria Zanchettin,et al.  Constrained model predictive control for mobile robotic manipulators , 2017, Robotica.

[8]  Martijn Wisse,et al.  Fast grasping of unknown objects using force balance optimization , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Dmitry Berenson,et al.  Humanoid motion planning for dual-arm manipulation and re-grasping tasks , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Tamim Asfour,et al.  Integrated Grasp and motion planning , 2010, 2010 IEEE International Conference on Robotics and Automation.

[11]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Kostas J. Kyriakopoulos,et al.  Model Predictive Control for the navigation of a nonholonomic vehicle with field-of-view constraints , 2013, 2013 American Control Conference.

[13]  H.G. Cai,et al.  Grasping unknown objects based on 3d model reconstruction , 2005, Proceedings, 2005 IEEE/ASME International Conference on Advanced Intelligent Mechatronics..

[14]  Gary M. Bone,et al.  Automated modeling and robotic grasping of unknown three-dimensional objects , 2008, 2008 IEEE International Conference on Robotics and Automation.

[15]  Erwin Prassler,et al.  KUKA youBot - a mobile manipulator for research and education , 2011, 2011 IEEE International Conference on Robotics and Automation.

[16]  Jeffrey C. Trinkle,et al.  Grasp analysis as linear matrix inequality problems , 2000, IEEE Trans. Robotics Autom..

[17]  António Paulo Moreira,et al.  Multi-Robot nonlinear model predictive formation control: the obstacle avoidance problem , 2014, Robotica.

[18]  Anis Sahbani,et al.  An overview of 3D object grasp synthesis algorithms , 2012, Robotics Auton. Syst..

[19]  Jun Wang,et al.  Synthesis of force-closure grasps on 3-D objects based on the Q distance , 2003, IEEE Trans. Robotics Autom..

[20]  Sukhan Lee,et al.  VIsion force control in task-oriented grasping and manipulation , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Matei T. Ciocarlie,et al.  Contact-reactive grasping of objects with partial shape information , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Danica Kragic,et al.  Selection of robot pre-grasps using box-based shape approximation , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Martijn Wisse,et al.  Grasping of unknown objects via curvature maximization using active vision , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Máximo A. Roa,et al.  Integrated grasp and motion planning using independent contact regions , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[25]  Matei T. Ciocarlie,et al.  Data-driven grasping with partial sensor data , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Henrik I. Christensen,et al.  Automatic grasp planning using shape primitives , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[27]  G. Seliger,et al.  Opportunities of Sustainable Manufacturing in Industry 4.0 , 2016 .

[28]  Alexei Makarenko,et al.  Randomised MPC-based motion-planning for mobile robot obstacle avoidance , 2009, 2009 IEEE International Conference on Robotics and Automation.

[29]  D. Koditschek,et al.  Robot navigation functions on manifolds with boundary , 1990 .

[30]  Kostas J. Kyriakopoulos,et al.  A robust self triggered Image Based Visual Servoing Model Predictive Control scheme for small autonomous robots , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[31]  Vijay Kumar,et al.  Robotic grasping and contact: a review , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[32]  António Paulo Moreira,et al.  Multi-robot nonlinear model predictive formation control: Moving target and target absence , 2013, Robotics Auton. Syst..

[33]  F. Allgöwer,et al.  Nonlinear Model Predictive Control: From Theory to Application , 2004 .