Visual Predictive Control of Robot Manipulators Using a 3D ToF Camera

This paper presents a visual servo strategy for an anthropomorphic manipulator with eye-to-hand configuration. As novelty, a time-of-flight (ToF) camera, together with a model-based predictive control (MPC), have been used to execute the manipulator tasks in an unstructured environment. The ToF camera provides 3-D data of a target object and the MPC cost function achieves convergence of robot motion. With this configuration, there is no requirement to estimate a 3-D feature point position from a 2-D image and the camera model. A procedure for the target segmentation on the depth data, which is based on a previous training, is also presented. Experimental results, obtained in a robotic platform, show that the proposed approach is well suited for visual servoing purposes.

[1]  Estelle Courtial,et al.  Visual predictive control for manipulators with catadioptric camera , 2008, 2008 IEEE International Conference on Robotics and Automation.

[2]  Eduardo F. Camacho,et al.  Model Predictive Controllers , 2007 .

[3]  John C. Russ,et al.  The Image Processing Handbook , 2016, Microscopy and Microanalysis.

[4]  François Chaumette,et al.  A New Formulation for Non-Linear Camera Calibration Using Virtual Visual Servoing , 2001 .

[5]  Ernst Jonsson.  Engine Testing by Minicomputer , 1973 .

[6]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[7]  Seth Hutchinson,et al.  Visual Servo Control Part I: Basic Approaches , 2006 .

[8]  Stephan Hussmann,et al.  Pseudo-Four-Phase-Shift Algorithm for Performance Enhancement of 3D-TOF Vision Systems , 2010, IEEE Transactions on Instrumentation and Measurement.

[9]  T. Martin McGinnity,et al.  A position based visual tracking system for a 7 DOF robot manipulator using a Kinect camera , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[10]  Joris De Schutter,et al.  Specification of force-controlled actions in the "task frame formalism"-a synthesis , 1996, IEEE Trans. Robotics Autom..

[11]  Jorge Pomares,et al.  Visual Control of Robots Using Range Images , 2010, Sensors.

[12]  Corneliu Lazar,et al.  Predictive control of nonlinear visual servoing systems using image moments , 2012 .

[13]  Ulrich Reiser,et al.  USING A 3D TIME-OF-FLIGHT RANGE CAMERA FOR VISUAL TRACKING , 2007 .

[14]  Gerd Hirzinger,et al.  Extrinsic and depth calibration of ToF-cameras , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[16]  Éric Marchand,et al.  Feature tracking for visual servoing purposes , 2005, Robotics Auton. Syst..

[17]  S. Foix,et al.  Lock-in Time-of-Flight (ToF) Cameras: A Survey , 2011, IEEE Sensors Journal.