Image-Based Visual Servoing With Light Field Cameras

This paper proposes the first derivation, implementation, and experimental validation of light field image-based visual servoing. Light field image Jacobians are derived based on a compact light field feature representation that is close to the form measured directly by light field cameras. We also enhance feature detection and correspondence by enforcing light field geometry constraints, and directly estimate the image Jacobian without knowledge of point depth. The proposed approach is implemented over a standard visual servoing control loop, and applied to a custom-mirror-based light field camera mounted on a robotic arm. Light field image-based visual servoing is then validated in both simulation and experiment. We show that the proposed method outperforms conventional monocular and stereo image-based visual servoing under field-of-view constraints and occlusions.

[1]  Yongtian Wang,et al.  Light field acquisition using a planar catadioptric system. , 2015, Optics express.

[2]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[3]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[4]  Leonard T. Bruton,et al.  A 4-D Dual-Fan Filter Bank for Depth Filtering in Light Fields , 2007, IEEE Transactions on Signal Processing.

[5]  Ryad Benosman,et al.  Plenoptic cameras in real-time robotics , 2013, Int. J. Robotics Res..

[6]  Peter I. Corke,et al.  Mirrored Light Field Video Camera Adapter , 2016, ArXiv.

[7]  Stefan B. Williams,et al.  Plenoptic flow: Closed-form visual odometry for light field cameras , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Markus Kächele,et al.  Design and Fabrication of Faceted Mirror Arrays for Light Field Capture , 2013, Comput. Graph. Forum.

[9]  Donald G. Dansereau,et al.  Plenoptic Signal Processing for Robust Vision in Field Robotics , 2013 .

[10]  Myung Jin Chung,et al.  Concepts of Augmented Image Space and Transformed Feature Space for Efficient Visual Servoing of an "Eye-in-Hand Robot" , 1991, Robotica.

[11]  Francois Chaumette,et al.  Potential problems of unstability and divergence in image-based and position-based visual servoing , 1999, 1999 European Control Conference (ECC).

[12]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[13]  Stefan B. Williams,et al.  Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  François Chaumette,et al.  Potential problems of stability and convergence in image-based and position-based visual servoing , 1997 .

[15]  Bastian Goldlücke,et al.  On Linear Structure from Motion for Light Field Cameras , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[17]  Seth Hutchinson,et al.  Visual Servo Control Part I: Basic Approaches , 2006 .

[18]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[19]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[20]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[21]  Alois Knoll,et al.  Uncalibrated 3D stereo image-based dynamic visual servoing for robot manipulators , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Peter I. Corke,et al.  Choice of image features for depth-axis control in image based visual servo control , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Christoph Walter,et al.  Enabling multi-purpose mobile manipulators: Localization of glossy objects using a light-field camera , 2015, 2015 IEEE 20th Conference on Emerging Technologies & Factory Automation (ETFA).