3D visual sensing technique based on focal stack for snake robotic applications

Abstract In this paper, we proposed a passive 3D visual sensing technique based on the focal stack and designed the snake robot’s 3D visual sensing prototype. The proposed technique consists of depth reconstruction, all-in-focus imaging and 3D point cloud generation based on the theoretical model of light field reconstruction from the focal stack. The 3D visual sensing prototype was realized by the focal stack acquisition system controlled by the servo motor. In the experiment, the depth map with high precision, all-in-focus image with high spatial resolution as well as the 3D point cloud can be reconstructed in the field-of-view (FOV) by the 3D visual sensing prototype. The proposed technique could be further applied to efficient visual navigation, simultaneous-localization-and-mapping (SLAM) and visual servoing of the robot.

[1]  Øyvind Stavdahl,et al.  Perception-Driven Obstacle-Aided Locomotion for Snake Robots: The State of the Art, Challenges and Possibilities † , 2017 .

[2]  Andrew Lumsdaine,et al.  Focused plenoptic camera and rendering , 2010, J. Electronic Imaging.

[3]  Shree K. Nayar,et al.  Ieee Transactions on Image Processing Computational Cameras: Convergence of Optics and Processing , 2022 .

[4]  Shree K. Nayar,et al.  Flexible Depth of Field Photography , 2008, ECCV.

[5]  Stefan B. Williams,et al.  Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Jitendra Malik,et al.  Depth from shading, defocus, and correspondence using light-field angular coherence , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Toshiaki Fujii,et al.  From Focal Stack to Tensor Light-Field Display , 2018, IEEE Transactions on Image Processing.

[8]  M. Levoy,et al.  The light field , 1939 .

[9]  Shree K. Nayar,et al.  Programmable Aperture Camera Using LCoS , 2010, IPSJ Trans. Comput. Vis. Appl..

[10]  Alexei A. Efros,et al.  Occlusion-Aware Depth Estimation Using Light-Field Cameras , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[12]  Peter Corke,et al.  An Introduction to Inertial and Visual Sensing , 2007, Int. J. Robotics Res..

[13]  A. Kak,et al.  Simultaneous Algebraic Reconstruction Technique (SART): A Superior Implementation of the Art Algorithm , 1984, Ultrasonic imaging.

[14]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[15]  L. Landweber.  An iteration formula for Fredholm integral equations of the first kind , 1951 .

[16]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Chang Liu,et al.  Iterative reconstruction of scene depth with fidelity based on light field data. , 2017, Applied optics.

[18]  Marc Levoy,et al.  Light Fields and Computational Imaging , 2006, Computer.

[19]  Pål Liljebäck,et al.  A review on modelling, implementation, and control of snake robots , 2012, Robotics Auton. Syst..

[20]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[21]  Yongtian Wang,et al.  Design of an optical see-through light-field near-eye display using a discrete lenslet array. , 2018, Optics express.

[22]  Jie Chen,et al.  Light Field Compression With Disparity-Guided Sparse Coding Based on Structural Key Views , 2016, IEEE Transactions on Image Processing.

[23]  Tom E. Bishop,et al.  Full-Resolution Depth Map Estimation from an Aliased Plenoptic Light Field , 2010, ACCV.

[24]  Kazunori Ohno,et al.  Real-Time Robot Trajectory Estimation and 3D Map Construction using 3D Camera , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Peter Cave,et al.  Biologically Inspired Robots: Serpentile Locomotors and Manipulators , 1993 .

[26]  Ren Ng.  Fourier slice photography , 2005, ACM Trans. Graph..

[27]  Øyvind Stavdahl,et al.  Snake Robots: Modelling, Mechatronics, and Control , 2012 .

[28]  M. Murugan,et al.  Modular Snake Robot with Mapping and Navigation: Urban Search and Rescue (USAR) Robot , 2015, 2015 International Conference on Computing Communication Control and Automation.

[29]  Ming Jiang,et al.  Light field reconstruction from projection modeling of focal stack. , 2017, Optics express.

[30]  Shugen Ma,et al.  Influence of two SLAM algorithms using serpentine locomotion in a featureless environment , 2015, 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[31]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[32]  Kazuo Tanaka,et al.  Range-Sensor-Based Semiautonomous Whole-Body Collision Avoidance of a Snake Robot , 2015, IEEE Transactions on Control Systems Technology.