Depth Sensing Beyond LiDAR Range

Depth sensing is a critical component of autonomous driving technologies, but today’s LiDAR- or stereo camera– based solutions have limited range. We seek to increase the maximum range of self-driving vehicles’ depth perception modules for the sake of better safety. To that end, we propose a novel three-camera system that utilizes small field of view cameras. Our system, along with our novel algorithm for computing metric depth, does not require full pre-calibration and can output dense depth maps with practically acceptable accuracy for scenes and objects at long distances not well covered by most commercial LiDARs.

[1]  Fisher Yu,et al.  3D Reconstruction from Accidental Motion , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Kyungdon Joo,et al.  High Quality Structure from Small Motion for Rolling Shutter Cameras , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[4]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[5]  Richard Szeliski,et al.  Shape Ambiguities in Structure From Motion , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Michael Happold,et al.  Hierarchical Deep Stereo Matching on High-Resolution Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Tae-Hyun Oh,et al.  A Closed-Form Solution to Rotation Estimation for Structure from Small Motion , 2018, IEEE Signal Processing Letters.

[8]  Jan-Michael Frahm,et al.  Pixelwise View Selection for Unstructured Multi-View Stereo , 2016, ECCV.

[9]  Charles T. Loop,et al.  Computing rectifying homographies for stereo vision , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[10]  Kyungdon Joo,et al.  Accurate 3D Reconstruction from Small Motion Clip for Rolling Shutter Cameras , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[12]  Richard I. Hartley,et al.  Theory and Practice of Projective Rectification , 1999, International Journal of Computer Vision.

[13]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[14]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[15]  Ruigang Yang,et al.  Calibrating Pan-Tilt Cameras with Telephoto Lenses , 2007, ACCV.

[16]  Christian Beder,et al.  Determining an Initial Image Pair for Fixing the Scale of a 3D Reconstruction from an Image Sequence , 2006, DAGM-Symposium.

[17]  In-So Kweon,et al.  High-Quality Depth from Uncalibrated Small Motion Clip , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Richard Szeliski,et al.  Building Rome in a day , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[19]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[20]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .