Plan3D

We introduce a new method that efficiently computes a set of viewpoints and trajectories for high-quality 3D reconstructions in outdoor environments. Our goal is to automatically explore an unknown area and obtain a complete 3D scan of a region of interest (e.g., a large building). Images from a commodity RGB camera, mounted on an autonomously navigated quadcopter, are fed into a multi-view stereo reconstruction pipeline that produces high-quality results but is computationally expensive. In this setting, the scanning result is constrained by the restricted flight time of quadcopters. To this end, we introduce a novel optimization strategy that respects these constraints by maximizing the information gain from sparsely sampled viewpoints while limiting the total travel distance of the quadcopter. At the core of our method lies a hierarchical volumetric representation that allows the algorithm to distinguish between unknown, free, and occupied space. Furthermore, our information gain-based formulation leverages this representation to handle occlusions in an efficient manner. In addition to the surface geometry, we utilize free-space information to avoid obstacles and determine collision-free flight paths. Our tool can be used to specify the region of interest and to plan trajectories. We demonstrate our method by obtaining a number of compelling 3D reconstructions, and we provide a thorough quantitative evaluation showing improvement over previous state-of-the-art and regular patterns.

[1]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[2]  Daniel Cremers,et al.  Dense Tracking and Mapping with a Quadrocopter , 2013 .

[3]  Leif Kobbelt,et al.  Image selection for improved Multi-View Stereo , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Roland Siegwart,et al.  Vision-Controlled Micro Flying Robots: From System Design to Autonomous Navigation and Mapping in GPS-Denied Environments , 2014, IEEE Robotics & Automation Magazine.

[5]  Clive S. Fraser Network design considerations for non-topographic photogrammetry , 1994 .

[6]  Richard Szeliski,et al.  Building Rome in a day , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[7]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[8]  Andreas Krause,et al.  Submodular Function Maximization , 2014, Tractability.

[9]  Wei Sun,et al.  Autoscanning for coupled scene reconstruction and proactive object analysis , 2015, ACM Trans. Graph..

[10]  Laurence A. Wolsey,et al.  An analysis of the greedy algorithm for the submodular set covering problem , 1982, Comb..

[11]  Michael Goesele,et al.  Let There Be Color! Large-Scale Texturing of 3D Reconstructions , 2014, ECCV.

[12]  ARNO KNAPITSCH,et al.  Tanks and temples , 2017, ACM Trans. Graph..

[13]  S. Mason Heuristic reasoning strategy for automated sensor placement , 1997 .

[14]  Horst Bischof,et al.  Photogrammetric Camera Network Design for Micro Aerial Vehicles , 2012 .

[15]  Pushmeet Kohli,et al.  Tractability: Practical Approaches to Hard Problems , 2013 .

[16]  Wolfram Burgard,et al.  OctoMap: an efficient probabilistic 3D mapping framework based on octrees , 2013, Autonomous Robots.

[17]  R. C. Coulter,et al.  Implementation of the Pure Pursuit Path Tracking Algorithm , 1992 .

[18]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[19]  Kiriakos N. Kutulakos,et al.  Recovering shape by purposive viewpoint adjustment , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Chandra Chekuri,et al.  A recursive greedy algorithm for walks in directed graphs , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[21]  Daniel Cohen-Or,et al.  Quality-driven poisson-guided autoscanning , 2014, ACM Trans. Graph..

[22]  Jiawen Chen,et al.  Scalable real-time volumetric surface reconstruction , 2013, ACM Trans. Graph..

[23]  Torsten Sattler,et al.  A Multi-view Stereo Benchmark with High-Resolution Images and Multi-camera Videos , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Danping Zou,et al.  Active Image-Based Modeling with a Toy Drone , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Matthias Nießner,et al.  BundleFusion , 2016, TOGS.

[26]  M. Goesele,et al.  Floating scale surface reconstruction , 2014, ACM Trans. Graph..

[27]  Sang Wook Lee,et al.  View Selection Strategies for Multi-View, Wide-Baseline Stereo , 1994 .

[28]  Mateu Sbert,et al.  Automatic View Selection Using Viewpoint Entropy and its Application to Image‐Based Modelling , 2003, Comput. Graph. Forum.

[29]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[30]  Michael M. Kazhdan,et al.  Screened poisson surface reconstruction , 2013, TOGS.

[31]  Marc Pollefeys,et al.  Vision-based autonomous mapping and exploration using a quadrotor MAV , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[34]  Tomás Pajdla,et al.  Multi-view reconstruction preserving weakly-supported surfaces , 2011, CVPR 2011.

[35]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Simon Fuhrmann,et al.  MVE - A Multi-View Reconstruction Environment , 2014, GCH.

[37]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[38]  Yevgeniy Vorobeychik,et al.  Submodular Optimization with Routing Constraints , 2016, AAAI.

[39]  Richard Szeliski,et al.  Bundle Adjustment in the Large , 2010, ECCV.

[40]  Alejandro F. Frangi,et al.  Image-based modelling , 2014 .

[41]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[42]  Szymon Rusinkiewicz,et al.  Automated view and path planning for scalable multi-object 3D scanning , 2016, ACM Trans. Graph..

[43]  Jan-Michael Frahm,et al.  Pixelwise View Selection for Unstructured Multi-View Stereo , 2016, ECCV.

[44]  Michael Suppa,et al.  Efficient next-best-scan planning for autonomous 3D surface reconstruction of unknown objects , 2015, Journal of Real-Time Image Processing.

[45]  Michael Goesele,et al.  Multi-View Stereo Revisited , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[46]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Roland Siegwart,et al.  Structural inspection path planning via iterative viewpoint resampling with application to aerial robotics , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[48]  Davide Scaramuzza,et al.  Appearance-based Active, Monocular, Dense Reconstruction for Micro Aerial Vehicles , 2014, Robotics: Science and Systems.

[49]  Horst Bischof,et al.  Online Feedback for Structure-from-Motion Image Acquisition , 2012, BMVC.

[50]  Weihua Sheng,et al.  Design and Evaluation of a Teleoperated Robotic 3-D Mapping System using an RGB-D Sensor , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[51]  Vijay Kumar,et al.  Cooperative localization and mapping of MAVs using RGB-D sensors , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[52]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[53]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Michael Goesele,et al.  Multi-View Stereo for Community Photo Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[55]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[56]  Brian Yamauchi,et al.  A frontier-based approach for autonomous exploration , 1997, Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97. 'Towards New Computational Principles for Robotics and Automation'.

[57]  David Fofi,et al.  An efficient method for fully automatic 3D digitization of unknown objects , 2013, Comput. Ind..

[58]  Pat Hanrahan,et al.  Submodular Trajectory Optimization for Aerial 3D Scanning , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[59]  Anders Heyden,et al.  Covariance Propagation and Next Best View Planning for 3D Reconstruction , 2012, ECCV.

[60]  Danping Zou,et al.  Active Image-based Modeling , 2017, ArXiv.

[61]  Nicolas Pugeault,et al.  Taking the Scenic Route to 3D: Optimising Reconstruction from Moving Cameras , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[62]  Heinrich Niemann,et al.  Active Visual Object Reconstruction using D-, E-, and T-Optimal Next Best Views , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Shengyong Chen,et al.  Active vision in robotic systems: A survey of recent developments , 2011, Int. J. Robotics Res..

[64]  Marc Pollefeys,et al.  Real-time photo-realistic 3D mapping for micro aerial vehicles , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[65]  Vijay Kumar,et al.  Autonomous multi-floor indoor navigation with a computationally constrained MAV , 2011, 2011 IEEE International Conference on Robotics and Automation.

[66]  Matthias Nießner,et al.  Real-time 3D reconstruction at scale using voxel hashing , 2013, ACM Trans. Graph..

[67]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[68]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[69]  Roger Mohr,et al.  Optimal camera placement for accurate reconstruction , 2002, Pattern Recognit..

[70]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[71]  Jan-Michael Frahm,et al.  Next Best View Planning for Active Model Improvement , 2009, BMVC.

[72]  Michael Goesele,et al.  Shading-Aware Multi-view Stereo , 2016, ECCV.

[73]  Johannes L. Schönberger,et al.  Supplementary Material for A MultiView Stereo Benchmark with High-Resolution Images and Multi-Camera Videos , 2017 .

[74]  Simon Fuhrmann,et al.  MVE - An image-based reconstruction environment , 2015, Comput. Graph..