Omnidirectional image capture on mobile devices for fast automatic generation of 2.5D indoor maps

We introduce a light-weight automatic method to quickly capture and recover 2.5D multi-room indoor environments scaled to real-world metric dimensions. To minimize the user effort required, we capture and analyze a single omni-directional image per room using widely available mobile devices. Through a simple tracking of the user movements between rooms, we iterate the process to map and reconstruct entire floor plans. In order to infer 3D clues with a minimal processing and without relying on the presence of texture or detail, we define a specialized spatial transform based on catadioptric theory to highlight the room's structure in a virtual projection. From this information, we define a parametric model of each room to formalize our problem as a global optimization solved by Levenberg-Marquardt iterations. The effectiveness of the method is demonstrated on several challenging real-world multi-room indoor scenes.

[1]  Burcu Akinci,et al.  Automatic Creation of Semantically Rich 3D Building Models from Laser Scanner Data , 2011 .

[2]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[3]  P. Sturm.  A method for 3D reconstruction of piecewise planar objects from single panoramic images , 2000, Proceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704).

[4]  智一 吉田,et al.  Efficient Graph-Based Image Segmentationを用いた圃場図自動作成手法の検討 , 2014 .

[5]  John J. Leonard,et al.  Kintinuous: Spatially Extended KinectFusion , 2012, AAAI 2012.

[6]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[7]  Akihiro Sugimoto,et al.  A Flexible Scene Representation for 3D Reconstruction Using an RGB-D Camera , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[9]  Derek Hoiem,et al.  Recovering the spatial layout of cluttered rooms , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  Ian D. Reid,et al.  Manhattan scene understanding using monocular, stereo, and 3D features , 2011, 2011 International Conference on Computer Vision.

[11]  Steven M. Seitz,et al.  Capturing indoor scenes with smartphones , 2012, UIST.

[12]  Krista A. Ehinger,et al.  Recognizing scene viewpoint using panoramic place representation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Ricardo Cabral,et al.  Piecewise Planar and Compact Floorplan Reconstruction from Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Hui Zhang,et al.  Modeling room structure from indoor panorama , 2014, VRCAI '14.

[15]  Marcel Worring,et al.  A semi-interactive panorama based 3D reconstruction framework for indoor scenes , 2011, Comput. Vis. Image Underst..

[16]  T. Kanade,et al.  Geometric reasoning for single image structure recovery , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Josechu J. Guerrero,et al.  Hypercatadioptric line images for 3D orientation and image rectification , 2012, Robotics Auton. Syst..

[18]  Manfred Lau,et al.  Democratizing Digital Content Creation Using Mobile Devices with Inbuilt Sensors , 2015, IEEE Computer Graphics and Applications.

[19]  Kostas Daniilidis,et al.  A Unifying Theory for Central Panoramic Systems and Practical Applications , 2000, ECCV.

[20]  Wolfram Burgard,et al.  Learning compact 3D models of indoor and outdoor environments with a mobile robot , 2003, Robotics Auton. Syst..

[21]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.

[22]  William Ribarsky,et al.  Interactive analysis and visualization of situationally aware building evacuations , 2015, Inf. Vis..

[23]  Ian D. Reid,et al.  A Dynamic Programming Approach to Reconstructing Building Interiors , 2010, ECCV.

[24]  Young Min Kim,et al.  Interactive acquisition of residential floor plans , 2012, 2012 IEEE International Conference on Robotics and Automation.

[25]  Avideh Zakhor,et al.  Planar 3D modeling of building interiors from point cloud data , 2012, 2012 19th IEEE International Conference on Image Processing.

[26]  Renato Pajarola,et al.  Robust Reconstruction of Interior Building Structures with Multiple Rooms under Clutter and Occlusions , 2013, 2013 International Conference on Computer-Aided Design and Computer Graphics.

[27]  Renato Pajarola,et al.  Automatic room detection and reconstruction in cluttered indoor environments with complex room layouts , 2014, Comput. Graph..

[28]  F. Tarsha-Kurdi,et al.  Hough-Transform and Extended RANSAC Algorithms for Automatic Detection of 3D Building Roof Planes from Lidar Data , 2007 .

[29]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[30]  Burcu Akinci,et al.  Automatic Creation of Semantically Rich 3D Building Models from Laser Scanner Data , 2013 .

[31]  Josechu J. Guerrero,et al.  Scene structure recovery from a single omnidirectional image , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[32]  Enrico Gobbetti,et al.  Effective mobile mapping of multi-room indoor structures , 2014, The Visual Computer.

[33]  Marc Pollefeys,et al.  Real-Time Direct Dense Matching on Fisheye Images Using Plane-Sweeping Stereo , 2014, 2014 2nd International Conference on 3D Vision.