Block world reconstruction from spherical stereo image pairs

We propose a 3D block-based scene reconstruction system using spherical images.Facade alignment and cubic projection produce central-point perspective images.Block representation is recovered based on optimal occupancy and physical stability.Optional user interaction to constrain primitive reconstruction is provided.Texture mapping from the original images gives a quick rendering of the scene. We propose a block-based scene reconstruction method using multiple stereo pairs of spherical images. We assume that the urban scene consists of axis-aligned planar structures (Manhattan world). Captured spherical stereo images are converted into six central-point perspective images by cubic projection and facade alignment. Depth information is recovered by stereo matching between images. Semantic regions are segmented based on colour, edge and normal information. Independent 3D rectangular planes are constructed by fitting planes aligned with the principal axes of the segmented 3D points. Finally cuboid-based scene structure is recovered from multiple viewpoints by merging and refining planes based on connectivity and visibility. The reconstructed model efficiently shows the structure of the scene with a small amount of data.

[1]  Nikos Paragios,et al.  Random Exploration of the Procedural Space for Single-View 3D Modeling of Buildings , 2011, International Journal of Computer Vision.

[2]  Suya You,et al.  3D Reconstruction of Urban Areas , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[3]  Andrea Fusiello,et al.  Photo-Consistent Planar Patches from Unstructured Cloud of Points , 2010, ECCV.

[4]  Katsushi Ikeuchi,et al.  Omnidirectional texturing based on robust 3D registration through Euclidean reconstruction from two spherical images , 2010, Comput. Vis. Image Underst..

[5]  Christian Früh,et al.  Google Street View: Capturing the World at Street Level , 2010, Computer.

[6]  Marc Pollefeys,et al.  A Patch Prior for Dense 3D Reconstruction in Man-Made Environments , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[7]  KimHansung,et al.  3D Scene Reconstruction from Multiple Spherical Stereo Pairs , 2013 .

[8]  Jan-Michael Frahm,et al.  Piecewise planar and non-planar stereo for urban scene reconstruction , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Alessandro De Gloria,et al.  An architectural approach to efficient 3D urban modeling , 2011, Comput. Graph..

[10]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Marc Pollefeys,et al.  Interactive 3D architectural modeling from unordered photo collections , 2008, SIGGRAPH 2008.

[12]  Jean-Philippe Pons,et al.  Robust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Richard Szeliski,et al.  Interactive 3D architectural modeling from unordered photo collections , 2008, SIGGRAPH Asia '08.

[14]  Jianxiong Xiao,et al.  Reconstructing the World’s Museums , 2012, International Journal of Computer Vision.

[15]  Jiri Matas,et al.  Robust Detection of Lines Using the Progressive Probabilistic Hough Transform , 2000, Comput. Vis. Image Underst..

[16]  Richard Szeliski,et al.  Piecewise planar stereo for image-based rendering , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[18]  Luc Van Gool,et al.  Procedural 3D Building Reconstruction Using Shape Grammars and Detectors , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[19]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[20]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[21]  Alexei A. Efros,et al.  Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics , 2010, ECCV.

[22]  Kang-Hyun Jo,et al.  Reconstruction of Planar Objects using the Properties of Plane and Vanishing Points from a Single Panoramic Image , 2006 .

[23]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  Mariette Yvinec,et al.  Surface Reconstruction from Multi-View Stereo , 2009 .

[25]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[26]  Jianxiong Xiao,et al.  Image-based street-side city modeling , 2009, SIGGRAPH 2009.

[27]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Andrea Fusiello,et al.  Robust Multiple Structures Estimation with J-Linkage , 2008, ECCV.

[29]  Ulrich Neumann,et al.  2.5D Dual Contouring: A Robust Approach to Creating Building Models from Aerial LiDAR Point Clouds , 2010, ECCV.

[30]  Frank Dellaert,et al.  Line-Based Structure from Motion for Urban Environments , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[31]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[32]  Martial Hebert,et al.  3DNN: 3D Nearest Neighbor , 2014, International Journal of Computer Vision.

[33]  Jana Kosecka,et al.  Piecewise planar city 3D modeling from street view panoramic sequences , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Daphna Weinshall,et al.  Realtime IBR with omnidirectional crossed-slits projection , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[35]  P. Sturm.  A method for 3D reconstruction of piecewise planar objects from single panoramic images , 2000, Proceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704).

[36]  Tsuhan Chen,et al.  Active learning for piecewise planar 3D reconstruction , 2011, CVPR 2011.

[37]  Adrian Hilton,et al.  Scene modelling from sparse 3D data , 2005, Image Vis. Comput..

[38]  H. Mayer,et al.  FINDING CUBOID-BASED BUILDING MODELS IN POINT CLOUDS , 2012 .

[39]  Tony Mullen.  Mastering Blender , 2009 .

[40]  Weihong Li,et al.  Lightweight 3D Modeling of Urban Buildings from Range Data , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[41]  Jana Kosecka,et al.  Piecewise planar city 3D modeling from street view panoramic sequences , 2009, CVPR.

[42]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[43]  Philip H. S. Torr,et al.  VideoTrace: rapid interactive scene modelling from video , 2007, SIGGRAPH 2007.

[44]  Jana Kosecka,et al.  Multi-view Superpixel Stereo in Urban Environments , 2010, International Journal of Computer Vision.

[45]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[46]  Luc Van Gool,et al.  Image-based procedural modeling of facades , 2007, SIGGRAPH 2007.

[47]  James M. Coughlan,et al.  Manhattan World: Orientation and Outlier Detection by Bayesian Inference , 2003, Neural Computation.

[48]  D. Eberly.  Least Squares Fitting of Data , 2016 .

[49]  J. M. P. van Waveren,et al.  Rendering massive virtual worlds , 2013, SIGGRAPH '13.

[50]  Alexei A. Efros,et al.  Recovering Surface Layout from an Image , 2007, International Journal of Computer Vision.

[51]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[52]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[53]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Adrian Hilton,et al.  Planar urban scene reconstruction from spherical images using facade alignment , 2013, IVMSP 2013.

[55]  Jean-Philippe Pons,et al.  Towards high-resolution large-scale multi-view stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.