Model-driven indoor scenes modeling from a single image

In this paper, we present a new approach of 3D indoor scenes modeling on single image. With a single input indoor image (including sofa, tea table, etc.), a 3D scene can be reconstructed using existing model library in two stages: image analysis and model retrieval. In the image analysis stage, we obtain the object information from input image using geometric reasoning technology combined with image segmentation method. In the model retrieval stage, line drawings are extracted from 2D objects and 3D models by using different line rendering methods. We exploit various tokens to represent local features and then organize them together as a star-graph to show a global description. Finally, by comparing similarity among the encoded line drawings, models are retrieved from the model library and then the scene is reconstructed. Experimental results show that, driven by the given model library, indoor scenes modeling from a single image could be achieved automatically and efficiently.

[1]  Ian D. Reid,et al.  Single View Metrology , 2000, International Journal of Computer Vision.

[2]  Adam Finkelstein,et al.  Suggestive contours for conveying shape , 2003, ACM Trans. Graph..

[3]  Leonidas J. Guibas,et al.  Estimating image depth using shape collections , 2014, ACM Trans. Graph..

[4]  Marc Alexa,et al.  Sketch-based shape retrieval , 2012, ACM Trans. Graph..

[5]  Dengsheng Zhang,et al.  A comparative study on shape retrieval using Fourier descriptiors with different shape signatures , 2001 .

[6]  Daniel Cohen-Or,et al.  3-Sweep , 2013, ACM Trans. Graph..

[7]  Daniel Cohen-Or,et al.  Sketch‐to‐Design: Context‐Based Part Assembly , 2012, Comput. Graph. Forum.

[8]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  Takeo Kanade,et al.  Geometric reasoning for single image structure recovery , 2009, CVPR.

[10]  Jianxiong Xiao,et al.  Localizing 3D cuboids in single-view images , 2012, NIPS.

[11]  Szymon Rusinkiewicz,et al.  Modeling by example , 2004, SIGGRAPH 2004.

[12]  Ligang Liu,et al.  Modeling by Drawing with Shadow Guidance , 2013, Comput. Graph. Forum.

[13]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Alireza Khotanzad,et al.  Invariant Image Recognition by Zernike Moments , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Joseph J. Lim,et al.  Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Ligang Liu,et al.  Photo-inspired model-driven 3D object modeling , 2011, ACM Trans. Graph..

[17]  Kun Zhou,et al.  Discriminative Sketch‐based 3D Model Retrieval via Robust Shape Matching , 2011, Comput. Graph. Forum.

[18]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[19]  Kang Chen,et al.  Automatic semantic modeling of indoor scenes from low-quality RGB-D data using contextual information , 2014, ACM Trans. Graph..

[20]  Thomas A. Funkhouser,et al.  Sketch-based search and composition of 3D models , 2008, SBM'08.

[21]  Seungyong Lee,et al.  Coherent line drawing , 2007, NPAR '07.

[22]  Kun Zhou,et al.  Interactive images , 2012, ACM Trans. Graph..

[23]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[24]  Shi-Min Hu,et al.  Structure recovery by part assembly , 2012, ACM Trans. Graph..

[25]  Takeo Kanade,et al.  Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces , 2010, NIPS.

[26]  Shi-Min Hu,et al.  Sketch2Scene: sketch-based co-retrieval and co-placement of 3D models , 2013, ACM Trans. Graph..

[27]  Leonidas J. Guibas,et al.  Probabilistic reasoning for assembly-based 3D modeling , 2011, SIGGRAPH 2011.

[28]  Gang Hua,et al.  Picking the best DAISY , 2009, CVPR.

[29]  Shi-Min Hu,et al.  Qualitative organization of collections of shapes via quartet analysis , 2013, ACM Trans. Graph..

[30]  Kun Zhou,et al.  An interactive approach to semantic modeling of indoor scenes with an RGBD camera , 2012, ACM Trans. Graph..

[31]  Bo Li,et al.  A comparison of methods for sketch-based 3D shape retrieval , 2014, Comput. Vis. Image Underst..

[32]  Ralph R. Martin,et al.  PatchNet: a patch-based image representation for interactive library-driven image editing , 2013, ACM Trans. Graph..

[33]  Ke Xie,et al.  A search-classify approach for cluttered indoor scene understanding , 2012, ACM Trans. Graph..

[34]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[35]  Shaogang Gong,et al.  Sketch Recognition by Ensemble Matching of Structured Features , 2013, BMVC.

[36]  Daniel Cohen-Or,et al.  Projective analysis for 3D shape segmentation , 2013, ACM Trans. Graph..

[37]  Alexei A. Efros,et al.  Seeing 3D Chairs: Exemplar Part-Based 2D-3D Alignment Using a Large Dataset of CAD Models , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.