Lifting 3D Manhattan Lines from a Single Image

We propose a novel and an efficient method for reconstructing the 3D arrangement of lines extracted from a single image, using vanishing points, orthogonal structure, and an optimization procedure that considers all plausible connectivity constraints between lines. Line detection identifies a large number of salient lines that intersect or nearly intersect in an image, but relatively a few of these apparent junctions correspond to real intersections in the 3D scene. We use linear programming (LP) to identify a minimal set of least-violated connectivity constraints that are sufficient to unambiguously reconstruct the 3D lines. In contrast to prior solutions that primarily focused on well-behaved synthetic line drawings with severely restricting assumptions, we develop an algorithm that can work on real images. The algorithm produces line reconstruction by identifying 95% correct connectivity constraints in York Urban database, with a total computation time of 1 second per image.

[1]  Lawrence G. Roberts,et al.  Machine Perception of Three-Dimensional Solids , 1963, Outstanding Dissertations in the Computer Sciences.

[2]  K. Sugihara.  Machine interpretation of line drawings , 1986, MIT Press series in artificial intelligence.

[3]  Hans-Peter Seidel,et al.  Exploiting global connectivity constraints for reconstruction of 3D line segments from images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Alexei A. Efros,et al.  Recovering Surface Layout from an Image , 2007, International Journal of Computer Vision.

[5]  Jaishanker K. Pillai,et al.  Manhattan Junction Catalogue for Spatial Reasoning of Indoor Scenes , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  David L. Waltz,et al.  Generating Semantic Descriptions From Drawings of Scenes With Shadows , 1972 .

[7]  Jana Kosecka,et al.  Detection and matching of rectilinear structures , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Peter Ashley Clifford Varley,et al.  Automatic creation of boundary-representation models from single line drawings , 2003 .

[9]  T. Kanade,et al.  Geometric reasoning for single image structure recovery , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Alan L. Yuille,et al.  Manhattan World: compass direction from a single image by Bayesian inference , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  D. A. Huffman,et al.  Impossible Objects as Nonsense Sentences , 2012 .

[12]  Ashutosh Saxena,et al.  3-D Depth Reconstruction from a Single Still Image , 2007, International Journal of Computer Vision.

[13]  Alexei A. Efros,et al.  People Watching: Human Actions as a Cue for Single View Geometry , 2012, International Journal of Computer Vision.

[14]  Ian D. Reid,et al.  Manhattan scene understanding using monocular, stereo, and 3D features , 2011, 2011 International Conference on Computer Vision.

[15]  Takeo Kanade,et al.  Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces , 2010, NIPS.

[16]  Feng Han,et al.  Bottom-Up/Top-Down Image Parsing with Attribute Grammar , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Frédo Durand,et al.  Shapecollage: Occlusion-Aware, Example-Based Shape Interpretation , 2012, ECCV.

[18]  Daniel G. Aliaga,et al.  Building reconstruction using manhattan-world grammars , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  S. Sutherland.  Seeing things , 1989, Nature.

[20]  Takeo Kanade,et al.  A Theory of Origami World , 1979, Artif. Intell..

[21]  Walter Whiteley,et al.  A matroid on hypergraphs, with applications in scene analysis and geometry , 1989, Discret. Comput. Geom..

[22]  Jitendra Malik,et al.  Interpreting line drawings of curved objects , 1986, International Journal of Computer Vision.

[23]  Stephen J. Maybank,et al.  A Method for Interactive 3D Reconstruction of Piecewise Planar Objects from Single Images , 1999, BMVC.

[24]  Wei Zhang,et al.  Video Compass , 2002, ECCV.

[25]  Jitendra Malik,et al.  Inferring spatial layout from a single image via depth-ordered grouping , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[26]  Marc Pollefeys,et al.  Efficient structured prediction for 3D indoor scene understanding , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  James H. Elder,et al.  Efficient Edge-Based Methods for Estimating Manhattan Frames in Urban Imagery , 2008, ECCV.

[28]  Derek Hoiem,et al.  Recovering the spatial layout of cluttered rooms , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[29]  Alexei A. Efros,et al.  Automatic photo pop-up , 2005, ACM Trans. Graph..

[30]  Ian D. Reid,et al.  Single View Metrology , 2000, International Journal of Computer Vision.

[31]  Honglak Lee,et al.  Automatic Single-Image 3d Reconstructions of Indoor Manhattan World Scenes , 2007, ISRR.

[32]  Alexei A. Efros,et al.  Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics , 2010, ECCV.