Wireframe Parsing With Guidance of Distance Map

We propose an end-to-end method for simultaneously detecting local junctions and global wireframe in man-made environment. Our pipeline consists of an anchor-free junction detection module, a distance map learning module, and a line segment proposing and verification module. A set of line segments are proposed from the predicted junctions with guidance of the learned distance map, and further verified by the proposal verification module. Experimental results show that our method outperforms the previous state-of-the-art wireframe parser by a descent margin. In terms of line segments detection, our method shows competitive performance on standard benchmarks. The proposed networks are end-to-end trainable and efficient.<xref ref-type="fn" rid="fn1"><sup>a</sup></xref><fn id="fn1"><label><sup>a</sup></label><p>The code will be released on github for reproduction of the results.</p></fn>

[1]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[3]  Yunchao Wei,et al.  Proposal-Free Network for Instance-Level Object Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Ashutosh Saxena,et al.  3-D Depth Reconstruction from a Single Still Image , 2007, International Journal of Computer Vision.

[5]  Andrew Calway,et al.  Recognising Planes in a Single Image , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Honglak Lee,et al.  Automatic Single-Image 3d Reconstructions of Indoor Manhattan World Scenes , 2007, ISRR.

[7]  Martial Hebert,et al.  Data-Driven 3D Primitives for Single Image Understanding , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Min Bai,et al.  Deep Watershed Transform for Instance Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Josh H. McDermott,et al.  Psychophysics with junctions in real images. , 2010, Perception.

[10]  Martial Hebert,et al.  Unfolding an Indoor Origami World , 2014, ECCV.

[11]  Hui Zhang,et al.  Efficient 3D Room Shape Recovery from a Single Panorama , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Zihan Zhou,et al.  Recovering 3D Planes from a Single Image via Convolutional Neural Networks , 2018, ECCV.

[13]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Jaishanker K. Pillai,et al.  Manhattan Junction Catalogue for Spatial Reasoning of Indoor Scenes , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  James H. Elder,et al.  MCMLSD: A Dynamic Programming Approach to Line Segment Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Ian D. Reid,et al.  Manhattan scene understanding using monocular, stereo, and 3D features , 2011, 2011 International Conference on Computer Vision.

[17]  Yoshihisa Shinagawa,et al.  Accurate and robust line segment extraction by analyzing distribution around peaks in Hough space , 2003, Comput. Vis. Image Underst..

[18]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[20]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Gui-Song Xia,et al.  Learning Attraction Field Representation for Robust Line Segment Detection , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Saeid Nahavandi,et al.  Intelligent Line Segment Perception With Cortex-Like Mechanisms , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[23]  Shenghua Gao,et al.  PPGNet: Learning Point-Pair Graph for Line Segment Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Rob Fergus,et al.  Depth Map Prediction from a Single Image using a Multi-Scale Deep Network , 2014, NIPS.

[25]  Bok-Suk Shin,et al.  Accurate and Robust Line Segment Extraction Using Minimum Entropy With Hough Transform , 2015, IEEE Transactions on Image Processing.

[26]  Jiri Matas,et al.  Robust Detection of Lines Using the Progressive Probabilistic Hough Transform , 2000, Comput. Vis. Image Underst..

[27]  Narciso García,et al.  Line segment detection using weighted mean shift procedures on a 2D slice sampling strategy , 2011, Pattern Analysis and Applications.

[28]  Varun Ramakrishna,et al.  Convolutional Pose Machines , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[31]  Matthew Brand,et al.  Lifting 3D Manhattan Lines from a Single Image , 2013, 2013 IEEE International Conference on Computer Vision.

[32]  Jonathan Tompson,et al.  PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model , 2018, ECCV.

[33]  Jitendra Malik,et al.  Using contours to detect and localize junctions in natural images , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  T. Kanade,et al.  Geometric reasoning for single image structure recovery , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Luc Van Gool,et al.  Convolutional Oriented Boundaries , 2016, ECCV.

[36]  James H. Elder,et al.  Efficient Edge-Based Methods for Estimating Manhattan Frames in Urban Imagery , 2008, ECCV.

[37]  Xiang Bai,et al.  Richer Convolutional Features for Edge Detection , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Yi Ma,et al.  End-to-End Wireframe Parsing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[39]  Ahmed M. Elgammal,et al.  Line-based relative pose estimation , 2011, CVPR 2011.

[40]  Kun Huang,et al.  Learning to Parse Wireframes in Images of Man-Made Environments , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  David Windridge,et al.  A generalisable framework for saliency-based line segment detection , 2015, Pattern Recognit..

[42]  Derek Hoiem,et al.  Predicting Complete 3D Models of Indoor Scenes , 2015, ArXiv.

[43]  Silvio Savarese,et al.  DeLay: Robust Spatial Layout Estimation for Cluttered Indoor Scenes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Julie Delon,et al.  Accurate Junction Detection and Characterization in Natural Images , 2013, International Journal of Computer Vision.

[45]  Rafael Grompone von Gioi,et al.  LSD: A Fast Line Segment Detector with a False Detection Control , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Ersin Yumer,et al.  PlaneNet: Piece-Wise Planar Reconstruction from a Single RGB Image , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[47]  C. Lawrence Zitnick,et al.  Structured Forests for Fast Edge Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[48]  C.-C. Jay Kuo,et al.  A Coarse-to-Fine Indoor Layout Estimation (CFILE) Method , 2016, ACCV.

[49]  Li Li,et al.  CannyLines: A parameter-free line segment detector , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[50]  Svetlana Lazebnik,et al.  Learning Informative Edge Maps for Indoor Scene Layout Prediction , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).