Image-consistent patches from unstructured points with J-linkage

Going from unstructured cloud of points to surfaces is a challenging problem. However, as points are produced by a structure-and-motion pipeline, image-consistency is a powerful clue that comes to the rescue. In this paper we present a method for extracting planar patches from an unstructured cloud of points, based on the detection of image-consistent planar patches with J-linkage, a robust algorithm for multiple model fitting. The method integrates several constraints inside J-linkage, optimizes the position of the points with regard to image-consistency and deploys a hierarchical processing scheme that decreases the computational load. With respect to previous work this approach has the advantage of starting from sparse data. Several results show the effectiveness of the proposed approach.

[1]  Pascal Fua,et al.  Dynamic and scalable large scale image reconstruction , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Michael Goesele,et al.  Multi-View Stereo for Community Photo Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[3]  Adrien Bartoli,et al.  A random sampling strategy for piecewise planar scene segmentation , 2007, Comput. Vis. Image Underst..

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  Peter F. Sturm,et al.  Using geometric constraints through parallelepipeds for calibration and 3D modeling , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Nora Ripperda.  EXTRACTION OF FAÇADES USING RJMCMC AND CONSTRAINT EQUATIONS , 2006 .

[7]  Tat-Jun Chin,et al.  The Random Cluster Model for robust geometric fitting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Andrea Fusiello,et al.  Robust Multiple Structures Estimation with J-Linkage , 2008, ECCV.

[9]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[10]  Jan-Michael Frahm,et al.  Piecewise planar and non-planar stereo for urban scene reconstruction , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Takeo Kanade,et al.  Image-consistent surface triangulation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[12]  Olga Veksler,et al.  Fast Fusion Moves for Multi-model Estimation , 2012, ECCV.

[13]  Jana Kosecka,et al.  Nonparametric Estimation of Multiple Structures with Outliers , 2006, WDV.

[14]  Peter Meer,et al.  Nonlinear Mean Shift for Clustering over Analytic Manifolds , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  Andrea Fusiello,et al.  Automatic Estimation of the Inlier Threshold in Robust Multiple Structures Fitting , 2009, ICIAP.

[16]  Charles V. Stewart,et al.  Bias in robust estimation caused by discontinuities and multiple structures , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Kenichi Kanatani,et al.  Optimizing a Triangular Mesh for Shape Reconstruction from Images , 2005, IEICE Trans. Inf. Syst..

[18]  Adrian Hilton,et al.  Scene modelling from sparse 3D data , 2005, Image Vis. Comput..

[19]  Tat-Jun Chin,et al.  Multi-structure model selection via kernel optimisation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Philip H. S. Torr,et al.  VideoTrace: rapid interactive scene modelling from video , 2007, SIGGRAPH 2007.

[21]  Jan-Michael Frahm,et al.  RECON: Scale-adaptive robust estimation via Residual Consensus , 2011, 2011 International Conference on Computer Vision.

[22]  Philip H. S. Torr.  An assessment of information criteria for motion model selection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Jitendra Malik,et al.  Reconstructing Polyhedral Models of Architectural Scenes from Photographs , 1996, ECCV.

[24]  Neill W. Campbell,et al.  Automatic Augmentation and Meshing of Sparse 3D Scene Structure , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[25]  Feng Han,et al.  Bayesian reconstruction of 3D shapes and scenes from a single image , 2003, First IEEE International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, 2003. HLK 2003..

[26]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[27]  Andrea Fusiello,et al.  Efficient Visualization of Architectural Models from a Structure and Motion Pipeline , 2008, Eurographics.

[28]  Hans-Peter Seidel,et al.  Multiresolution hierarchies on unstructured triangle meshes , 1999, Comput. Geom..

[29]  Konrad Schindler,et al.  A model-based method for building reconstruction , 2003, First IEEE International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, 2003. HLK 2003..

[30]  Andrea Fusiello,et al.  Improving the efficiency of hierarchical structure-and-motion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Roberto Cipolla,et al.  PhotoBuilder-3D models of architectural scenes from uncalibrated images , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[32]  Jean-Philippe Pons,et al.  Robust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  R. Duin,et al.  The dissimilarity representation for pattern recognition , a tutorial , 2009 .

[34]  Qing Zhu,et al.  Knowledge-based topological reconstruction for building façade surface patches , 2009 .

[35]  Erkki Oja,et al.  A new curve detection method: Randomized Hough transform (RHT) , 1990, Pattern Recognit. Lett..

[36]  Marc Pollefeys,et al.  A Patch Prior for Dense 3D Reconstruction in Man-Made Environments , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[37]  Andrea Fusiello,et al.  Photo-Consistent Planar Patches from Unstructured Cloud of Points , 2010, ECCV.

[38]  Roberto Cipolla,et al.  Modelling and Interpretation of Architecture from Several Images , 2004, International Journal of Computer Vision.

[39]  Tat-Jun Chin,et al.  Robust fitting of multiple structures: The statistical learning approach , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[40]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  B. S. Manjunath,et al.  The multiRANSAC algorithm and its application to detect planar homographies , 2005, IEEE International Conference on Image Processing 2005.

[42]  Luc Van Gool,et al.  Image-based procedural modeling of facades , 2007, SIGGRAPH 2007.

[43]  Hiroshi Kawakami,et al.  Detection of Planar Regions with Uncalibrated Stereo using Distributions of Feature Points , 2004, BMVC.

[44]  Alan Ryan,et al.  From left to right , 1997 .