A computational model of bounded developable surfaces with application to image‐based three‐dimensional reconstruction

Developable surfaces have been extensively studied in computer graphics because they are involved in a large body of applications. This type of surfaces has also been used in computer vision and document processing in the context of three‐dimensional (3D) reconstruction for book digitization and augmented reality. Indeed, the shape of a smoothly deformed piece of paper can be very well modeled by a developable surface. Most of the existing developable surface parameterizations do not handle boundaries or are driven by overly large parameter sets. These two characteristics become issues in the context of developable surface reconstruction from real observations. Our main contribution is a generative model of bounded developable surfaces that solves these two issues. Our model is governed by intuitive parameters whose number depends on the actual deformation and including the “flat shape boundary”. A vast majority of the existing image‐based paper 3D reconstruction methods either require a tightly controlled environment or restricts the set of possible deformations. We propose an algorithm for reconstructing our model's parameters from a general smooth 3D surface interpolating a sparse cloud of 3D points. The latter is assumed to be reconstructed from images of a static piece of paper or any other developable surface. Our 3D reconstruction method is well adapted to the use of keypoint matches over multiple images. In this context, the initial 3D point cloud is reconstructed by structure‐from‐motion for which mature and reliable algorithms now exist and the thin‐plate spline is used as a general smooth surface model. After initialization, our model's parameters are refined with model‐based bundle adjustment. We experimentally validated our model and 3D reconstruction algorithm for shape capture and augmented reality on seven real datasets. The first six datasets consist of multiple images or videos and a sparse set of 3D points obtained by structure‐from‐motion. The last dataset is a dense 3D point cloud acquired by structured light. Our implementation has been made publicly available on the authors' web home pages. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  D. Struik.  Lectures on classical differential geometry , 1951 .

[2]  D. Hilbert,et al.  Geometry and the Imagination , 1953 .

[3]  Jean Duchon,et al.  Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces , 1976 .

[4]  Carl de Boor,et al.  A Practical Guide to Splines , 1978, Applied Mathematical Sciences.

[5]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Günter Aumann,et al.  Interpolation with developable Bézier patches , 1991, Comput. Aided Geom. Des..

[7]  Otto Röschel,et al.  Developable (1, n) - Bézier surfaces , 1992, Comput. Aided Geom. Des..

[8]  Tosiyasu L. Kunii,et al.  Bending and creasing virtual paper , 1994, IEEE Computer Graphics and Applications.

[9]  Evangelos E. Milios,et al.  Optimal spline fitting to planar shape , 1994, Signal Process..

[10]  Meng Sun,et al.  A Technique for Constructing Developable Surfaces , 1996, Graphics Interface.

[11]  Sethna,et al.  Acoustic emission from crumpling paper. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[12]  Jeanny Hérault,et al.  Curvilinear component analysis: a self-organizing neural network for nonlinear mapping of data sets , 1997, IEEE Trans. Neural Networks.

[13]  Helmut Pottmann,et al.  Approximation of developable surfaces with cone spline surfaces , 1998, Comput. Aided Des..

[14]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[15]  Johannes Wallner,et al.  On Surface Approximation Using Developable Surfaces , 1999, Graph. Model. Image Process..

[16]  Johannes Wallner,et al.  Approximation algorithms for developable surfaces , 1999, Comput. Aided Geom. Des..

[17]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[18]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[19]  A. Pressley.  Elementary Differential Geometry , 2000 .

[20]  Ivan Poupyrev,et al.  The MagicBook: a transitional AR interface , 2001, Comput. Graph..

[21]  H. Pottmann,et al.  Computational Line Geometry , 2001 .

[22]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[23]  Mohammed Bennamoun,et al.  Automatic Bayesian knot placement for spline fitting , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[24]  Chih-Hsing Chu,et al.  Developable Bézier patches: properties and design , 2002, Comput. Aided Des..

[25]  Helmut Pottmann,et al.  Approximation with active B-spline curves and surfaces , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[26]  Günter Aumann,et al.  A simple algorithm for designing developable Bézier surfaces , 2003, Comput. Aided Geom. Des..

[27]  Martin Peternell,et al.  Developable surface fitting to point clouds , 2004, Comput. Aided Geom. Des..

[28]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[29]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[30]  Avinash C. Kak,et al.  Specularity elimination in range sensing for accurate 3D modeling of specular objects , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[31]  Pierre Gurdjos,et al.  Towards shape from shading under realistic photographic conditions , 2004, ICPR 2004.

[32]  Larry S. Davis,et al.  Structure of Applicable Surfaces from Single Views , 2004, ECCV.

[33]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[34]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[35]  Günter Aumann,et al.  Degree elevation and developable Be'zier surfaces , 2004, Comput. Aided Geom. Des..

[36]  Wenping Wang,et al.  Control point adjustment for B-spline curve approximation , 2004, Comput. Aided Des..

[37]  Avinash C. Kak,et al.  Specularity elimination in range sensing for accurate 3D modeling of specular objects , 2004 .

[38]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, CVPR.

[39]  Kai Tang,et al.  Developable Triangulations of a Strip , 2005 .

[40]  David S. Doermann,et al.  Flattening curved documents in images , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[41]  Johannes Wallner,et al.  Geometric modeling with conical meshes and developable surfaces , 2006, SIGGRAPH 2006.

[42]  Georg Glaeser,et al.  Developable surfaces in contemporary architecture , 2007 .

[43]  W. B. Seales,et al.  Restoring 2D Content from Distorted Documents , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  P. Slavík,et al.  Geodesic-Controlled Developable Surfaces for Modeling Paper Bending , 2007 .

[45]  L. Fernández-Jambrina.  B-spline control nets for developable surfaces , 2007, Comput. Aided Geom. Des..

[46]  Adrien Bartoli,et al.  A Quasi-Minimal Model for Paper-Like Surfaces , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  T. Banchoff,et al.  Differential Geometry of Curves and Surfaces , 2010 .