Robust deformation capture from temporal range data for surface rendering

Imagine an object such as a paper sheet being waved in front of some sensor. Reconstructing the time‐varying 3D shape of the object finds direct applications in computer animation. The goal of this paper is to provide such a deformation capture system for surfaces. It uses temporal range data obtained by sensors such as those based on structured light or stereo. So as to deal with many different kinds of material, we do not make the usual assumption that the object surface has textural information. This rules out those techniques based on detecting and matching keypoints or directly minimizing color discrepancy. The proposed method is based on a planar mesh that is deformed so as to fit each of the range images. We show how to achieve this by minimizing a compound cost function combining several data and regularization terms, needed to make the overall system robust so that it can deal with low quality datasets. Carefully examining the parameter to residual relationship shows that this cost function can be minimized very efficiently by coupling nonlinear least squares methods with sparse matrix operators. Experimental results for challenging datasets coming from different kinds of range sensors are reported. The algorithm is reasonably fast and is shown to be robust to missing and erroneous data points. Copyright © 2008 John Wiley & Sons, Ltd.

[1]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[2]  Sergio Pissanetzky,et al.  Sparse Matrix Technology , 1984 .

[3]  Sartaj Sahni,et al.  An efficient motion estimator with application to medical image registration , 1998, Medical Image Anal..

[4]  Pascal Fua,et al.  Physically Valid Shape Parameterization for Monocular 3-D Deformable Surface Tracking , 2005, BMVC.

[5]  Andrew W. Fitzgibbon,et al.  Single View Reconstruction of Curved Surfaces , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[6]  Karl Rohr,et al.  Radial basis functions with compact support for elastic registration of medical images , 2001, Image Vis. Comput..

[7]  Szymon Rusinkiewicz,et al.  Non-rigid range-scan alignment using thin-plate splines , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[8]  Marc Rioux,et al.  Direct estimation of deformable motion parameters from range image sequence , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[9]  Anand Rangarajan,et al.  Groupwise point pattern registration using a novel CDF-based Jensen-Shannon Divergence , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[10]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[11]  Andrew W. Fitzgibbon,et al.  Robust Registration of 2D and 3D Point Sets , 2003, BMVC.

[12]  Wolfgang Heidrich,et al.  Cloth Motion Capture , 2003, SIGGRAPH '03.

[13]  B. Schölkopf,et al.  Non-rigid point set registration: Coherent Point Drift , 2007 .

[14]  Miguel Á. Carreira-Perpiñán,et al.  Non-rigid point set registration: Coherent Point Drift , 2006, NIPS.

[15]  Takeo Kanade,et al.  A Correlation-Based Approach to Robust Point Set Registration , 2004, ECCV.

[16]  Anand Rangarajan,et al.  A new point matching algorithm for non-rigid registration , 2003, Comput. Vis. Image Underst..

[17]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[18]  Luc Van Gool,et al.  Real-time range scanning of deformable surfaces by adaptively coded structured light , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[19]  Stefan May,et al.  3D time-of-flight cameras for mobile robotics , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Baba C. Vemuri,et al.  A robust algorithm for point set registration using mixture of Gaussians , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[21]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[22]  Andrea Fusiello,et al.  Registration of Multiple Acoustic Range Views for Underwater Scene Reconstruction , 2002, Comput. Vis. Image Underst..

[23]  Umberto Castellani,et al.  Joint Reconstruction and Registration of a Deformable Planar Surface Observed by a 3D Sensor , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[24]  Steven J. Gortler,et al.  Geometry images , 2002, SIGGRAPH.

[25]  Volker Scholz,et al.  Cloth Motion from Optical Flow , 2004, VMV.

[26]  J. Marks,et al.  Garment Motion Capture Using Color‐Coded Patterns , 2005, SIGGRAPH '05.

[27]  Igor Guskov,et al.  Trackable surfaces , 2003, SCA '03.

[28]  Marc Alexa,et al.  Recent Advances in Mesh Morphing , 2002, Comput. Graph. Forum.