Reference consistent reconstruction of 3D cloth surface

Highlights? Multiview method for reconstructing a folded cloth surface with color patches. ? Error in 3D surface reconstruction can be recovered with consistency of the patches. ? Single-view patches can be extrapolated with regularity of the patches. ? 3D surface deformation is obtained from the patches on the reconstructed surface. We propose a multiview method for reconstructing a folded cloth surface on which regularly-textured color patches are printed. These patches provide not only easy pixel-correspondence between multiviews but also the following two new functions. (1) Error recovery: errors in 3D surface reconstruction (e.g. errors in occlusion boundaries and shaded regions) can be recovered based on the spatio-temporal consistency of the patches. (2) Single-view hole filling: patches that are visible only from a single view can be extrapolated from the reconstructed ones based on the regularity of the patches. Using these functions for improving 3D reconstruction also produces the patch configuration on the reconstructed surface, showing how the cloth is deformed from its reference shape. Experimental results demonstrate the above improvements and the accurate patch configurations produced by our method.

[1]  Vladimir Kolmogorov,et al.  Optimizing Binary MRFs via Extended Roof Duality , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Hans-Peter Seidel,et al.  Robust fusion of dynamic shape and normal capture for high-quality reconstruction of time-varying geometry , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Shuntaro Yamazaki,et al.  Temporal Dithering of Illumination for Fast Active Vision , 2008, ECCV.

[5]  Jovan Popović,et al.  Mesh-based inverse kinematics , 2005, SIGGRAPH 2005.

[6]  Hans-Peter Seidel,et al.  A system for articulated tracking incorporating a clothing model , 2007, Machine Vision and Applications.

[7]  Ronald Fedkiw,et al.  Robust treatment of collisions, contact and friction for cloth animation , 2002, SIGGRAPH Courses.

[8]  Manolis I. A. Lourakis,et al.  Is Levenberg-Marquardt the most efficient optimization algorithm for implementing bundle adjustment? , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[9]  Wolfgang Heidrich,et al.  Cloth Motion Capture , 2003, Comput. Graph. Forum.

[10]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[11]  Steve Marschner,et al.  Filling holes in complex surfaces using volumetric diffusion , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[12]  David A. Forsyth,et al.  Capturing and animating occluded cloth , 2007, ACM Trans. Graph..

[13]  Derek Bradley,et al.  Markerless garment capture , 2008, SIGGRAPH 2008.

[14]  Derek Bradley,et al.  Accurate multi-view reconstruction using robust binocular stereo and surface meshing , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Shree K. Nayar,et al.  Projection defocus analysis for scene capture and image display , 2006, SIGGRAPH 2006.

[16]  Björn Stenger,et al.  Non-rigid Photometric Stereo with Colored Lights , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  Olga Veksler,et al.  Markov random fields with efficient approximations , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[18]  Michael J. Black,et al.  The Naked Truth: Estimating Body Shape Under Clothing , 2008, ECCV.

[19]  Andrew W. Fitzgibbon,et al.  Global stereo reconstruction under second order smoothness priors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Jean Ponce,et al.  Dense 3D motion capture from synchronized video streams , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Pascal Fua,et al.  Local deformation models for monocular 3D shape recovery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Volker Scholz,et al.  Garment Motion Capture Using Color‐Coded Patterns , 2005, Comput. Graph. Forum.

[24]  Nadia Magnenat-Thalmann,et al.  A data-driven approach for real-time clothes simulation , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..

[25]  Andrew P. Witkin,et al.  Large steps in cloth simulation , 1998, SIGGRAPH.

[26]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[27]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[29]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Hujun Bao,et al.  Recovering consistent video depth maps via bundle optimization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Pierre Hansen,et al.  Roof duality, complementation and persistency in quadratic 0–1 optimization , 1984, Math. Program..

[32]  Pascal Fua,et al.  Surface Deformation Models for Nonrigid 3D Shape Recovery , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Jessica K. Hodgins,et al.  Estimating cloth simulation parameters from video , 2003, SCA '03.