Garment Motion Capture Using Color‐Coded Patterns

In this paper we present an image-based algorithm for surface reconstruction of moving garments from multiple calibrated video cameras. Using a color-coded cloth texture allows reliable matching of circular features between different camera views. We use an a priori known triangle mesh as surface model. By identifying the mesh vertices with texture elements we obtain a consistent parameterization of the surface over time without further processing. Missing data points resulting from self-shadowing are plausibly interpolated by minimizing a thin-plate functional. The deforming geometry can be used for different graphics applications, e.g. for realistic retexturing. We show results for real garments demonstrating the accuracy of the recovered flexible shape.

[1]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Cengizhan Ozturk,et al.  Structured Light Using Pseudorandom Codes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Henrique S. Malvar,et al.  Making Faces , 2019, Topoi.

[4]  Hans-Peter Seidel,et al.  Interactive multi-resolution modeling on arbitrary meshes , 1998, SIGGRAPH.

[5]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[7]  Lorenzo Torresani,et al.  Tracking and modeling non-rigid objects with rank constraints , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[8]  Nadia Magnenat-Thalmann,et al.  Comparing efficiency of integration methods for cloth simulation , 2001, Proceedings. Computer Graphics International 2001.

[9]  Kwang-Jin Choi,et al.  Stable but responsive cloth , 2002, SIGGRAPH 2002.

[10]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[11]  Wolfgang Heidrich,et al.  Cloth Motion Capture , 2003, SIGGRAPH '03.

[12]  Igor Guskov,et al.  Trackable surfaces , 2003, SCA '03.

[13]  Wolfgang Straßer,et al.  A fast finite element solution for cloth modelling , 2003, 11th Pacific Conference onComputer Graphics and Applications, 2003. Proceedings..

[14]  Joaquim Salvi,et al.  Overview of coded light projection techniques for automatic 3D profiling , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[15]  Ralf Sarlette,et al.  Efficient and Realistic Visualization of Cloth , 2003, Rendering Techniques.

[16]  Vladimir Vezhnevets,et al.  A Survey on Pixel-Based Skin Color Detection Techniques , 2003 .

[17]  Jessica K. Hodgins,et al.  Estimating cloth simulation parameters from video , 2003, SCA '03.

[18]  Ronald Fedkiw,et al.  Simulation of clothing with folds and wrinkles , 2003, SCA '03.

[19]  Markus Wacker,et al.  Tailor tools for interactive design of clothing in virtual environments , 2004, VRST '04.

[20]  David A. Forsyth,et al.  Recovering shape and irradiance maps from rich dense texton fields , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[21]  Kiriakos N. Kutulakos,et al.  Multi-View Scene Capture by Surfel Sampling: From Video Streams to Non-Rigid 3D Motion, Shape and Reflectance , 2002, International Journal of Computer Vision.

[22]  Ralf Sarlette,et al.  Acquisition, Synthesis, and Rendering of Bidirectional Texture Functions , 2005, Comput. Graph. Forum.

[23]  Volker Scholz,et al.  Garment motion capture using color-coded patterns , 2005, SIGGRAPH '05.

[24]  T. Munz.  Cloth Simulation , 2009 .

[25]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.