Structure-consistent customized virtual mannequin reconstruction from 3D scans based on optimization

The 3D virtual mannequin has been widely used in apparel industry, and its importance is also increasing. This work develops a new 3D virtual mannequin reconstruction system based on optimization. All the mannequins reconstructed by the proposed approach share the identical topology, that is, there is a point-to-point correspondence among the mannequins, which will significantly facilitate much subsequent processing in fashion design, made-to-measure, and virtual try-on. The inputs to the proposed system contain a template human body, a raw scan (represented in mesh), and a very sparse corresponding landmarks set. The proposed approach substantially utilizes the optimization technology to drive the template to deform into a real scan. There is no special requirement on the raw meshes. The raw meshes may have a different number of vertices and triangles or may even be incomplete. The proposed method only needs 21 landmarks as hard-constraints to reconstruct a mannequin with tens of thousands of vertices. These landmarks can be extracted automatically for standard mannequin reconstruction. Besides the standard mannequin, the proposed system can also be used to reconstruct display mannequins, that is, mannequins with various poses. The experiments visualize the optimization procedure and verify that the optimization is efficient and effective. Quantitative analysis also proves that the reconstruction error satisfies the requirements of fashion design and tailoring.

[1]  Yueqi Zhong,et al.  Automatic body measurement based on slicing loops , 2018, International Journal of Clothing Science and Technology.

[2]  Markus H. Gross,et al.  Human Shape from Silhouettes Using Generative HKS Descriptors and Cross-Modal Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Christian Theobalt,et al.  MonoPerfCap , 2017, ACM Trans. Graph..

[4]  Michael J. Black,et al.  Estimating human shape and pose from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Alexander M. Bronstein,et al.  Efficient Computation of Isometry-Invariant Distances Between Surfaces , 2006, SIAM J. Sci. Comput..

[6]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[7]  Bugao Xu,et al.  Body scanning and modeling for custom fit garments , 2002 .

[8]  Michael J. Black,et al.  Combined discriminative and generative articulated pose and non-rigid shape estimation , 2007, NIPS.

[9]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH '05.

[10]  Shih-Wen Hsiao,et al.  A study of surface reconstruction for 3D mannequins based on feature curves , 2013, Comput. Aided Des..

[11]  Hans-Peter Seidel,et al.  General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues , 2016, ECCV.

[12]  Charlie C. L. Wang,et al.  Parameterization and parametric design of mannequins , 2005, Comput. Aided Des..

[13]  Peter V. Gehler,et al.  DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Markus H. Gross,et al.  HS-Nets: Estimating Human Body Shape from Silhouettes with Convolutional Neural Networks , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[15]  Peter V. Gehler,et al.  Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image , 2016, ECCV.

[16]  Yinghao Huang,et al.  Towards Accurate Marker-Less Human Shape and Pose Estimation over Time , 2017, 2017 International Conference on 3D Vision (3DV).

[17]  Zheng Li,et al.  Layer-Based Mannequin Reconstruction and Parameterization from 3D Range Data , 2008, GMP.

[18]  Bernt Schiele,et al.  Building statistical shape spaces for 3D human modeling , 2015, Pattern Recognit..

[19]  Bodo Rosenhahn,et al.  Optical Flow-Based 3D Human Motion Estimation from Monocular Video , 2017, GCPR.

[20]  Thomas A. Funkhouser,et al.  A benchmark for 3D mesh segmentation , 2009, ACM Trans. Graph..

[21]  Ankur Agarwal,et al.  Learning to track 3D human motion from silhouettes , 2004, ICML.

[22]  Peter V. Gehler,et al.  Unite the People: Closing the Loop Between 3D and 2D Human Representations , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Shih-Wen Hsiao,et al.  Applying Kinect on the Development of a Customized 3D Mannequin , 2015 .

[24]  Ruigang Yang,et al.  Semantic Parametric Reshaping of Human Body Models , 2014, 2014 2nd International Conference on 3D Vision.

[25]  Markus H. Gross,et al.  Shape from Selfies: Human Body Shape Estimation Using CCA Regression Forests , 2016, ECCV.

[26]  Peter V. Gehler,et al.  Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation , 2018, 2018 International Conference on 3D Vision (3DV).

[27]  Jitendra Malik,et al.  End-to-End Recovery of Human Shape and Pose , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Sungmin Kim,et al.  Parametric body model generation for garment drape simulation , 2004 .

[29]  CurlessBrian,et al.  The space of human body shapes , 2003 .

[30]  Charlie C. L. Wang,et al.  From laser-scanned data to feature human model: a system based on fuzzy logic concept , 2003, Comput. Aided Des..

[31]  Olga Sorkine-Hornung,et al.  Instant field-aligned meshes , 2015, ACM Trans. Graph..

[32]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[33]  Yueqi Zhong.  Two-step Registration in 3D Human Body Scanning Based on Multiple RGB-D Sensors , 2015 .

[34]  Qing Wu,et al.  Shape-from-Mask: A Deep Learning Based Human Body Shape Reconstruction from Binary Mask Images , 2018, ArXiv.

[35]  Bugao Xu,et al.  Automatic segmenting and measurement on scanned human body , 2006 .

[36]  Chi Zhou,et al.  Parametric design for human body modeling by wireframe-assisted deep learning , 2019, Comput. Aided Des..

[37]  Taku Komura,et al.  Personalized 3D mannequin reconstruction based on 3D scanning , 2018 .