3D Clothing Transfer in Virtual Fitting Based on UV Mapping

3D virtual fitting based on human body contours can greatly improve the try-on effect of some tight-fitting clothes. This paper presents a fitting approach requiring merely a single RGB image and transferring clothing texture onto the human body contour, under the instruction of a UV texture guide map. DensePose is used to segment a human body into 24 patches and predict UV coordinates of each patch. 24 clothing UV patches and their textures are extracted from SURREAL dataset. Clothing patches are filtered by the patch index and the alpha value of each pixel in the patch, avoiding unnecessary replacement such as users' heads and exposed skin. They are subsequently transformed into human body corresponding patches by UV mapping. Human bodies with various figures, poses and perspectives can be fitted with clothes. Some users and clothes are tested, and satisfactory results are attained.

[1]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[2]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[3]  Hang Zhang,et al.  Photo-Realistic Facial Texture Transfer , 2017, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[4]  Lu Li,et al.  3D reconstruction based on Kinect , 2014, 2014 12th International Conference on Signal Processing (ICSP).

[5]  Stephen Lin,et al.  Image-based clothes animation for virtual fitting , 2012, SIGGRAPH Asia Technical Briefs.

[6]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Henry Dreyfuss,et al.  Measure of Man and Woman: Human Factors in Design , 1993 .

[8]  Ting Liu,et al.  Real-time 3D virtual dressing based on users' skeletons , 2017, 2017 4th International Conference on Systems and Informatics (ICSAI).

[9]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Mário Alves,et al.  Virtual Fitting Room Augmented Reality Techniques for e-Commerce , 2011, CENTERIS.

[11]  Jiajun Wu,et al.  Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  Silvio Savarese,et al.  DeformNet: Free-Form Deformation Network for 3D Shape Reconstruction from a Single Image , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[13]  Chen Kong,et al.  Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction , 2017, AAAI.

[14]  Iasonas Kokkinos,et al.  DensePose: Dense Human Pose Estimation in the Wild , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Ioannis Pachoulakis,et al.  AUGMENTED REALITY PLATFORMS FOR VIRTUAL FITTING ROOMS , 2012 .

[16]  Bo Yang,et al.  3D Object Reconstruction from a Single Depth View with Adversarial Learning , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[17]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Jiawen Chen,et al.  Texture transfer using geometry correlation , 2006, EGSR '06.

[19]  Cordelia Schmid,et al.  Learning from Synthetic Humans , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[21]  장운석,et al.  Method for converting 2d image into pseudo 3d image and user-adapted total coordination method in use artificial intelligence, and service besiness method thereof , 2004 .