Augmented Reality Fashion Show Using Personalized 3D Human Models

When purchasing clothes online, consumers usually have to imagine what they might look like when wearing them in real life surroundings. This can hinder their ability to make appropriate purchasing decisions. In order to alleviate this uncertainty, we propose using augmenting fashion show, a new immersive experience to enable consumers to create personalized 3D models of themselves and have them fitted with a variety of purchasable clothing using Augmented Reality. Furthermore, their models are enhanced with animations (walking and a variety of poses) depending on the real environment. In this way, consumers can see a representation of themselves wearing a variety of clothes, and get a sense of what they would look like in real life surroundings. A prototype of our system was demonstrated, and a preliminary evaluation was conducted to verify our system. We have received positive feedback in terms of effectiveness, assistance and potential application.

[1]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Marcus A. Magnor,et al.  Video Based Reconstruction of 3D People Models , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  S. Gray,et al.  In virtual fashion , 1998 .

[4]  Hans-Peter Seidel,et al.  A Statistical Model of Human Pose and Body Shape , 2009, Comput. Graph. Forum.

[5]  Masashi Nishiyama,et al.  Virtual Fitting by Single-Shot Body Shape Estimation , 2014 .

[6]  Dimitris N. Metaxas,et al.  Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Jiaolong Yang,et al.  Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Rong Li,et al.  Research of Interactive 3D Virtual Fitting Room on Web Environment , 2011, 2011 Fourth International Symposium on Computational Intelligence and Design.

[9]  Luc Van Gool,et al.  One-Shot Video Object Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Susu Yao,et al.  A Mixed Reality Virtual Clothes Try-On System , 2013, IEEE Transactions on Multimedia.

[11]  Yoshihiro Kanamori,et al.  Image-Based Virtual Fitting System with Garment Image Reshaping , 2014, 2014 International Conference on Cyberworlds.

[12]  Lu Wang,et al.  Magic Mirror: A virtual handbag shopping system , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[13]  Derek Nowrouzezahrai,et al.  Learning hatching for pen-and-ink illustration of surfaces , 2012, TOGS.

[14]  Peter V. Gehler,et al.  Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image , 2016, ECCV.

[15]  Alla Sheffer,et al.  Virtual Garments: A Fully Geometric Approach for Clothing Design , 2006, Comput. Graph. Forum.

[16]  Michael J. Black,et al.  The stitched puppet: A graphical model of 3D human shape and pose , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Larry S. Davis,et al.  VITON: An Image-Based Virtual Try-on Network , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Björn Stenger,et al.  Virtual Fashion Show Using Real-Time Markerless Motion Capture , 2006, ACCV.

[19]  Anna Hilsmann,et al.  Tracking and Retexturing Cloth for Real-Time Virtual Clothing Applications , 2009, MIRAGE.

[20]  Xiaogang Jin,et al.  Interactive virtual try-on clothing design systems , 2010, Comput. Aided Des..

[21]  Pascal Fua,et al.  Articulated Soft Objects for Video-based Body Modeling , 2001, ICCV.

[22]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[23]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.