Interactive Virtual Try-On Based on Real-Time Motion Capture

In this paper, we present an augmented reality system for interactive virtual try-on of garments based on real-time motion capture. The system uses a commodity depth sensor to obtain depth images and joint motion data of a user. We first apply a novel evaluation method over two specific depth images to determine the user's body measurements, according to which garment models are deformed to fit his/her actual size. Torso joint of the selected garment model is translated to follow the user's position, while other joints are rotated to mimic the user's postures. Then the system superimposes the transformed model onto the same user in a live image sequence and move with him/her accurately. We also propose a method for handling model dangling artifacts due to the limitations of current tracking techniques in producing smooth, noise free motion data in real-time. The system can be used at user's home as well as retailer's shop to present an interactive try-on experience.

[1]  Noel E. O'Connor,et al.  Evaluating a dancer's performance using kinect-based skeleton tracking , 2011, ACM Multimedia.

[2]  Sebastian Thrun,et al.  Video-based reconstruction of animatable human characters , 2010, SIGGRAPH 2010.

[3]  Ilya Baran,et al.  Automatic rigging and animation of 3D characters , 2007, SIGGRAPH 2007.

[4]  Alon Lerner,et al.  Enhanced interactive gaming by blending full-body tracking and gesture animation , 2010, SA '10.

[5]  James Begole,et al.  An intelligent fitting room using multi-camera perception , 2008, IUI '08.

[6]  Nadia Magnenat-Thalmann,et al.  A data-driven approach for real-time clothes simulation , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..

[7]  Hans-Peter Seidel,et al.  MovieReshape: tracking and reshaping of humans in videos , 2010, SIGGRAPH 2010.

[8]  Jirí Zára,et al.  Geometric skinning with approximate dual quaternion blending , 2008, TOGS.

[9]  Nadia Magnenat-Thalmann,et al.  Made-to-Measure Technologies for an Online Clothing Store , 2003, IEEE Computer Graphics and Applications.

[10]  Daniel Thalmann,et al.  Joint-dependent local deformations for hand animation and object grasping , 1989 .

[11]  Douglas C. Montgomery,et al.  Introduction to linear regression analysis : solutions manual to accompany , 2013 .

[12]  Nadia Magnenat-Thalmann,et al.  Sizing avatars from skin weights , 2009, VRST '09.

[13]  Miaolong Yuan,et al.  A mixed reality system for virtual glasses try-on , 2011, VRCAI.

[14]  Xiaogang Jin,et al.  Interactive virtual try-on clothing design systems , 2010, Comput. Aided Des..

[15]  Douglas C. Montgomery,et al.  Introduction to Linear Regression Analysis, Solutions Manual (Wiley Series in Probability and Statistics) , 2007 .

[16]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[17]  Jean-Paul Chilès,et al.  Wiley Series in Probability and Statistics , 2012 .

[18]  Gerhard Reitmayr,et al.  Free viewpoint virtual try-on with commodity depth cameras , 2011, VRCAI.

[19]  Jirí Zára,et al.  Spherical blend skinning: a real-time deformation of articulated models , 2005, I3D '05.

[20]  Olga Sorkine-Hornung,et al.  Stretchable and Twistable Bones for Skeletal Shape Deformation , 2011, ACM Trans. Graph..

[21]  Nadia Magnenat-Thalmann,et al.  Parameterized Human Body Model for Real-Time Applications , 2007, CW 2007.

[22]  Sebastian Thrun,et al.  Real time motion capture using a single time-of-flight camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.