A virtual try-on system in augmented reality using RGB-D cameras for footwear personalization

Abstract This paper presents a system for design evaluation of footwear using commercial depth-sensing technologies. In a mixed reality environment, the system allows users to virtually try on 3D shoe models in a live video stream. A two-stage object tracking algorithm was developed to correctly align shoe models to moving feet during the try-on process. Color markers on the user's foot enabled markerless tracking. Tracking was driven by an iterative closest point (ICP) algorithm that superimposed the captured depth data and predefined reference foot models. Test data showed that the two-stage approach resulted in increased positional accuracy compared with tracking using only surface registration. Trimming the reference model using the instant view angle increased the computational efficiency of the ICP algorithm. The proposed virtual try-on function is an effective tool for realizing human-centered design. This study also demonstrated a new application of RGB-D cameras to product design.

[1]  Peihua Gu,et al.  Automatic localization and comparison for free-form surface inspection , 2006 .

[2]  Caterina Rizzi,et al.  3D virtual apparel design for industrial applications , 2005, Comput. Aided Des..

[3]  Charlie C. L. Wang,et al.  Volume Parameterization for Design Automation of Customized Free-Form Products , 2007, IEEE Transactions on Automation Science and Engineering.

[4]  D. Zhang,et al.  Principle Component Analysis , 2004 .

[5]  M. Rouse,et al.  The Customer Centric Enterprise : Advances in Mass Customization and Personalization , 2003 .

[6]  Robert B. Fisher,et al.  Estimating 3-D rigid body transformations: a comparison of four major algorithms , 1997, Machine Vision and Applications.

[7]  Ralf Sarlette,et al.  Virtual Try-On : Topics in Realistic, Individualized Dressing in Virtual Reality , 2004 .

[8]  Ching-Sheng Wang,et al.  Intuitional 3D Museum Navigation System Using Kinect , 2013, ITCS.

[9]  Chih-Hsing Chu,et al.  Human-centric design personalization of 3D glasses frame in markerless augmented reality , 2012, Adv. Eng. Informatics.

[10]  Walter H. Buchsbaum,et al.  Color TV servicing , 1975 .

[11]  R. T. Hutchings,et al.  Color atlas of human anatomy , 1977 .

[12]  Jimeno-MorenillaAntonio,et al.  Augmented and Virtual Reality techniques for footwear , 2013 .

[13]  H. Deutsch.  Principle Component Analysis , 2004 .

[14]  S. Jack Hu,et al.  Product architecting for personalization , 2013 .

[15]  Li Bai,et al.  Fitting 3D garment models onto individual human models , 2010, Comput. Graph..

[16]  Gerhard Reitmayr,et al.  Virtual Try-On through Image-Based Rendering , 2013, IEEE Transactions on Visualization and Computer Graphics.

[17]  Xiaogang Jin,et al.  Interactive virtual try-on clothing design systems , 2010, Comput. Aided Des..

[18]  Jau-Chuan Ke,et al.  Analysis of a discrete-time queue with server subject to vacations and breakdowns , 2013 .

[19]  Tsuyoshi Nishiwaki,et al.  Footwear Fitting Design , 2011 .

[20]  Yao-Jen Chang,et al.  A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. , 2011, Research in developmental disabilities.

[21]  Charlie C. L. Wang,et al.  Flexible shape control for automatic resizing of apparel products , 2012, Comput. Aided Des..

[22]  Rolf Adams,et al.  Seeded Region Growing , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  José-Luis Sánchez-Romero,et al.  Augmented and Virtual Reality techniques for footwear , 2013, Comput. Ind..

[24]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Kyoung-Yun Kim,et al.  Facial configuration and BMI based personalized face and upper body modeling for customer-oriented wearable product design , 2010, Comput. Ind..

[26]  Rajit Gadh,et al.  Virtual and Augmented Reality Technologies for Product Realization , 1999 .

[27]  Chih-Hsing Chu,et al.  Augmented reality-based design customization of footwear for children , 2012, Journal of Intelligent Manufacturing.

[28]  Michael T. Fralix,et al.  FROM MASS PRODUCTION TO MASS CUSTOMIZATION , 2001 .

[29]  Ligang Liu,et al.  Scanning 3D Full Human Bodies Using Kinects , 2012, IEEE Transactions on Visualization and Computer Graphics.

[30]  Chao Ma,et al.  Mixed Collaborative Detection Based on the SNR , 2013 .

[31]  Jidong Huang,et al.  Study on the use of Microsoft Kinect for robotics applications , 2012, Proceedings of the 2012 IEEE/ION Position, Location and Navigation Symposium.