Detailed Garment Recovery from a Single-View Image

Most recent garment capturing techniques rely on acquiring multiple views of clothing, which may not always be readily available, especially in the case of pre-existing photographs from the web. As an alternative, we pro- pose a method that is able to compute a rich and realistic 3D model of a human body and its outfits from a single photograph with little human in- teraction. Our algorithm is not only able to capture the global shape and geometry of the clothing, it can also extract small but important details of cloth, such as occluded wrinkles and folds. Unlike previous methods using full 3D information (i.e. depth, multi-view images, or sampled 3D geom- etry), our approach achieves detailed garment recovery from a single-view image by using statistical, geometric, and physical priors and a combina- tion of parameter estimation, semantic parsing, shape recovery, and physics- based cloth simulation. We demonstrate the effectiveness of our algorithm by re-purposing the reconstructed garments for virtual try-on and garment transfer applications, as well as cloth animation for digital characters.

[1]  Ronan Collobert,et al.  Recurrent Convolutional Neural Networks for Scene Labeling , 2014, ICML.

[2]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.

[3]  Dinesh Manocha,et al.  ICCD: Interactive Continuous Collision Detection between Deformable Models Using Connectivity-Based Culling , 2008, IEEE Transactions on Visualization and Computer Graphics.

[4]  David E. Breen,et al.  Cloth modeling and animation , 2000 .

[5]  Hans-Peter Seidel,et al.  Physically Based Tracking of Cloth , 2006 .

[6]  Yu Wang,et al.  Design automation for customized apparel products , 2005, Comput. Aided Des..

[7]  D. Cohen-Or,et al.  Parametric reshaping of human bodies in images , 2010, ACM Trans. Graph..

[8]  Jian Sun,et al.  Lazy snapping , 2004, SIGGRAPH 2004.

[9]  Hans-Peter Seidel,et al.  A Statistical Model of Human Pose and Body Shape , 2009, Comput. Graph. Forum.

[10]  Nadia Magnenat-Thalmann,et al.  An automatic modeling of human bodies from sizing parameters , 2003, I3D '03.

[11]  Dinesh Manocha,et al.  Fast collision detection for deformable models using representative-triangles , 2008, I3D '08.

[12]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH '05.

[13]  J. Marks,et al.  Garment Motion Capture Using Color‐Coded Patterns , 2005, SIGGRAPH '05.

[14]  Tomás Pajdla,et al.  Multi-view reconstruction preserving weakly-supported surfaces , 2011, CVPR 2011.

[15]  Derek Bradley,et al.  Wrinkling Captured Garments Using Space‐Time Data‐Driven Deformation , 2009, Comput. Graph. Forum.

[16]  Richard L. Grimsdale,et al.  Computer graphics techniques for modeling cloth , 1996, IEEE Computer Graphics and Applications.

[17]  Huamin Wang,et al.  Data-driven elastic models for cloth: modeling and measurement , 2011, ACM Trans. Graph..

[18]  Lin Wang,et al.  Garment modeling with a depth camera , 2015, ACM Trans. Graph..

[19]  Harry Shum,et al.  Lazy snapping , 2004, ACM Trans. Graph..

[20]  Katsu Yamane,et al.  High Marker Density Motion Capture by Retroreflective Mesh Suit , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[21]  Pieter Peers,et al.  Temporally coherent completion of dynamic shapes , 2012, TOGS.

[22]  Camillo J. Taylor,et al.  Reconstruction of Articulated Objects from Point Correspondences in a Single Uncalibrated Image , 2000, Comput. Vis. Image Underst..

[23]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[24]  Ronald Fedkiw,et al.  Robust treatment of collisions, contact and friction for cloth animation , 2002, SIGGRAPH Courses.

[25]  Hao Li,et al.  Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans , 2008, Comput. Graph. Forum.

[26]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Jerry Weil,et al.  The synthesis of cloth objects , 1986, SIGGRAPH.

[28]  Ronald Fedkiw,et al.  Simulation of clothing with folds and wrinkles , 2003, SCA '03.

[29]  Jovan Popovic,et al.  Deformation transfer for triangle meshes , 2004, ACM Trans. Graph..

[30]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  John F. Hughes,et al.  A Sketch-Based Interface for Clothing Virtual Characters , 2007, IEEE Computer Graphics and Applications.

[32]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Jessica K. Hodgins,et al.  A perceptual control space for garment simulation , 2015, ACM Trans. Graph..

[34]  AgarwalAnkur,et al.  Recovering 3D Human Pose from Monocular Images , 2006 .

[35]  Peter-Pike J. Sloan,et al.  Fast and Efficient Skinning of Animated Meshes , 2010, Comput. Graph. Forum.

[36]  Dinesh Manocha,et al.  Fast continuous collision detection among deformable models using graphics processors , 2006, EGVE'06.

[37]  Pascal Volino,et al.  Fast Geometrical Wrinkles on Animated Surfaces , 1998 .

[38]  Derek Bradley,et al.  High-quality capture of eyes , 2014, ACM Trans. Graph..

[39]  Huamin Wang,et al.  Real-time Human Pose and Shape Estimation for Virtual Try-On Using a Single Commodity Depth Camera , 2014, IEEE Trans. Vis. Comput. Graph..

[40]  Huamin Wang,et al.  Multi-resolution isotropic strain limiting , 2010, ACM Trans. Graph..

[41]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[42]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[43]  Ruigang Yang,et al.  Semantic Parametric Reshaping of Human Body Models , 2014, 2014 2nd International Conference on 3D Vision.

[44]  James F. O'Brien,et al.  Adaptive anisotropic remeshing for cloth simulation , 2012, ACM Trans. Graph..

[45]  Andrew P. Witkin,et al.  Large steps in cloth simulation , 1998, SIGGRAPH.

[46]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[47]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[48]  Michael J. Black,et al.  Detailed Human Shape and Pose from Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Ankur Agarwal,et al.  Recovering 3D human pose from monocular images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Nadia Magnenat-Thalmann,et al.  A body and Garment Creation Method for an Internet Based Virtual Fitting Room. , 2002 .

[51]  Jernej Barbic,et al.  Skin microstructure deformation with displacement map convolution , 2015, ACM Trans. Graph..

[52]  Alla Sheffer,et al.  Context-aware garment modeling from sketches , 2011, Comput. Graph..

[53]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[54]  David A. Forsyth,et al.  Capturing and animating occluded cloth , 2007, ACM Trans. Graph..

[55]  Alexander M. Bronstein,et al.  Efficient Computation of Isometry-Invariant Distances Between Surfaces , 2006, SIAM J. Sci. Comput..

[56]  Kun Zhou,et al.  3D shape regression for real-time facial animation , 2013, ACM Trans. Graph..

[57]  Charlie C. L. Wang,et al.  Flexible shape control for automatic resizing of apparel products , 2012, Comput. Aided Des..

[58]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[59]  Alla Sheffer,et al.  Animation wrinkling: augmenting coarse cloth simulations with realistic-looking wrinkles , 2010, ACM Trans. Graph..

[60]  Eitan Grinspun,et al.  Parsing sewing patterns into 3D garments , 2013, ACM Trans. Graph..

[61]  Alla Sheffer,et al.  Design preserving garment transfer , 2012, ACM Trans. Graph..

[62]  Bin Zhou,et al.  Garment Modeling from a Single Image , 2013, Comput. Graph. Forum.

[63]  Hyeong-Seok Ko,et al.  Garment capture from a photograph , 2015, Comput. Animat. Virtual Worlds.

[64]  Alexander M. Bronstein,et al.  Calculus of Nonrigid Surfaces for Geometry and Texture Manipulation , 2007, IEEE Transactions on Visualization and Computer Graphics.

[65]  Kun Zhou,et al.  Single-view hair modeling for portrait manipulation , 2012, ACM Trans. Graph..

[66]  Tamara L. Berg,et al.  Paper Doll Parsing: Retrieving Similar Styles to Parse Clothing Items , 2013, 2013 IEEE International Conference on Computer Vision.

[67]  Alla Sheffer,et al.  Virtual Garments: A Fully Geometric Approach for Clothing Design , 2006, Comput. Graph. Forum.

[68]  Robert Bridson,et al.  Animating developable surfaces using nonconforming elements , 2008, ACM Trans. Graph..

[69]  Volker Scholz,et al.  Texture replacement of garments in monocular video sequences , 2006, EGSR '06.

[70]  Yu Guo,et al.  Deformable model for estimating clothed and naked human shapes from a single image , 2013, The Visual Computer.

[71]  Eitan Grinspun,et al.  To appear in the ACM SIGGRAPH conference proceedings Efficient Simulation of Inextensible Cloth , 2007 .