Physical parameter prediction by embedding human perceptual parameter for 3D garment modeling

To model garments into a virtual environment, it is crucial to predict the physical parameters of the simulated model. However, it is troublesome for a user or technical director to intuitively reflect their aesthetic intention using physical parameters. In this paper, we propose a framework that predicts various physical parameters (e.g., stretch resistance, bend resistance, …) by embedding human perceptual parameters (e.g., wrinkly, stretchy,…) in multi-task learning (MTL) perspective. By predicting both physical and perceptual parameters, we can effectively solve this problem, and can give an important cue to model a 3D garment maximizing users visual presence. Furthermore, by taking a class activation mapping method, our model seeks the intermediate visual understanding of physical and perceptual parameters. Through the rigorous experiments, we demonstrate that the predicted physical and perceptual parameters agree with subjective values.

[1]  Jessica K. Hodgins,et al.  Estimating cloth simulation parameters from video , 2003, SCA '03.

[2]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[3]  Mark Pauly,et al.  Projective dynamics , 2014, ACM Trans. Graph..

[4]  Rama Chellappa,et al.  HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Andrew P. Witkin,et al.  Large steps in cloth simulation , 1998, SIGGRAPH.

[6]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[8]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[9]  Sanghoon Lee,et al.  Deep Learning of Human Visual Sensitivity in Image Quality Assessment Framework , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Alain Trémeau,et al.  Multi-task, multi-domain learning: Application to semantic segmentation and pose regression , 2017, Neurocomputing.

[11]  Jongyoo Kim,et al.  Deep CNN-Based Blind Image Quality Predictor , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[13]  Jessica K. Hodgins,et al.  A perceptual control space for garment simulation , 2015, ACM Trans. Graph..

[14]  Ming C. Lin,et al.  Learning-Based Cloth Material Recovery from Video , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Béla Julesz,et al.  Visual Pattern Discrimination , 1962, IRE Trans. Inf. Theory.

[16]  William T. Freeman,et al.  Estimating the Material Properties of Fabric from Video , 2013, 2013 IEEE International Conference on Computer Vision.

[17]  Jinwoo Kim,et al.  Deep Video Quality Assessor: From Spatio-Temporal Visual Sensitivity to a Convolutional Neural Aggregation Network , 2018, ECCV.