Fast character modeling with sketch-based PDE surfaces

Virtual characters are 3D geometric models of characters. They have a lot of applications in multimedia. In this paper, we propose a new physics-based deformation method and efficient character modelling framework for creation of detailed 3D virtual character models. Our proposed physics-based deformation method uses PDE surfaces. Here PDE is the abbreviation of Partial Differential Equation, and PDE surfaces are defined as sculpting force-driven shape representations of interpolation surfaces. Interpolation surfaces are obtained by interpolating key cross-section profile curves and the sculpting force-driven shape representation uses an analytical solution to a vector-valued partial differential equation involving sculpting forces to quickly obtain deformed shapes. Our proposed character modelling framework consists of global modeling and local modeling. The global modeling is also called model building, which is a process of creating a whole character model quickly with sketch-guided and template-based modeling techniques. The local modeling produces local details efficiently to improve the realism of the created character model with four shape manipulation techniques. The sketch-guided global modeling generates a character model from three different levels of sketched profile curves called primary, secondary and key cross-section curves in three orthographic views. The template-based global modeling obtains a new character model by deforming a template model to match the three different levels of profile curves. Four shape manipulation techniques for local modeling are investigated and integrated into the new modelling framework. They include: partial differential equation-based shape manipulation, generalized elliptic curve-driven shape manipulation, sketch assisted shape manipulation, and template-based shape manipulation. These new local modeling techniques have both global and local shape control functions and are efficient in local shape manipulation. The final character models are represented with a collection of surfaces, which are modeled with two types of geometric entities: generalized elliptic curves (GECs) and partial differential equation-based surfaces. Our experiments indicate that the proposed modeling approach can build detailed and realistic character models easily and quickly.

[1]  Cecil Howard Jensen,et al.  Engineering Drawing And Design , 1979 .

[2]  Mikhail Bessmeltsev,et al.  Deep Sketch-Based Modeling of Man-Made Shapes , 2019, ArXiv.

[3]  Niloy J. Mitra,et al.  Designing chain reaction contraptions from causal graphs , 2019, ACM Trans. Graph..

[4]  Za'er Salim Abo-Hammour,et al.  Numerical solution of systems of second-order boundary value problems using continuous genetic algorithm , 2014, Inf. Sci..

[5]  Alec R. Rivers,et al.  3D modeling with silhouettes , 2010, ACM Trans. Graph..

[6]  Mark Pauly,et al.  Projective dynamics , 2014, ACM Trans. Graph..

[7]  Yang Liu,et al.  Adaptive O-CNN , 2018, ACM Trans. Graph..

[8]  Hans-Peter Seidel,et al.  An efficient construction of reduced deformable objects , 2013, ACM Trans. Graph..

[9]  Marc Alexa,et al.  Sketch-based shape retrieval , 2012, ACM Trans. Graph..

[10]  Wenping Wang,et al.  Robust flow-guided neural prediction for sketch-based freeform surface modeling , 2018, ACM Trans. Graph..

[11]  Shi-Min Hu,et al.  Sketch2Scene: sketch-based co-retrieval and co-placement of 3D models , 2013, ACM Trans. Graph..

[12]  Sehoon Ha,et al.  Iterative Training of Dynamic Skills Inspired by Human Coaching Techniques , 2014, ACM Trans. Graph..

[13]  Alla Sheffer,et al.  Modeling from contour drawings , 2009, SBIM '09.

[14]  Olga Sorkine-Hornung,et al.  On Linear Variational Surface Deformation Methods , 2008, IEEE Transactions on Visualization and Computer Graphics.

[15]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[16]  Bo Li,et al.  A comparison of methods for sketch-based 3D shape retrieval , 2014, Comput. Vis. Image Underst..

[17]  Adrien Bousseau,et al.  True2Form: 3D curve networks from 2D sketches via selective regularization , 2014, ACM Trans. Graph..

[18]  Ben Jones,et al.  Example-based plastic deformation of rigid bodies , 2016, ACM Trans. Graph..

[19]  T. Metin Sezgin,et al.  Generation of 3D Human Models and Animations Using Simple Sketches , 2020, Graphics Interface.

[20]  Ismail Khalid Kazmi,et al.  Efficient sketch‐based creation of detailed character models through data‐driven mesh deformations , 2015, Comput. Animat. Virtual Worlds.

[21]  Leonidas J. Guibas,et al.  Shape google: Geometric words and expressions for invariant shape retrieval , 2011, TOGS.

[22]  Lihua You,et al.  Blending surface generation using a fast and accurate analytical solution of a fourth-order PDE with three shape control parameters , 2004, The Visual Computer.

[23]  Craig Gotsman,et al.  Online reconstruction of 3D objects from arbitrary cross-sections , 2011, TOGS.

[24]  Szymon Rusinkiewicz,et al.  Modeling by example , 2004, ACM Trans. Graph..

[25]  Subhransu Maji,et al.  3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks , 2017, 2017 International Conference on 3D Vision (3DV).

[26]  Lihua You,et al.  Solid modelling based on sixth order partial differential equations , 2011, Comput. Aided Des..

[27]  Jianguo Xiao,et al.  Sketch based modeling and editing via shape space exploration , 2020, Multimedia Tools and Applications.

[28]  Tao Ju,et al.  Topology-constrained surface reconstruction from cross-sections , 2015, ACM Trans. Graph..

[29]  Carl Leake,et al.  Least-Squares Solutions of Eighth-Order Boundary Value Problems Using the Theory of Functional Connections , 2020, Mathematics.

[30]  Lihua You,et al.  PDE based surface representation - vase design , 2002, Comput. Graph..

[31]  Derek Nowrouzezahrai,et al.  Learning hatching for pen-and-ink illustration of surfaces , 2012, TOGS.

[32]  Young J. Kim,et al.  Interactive generalized penetration depth computation for rigid and articulated models using object norm , 2014, ACM Trans. Graph..

[33]  Wolfgang Straßer,et al.  Interactive physically-based shape editing , 2008, SPM '08.

[34]  Chenxi Huang,et al.  Sketch Augmentation-Driven Shape Retrieval Learning Framework Based on Convolutional Neural Networks , 2020, IEEE Transactions on Visualization and Computer Graphics.

[35]  Christian Rössl,et al.  Laplacian surface editing , 2004, SGP '04.

[36]  Lihua You,et al.  PDE blending surfaces with C2 continuity , 2004, Comput. Graph..

[37]  Steve Marschner,et al.  Matching Real Fabrics with Micro-Appearance Models , 2015, ACM Trans. Graph..

[38]  Leif Kobbelt,et al.  An intuitive framework for real-time freeform modeling , 2004, ACM Trans. Graph..

[39]  Bin Wang,et al.  Deformation capture and modeling of soft objects , 2015, ACM Trans. Graph..

[40]  Yizhou Yu,et al.  DeepSketch2Face , 2017, ACM Trans. Graph..

[41]  Adrien Bousseau,et al.  OpenSketch: a richly-annotated dataset of product design sketches , 2019, ACM Trans. Graph..

[42]  Christopher Wojtan,et al.  Fast approximations for boundary element based brittle fracture simulation , 2016, ACM Trans. Graph..

[43]  Marc Alexa,et al.  SilSketch: automated sketch-based editing of surface meshes , 2007, SBIM '07.

[44]  Ming C. Lin,et al.  Free-flowing granular materials with two-way solid coupling , 2010, SIGGRAPH 2010.

[45]  Leif Kobbelt,et al.  Character animation from 2D pictures and 3D motion data , 2007, TOGS.

[46]  Martin Wattenberg,et al.  Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow , 2018, IEEE Transactions on Visualization and Computer Graphics.

[47]  Ismail Khalid Kazmi,et al.  A Survey of Sketch Based Modeling Systems , 2014, 2014 11th International Conference on Computer Graphics, Imaging and Visualization.

[48]  Guoliang Luo,et al.  A new sketch-based 3D model retrieval method by using composite features , 2017, Multimedia Tools and Applications.

[49]  Fei Hou,et al.  RegionSketch: Interactive and Rapid Creation of 3D Models with Rich Details , 2019, PG.

[50]  Andrew Nealen,et al.  Physically Based Deformable Models in Computer Graphics , 2006, Comput. Graph. Forum.

[51]  Mario Costa Sousa,et al.  Sketch-based modeling: A survey , 2009, Comput. Graph..

[52]  Jinyuan Jia,et al.  A learning framework for shape retrieval based on multilayer perceptrons , 2019, Pattern Recognit. Lett..

[53]  Tasawar Hayat,et al.  Application of reproducing kernel algorithm for solving second-order, two-point fuzzy boundary value problems , 2017, Soft Comput..

[54]  David,et al.  Engineering Drawing and Design , 2001 .

[55]  Takeo Igarashi,et al.  Structured annotations for 2D-to-3D modeling , 2009, ACM Trans. Graph..

[56]  Ersin Yumer,et al.  Shape Synthesis from Sketches via Procedural Models and Convolutional Networks , 2017, IEEE Transactions on Visualization and Computer Graphics.

[57]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[58]  Siddhartha Chaudhuri,et al.  CustomCut: On‐demand Extraction of Customized 3D Parts with 2D Sketches , 2016, Comput. Graph. Forum.

[59]  Luisa Di Piazza,et al.  Decompositions of Weakly Compact Valued Integrable Multifunctions , 2020, Mathematics.

[60]  Yi Chen,et al.  Wasserstein blue noise sampling , 2017, TOGS.

[61]  Omar Abu Arqub,et al.  Adaptation of reproducing kernel algorithm for solving fuzzy Fredholm–Volterra integrodifferential equations , 2017, Neural Computing and Applications.

[62]  Marc Alexa,et al.  FiberMesh: designing freeform surfaces with 3D curves , 2007, ACM Trans. Graph..