Physically-based modeling techniques for interactive digital painting

WILLIAM VALENTINE BAXTER III: Physically-Based Modeling Techniques for Interactive Digital Painting (Under the direction of Ming C. Lin) In this dissertation I present a novel, physically-based approach to digital painting. With the interactive simulation techniques I present, digital painters can work with digital brushes and paints whose behavior is similar to real ones. Using this physicallybased approach, a digital painting system can provide artists with a versatile and expressive creative tool, while at the same time providing a more natural style of interaction enabled by the emulation of real-world implements. I introduce several specific modeling techniques for digital painting. First, I present a physically-based, 3D, deformable, virtual brush model based on non-linear quasi-static constrained energy minimization. The brush dynamics are computed using a skeletal physical model, which then determines the motion of a more complex geometric model. I also present three different models for capturing the dynamic behavior of viscous paint media, each offering a different trade-off between speed and fidelity—from 2D heuristics, to 3D partial differential equations. Accurate modeling of the optical behavior of paint mixtures and glazes is also important, and for this I present a real-time, physicallybased rendering technique, based on the Kubelka-Munk equations and an eight-sample color space. Finally, I present techniques for modeling the haptic response of brushes in an artist’s hand, and demonstrate that all these techniques can be combined to provide the digital painter with an interactive, virtual painting system with a working style similar to real-world painting.

[1]  Hiroo Iwata,et al.  Volume haptization , 1993, Proceedings of 1993 IEEE Research Properties in Virtual Reality Symposium.

[2]  David Salesin,et al.  Computer-generated pen-and-ink illustration , 1994, SIGGRAPH.

[3]  Andrew P. Witkin,et al.  Spacetime constraints , 1988, SIGGRAPH.

[4]  Ricardo S. Avila,et al.  A haptic interaction method for volume visualization , 1996, Proceedings of Seventh Annual IEEE Visualization '96.

[5]  Qinglian Guo,et al.  "Nijimi" rendering algorithm for creating quality black ink paintings , 2003, Proceedings Computer Graphics International 2003.

[6]  Xiaoyuan Tu,et al.  Artificial Animals for Computer Animation: Biomechanics, Locomotion, Perception, and Behavior , 1999, Lecture Notes in Computer Science.

[7]  Gary W. Meyer,et al.  Modeling pigmented materials for realistic image synthesis , 1992, TOGS.

[8]  Ramesh Raskar,et al.  Dynamic Shader Lamps: Painting on Real Objects , 2001 .

[9]  Malcolm Tunde Cockshott.  Wet and sticky : a novel model for computer-based painting , 1991 .

[10]  John L. Sibert,et al.  Charcoal sketching: returning control to the artist , 1988, TOGS.

[11]  John Kenneth Salisbury,et al.  Haptic rendering: programming touch interaction with virtual objects , 1995, I3D '95.

[12]  C. W. Hirt,et al.  Improved free surface boundary conditions for numerical incompressible-flow calculations , 1971 .

[13]  Aaron Hertzmann,et al.  Fast paint texture , 2002, NPAR '02.

[14]  Yunhe Pan,et al.  Advanced Design for a Realistic Virtual Brush , 2003, Comput. Graph. Forum.

[15]  T. N. Stevenson,et al.  Fluid Mechanics , 2021, Nature.

[16]  Dore Ashton,et al.  Picasso on Art: A Selection of Views , 1972 .

[17]  David England,et al.  Modelling the Texture of Paint , 1992, Comput. Graph. Forum.

[18]  Daniel J. Inman,et al.  Engineering Mechanics: Statics , 1998 .

[19]  John M. Hollerbach,et al.  Haptic manipulation of virtual mechanisms from mechanical CAD designs , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[20]  Oussama Khatib,et al.  The haptic display of complex graphical environments , 1997, SIGGRAPH.

[21]  Aaron Hertzmann,et al.  Painterly rendering with curved brush strokes of multiple sizes , 1998, SIGGRAPH.

[22]  Seibert Q. Duntley,et al.  The Optical Properties of Diffusing Materials , 1942 .

[23]  James F. O'Brien,et al.  A method for animating viscoelastic fluids , 2004, ACM Trans. Graph..

[24]  Der-Lor Way,et al.  Physical-based Model of Ink Diffusion in Chinese Paintings , 2003, WSCG.

[25]  A. Chorin.  Numerical solution of the Navier-Stokes equations , 1968 .

[26]  Tosiyasu L. Kunii,et al.  A diffusion model for computer animation of diffuse ink painting , 1995, Proceedings Computer Animation'95.

[27]  Binh Pham.  Expressive brush strokes , 1991, CVGIP Graph. Model. Image Process..

[28]  Jintae Lee.  Simulating Oriental Black-Ink Painting , 1999, IEEE Computer Graphics and Applications.

[29]  Simon Schofield.  Non-photorealistic rendering: a critical examination and proposed system. , 1994 .

[30]  Edwin Earl Catmull,et al.  A subdivision algorithm for computer display of curved surfaces. , 1974 .

[31]  Steve Strassmann,et al.  Hairy brushes , 1986, SIGGRAPH.

[32]  George Fitzmaurice,et al.  Bimanual Manipulation in a Curve Editing Task , 1998 .

[33]  C. W. Hirt,et al.  Free-surface stress conditions for incompressible-flow calculations☆ , 1968 .

[34]  Nelson Siu-Hang Chu,et al.  Real-time painting with an expressive virtual Chinese brush , 2004, IEEE Computer Graphics and Applications.

[35]  Lambertus Hesselink,et al.  Representation and display of vector field topology in fluid flow data sets , 1989, Computer.

[36]  Brian Wyvill,et al.  Rendering cracks in Batik , 2004, NPAR '04.

[37]  F. IAN G. RAWLINS,et al.  The Measurement of Colour , 1945, Nature.

[38]  Richard Greene,et al.  The drawing prism: a versatile graphic input device , 1985, SIGGRAPH.

[39]  Ming Ouhyoung,et al.  On the effects of haptic display in brush and ink simulation for Chinese painting and calligraphy , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[40]  F. Harlow,et al.  Numerical Calculation of Time‐Dependent Viscous Incompressible Flow of Fluid with Free Surface , 1965 .

[41]  Alvy Ray Smith.  Digital Paint Systems: An Anecdotal and Historical Overview , 2001, IEEE Ann. Hist. Comput..

[42]  Sarah F. Frisken.  Beyond volume rendering: visualization, haptic exploration, and physical modeling of voxel-based objects , 1995, Visualization in Scientific Computing.

[43]  Gerald H. Ristow,et al.  Particles moving in spatially bounded, viscous fluids , 1996 .

[44]  James J. Troy,et al.  Six degree-of-freedom haptic rendering using voxel sampling , 1999, SIGGRAPH.

[45]  Dieter W. Fellner,et al.  The circle-brush algorithm , 1988, TOGS.

[46]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[47]  David Salesin,et al.  Rendering parametric surfaces in pen and ink , 1996, SIGGRAPH.

[48]  Frank Van Reeth,et al.  A Paper Model for Real-time Watercolor Simulation , 2004 .

[49]  Guillaume Poirier,et al.  Human Skin Modelling and Rendering , 2004 .

[50]  Horace Ho-Shing Ip,et al.  Virtual brush: a model-based synthesis of Chinese calligraphy , 2000, Comput. Graph..

[51]  David H. Small,et al.  Simulating watercolor by modeling diffusion, pigment, and paper fibers , 1991, Electronic Imaging.

[52]  Paul Johnson,et al.  Art: A New History , 2003 .

[53]  Mario Costa Sousa,et al.  Computer-generated graphite pencil materials and rendering , 1999 .

[54]  G. R. Wilkinson.  Reflectance Spectroscopy , 1968, Nature.

[55]  Gerald H. Ristow.  Tumbling Motion Of Elliptical Particles In Viscous Two-Dimensional Fluids , 2001 .

[56]  William V. Baxter,et al.  DAB: Interactive Haptic Painting with 3D Virtual Brushes , 2001, SIGGRAPH Courses.

[57]  D. Kershaw.  The incomplete Cholesky—conjugate gradient method for the iterative solution of systems of linear equations , 1978 .

[58]  Gavin S. P. Miller,et al.  Rapid, stable fluid dynamics for computer graphics , 1990, SIGGRAPH.

[59]  Lambertus Hesselink,et al.  Surface representations of two- and three-dimensional fluid flow topology , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[60]  J. Lebensohn.  Color in Business, Science, and Industry , 1952 .

[61]  Aaron Hertzmann,et al.  Paint by relaxation , 2001, Proceedings. Computer Graphics International 2001.

[62]  Ming C. Lin,et al.  inTouch: interactive multiresolution modeling and 3D painting with a haptic interface , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[63]  J. Edward Colgate,et al.  Factors affecting the Z-Width of a haptic display , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[64]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[65]  Elaine Cohen,et al.  Haptic interfacing for virtual prototyping of mechanical CAD designs , 1997 .

[66]  Barbara J. Meier.  Painterly rendering for animation , 1996, SIGGRAPH.

[67]  Turner Whitted,et al.  Anti-aliased line drawing using brush extrusion , 1983, SIGGRAPH.

[68]  Jos Stam,et al.  Stable fluids , 1999, SIGGRAPH.

[69]  Ronald Fedkiw,et al.  Visual simulation of smoke , 2001, SIGGRAPH.

[70]  Lance Williams,et al.  3D paint , 1990, I3D '90.

[71]  C. W. Hirt,et al.  Volume of fluid (VOF) method for the dynamics of free boundaries , 1981 .

[72]  P. Kubelka,et al.  New Contributions to the Optics of Intensely Light-Scattering Materials. Part I , 1948 .

[73]  Bruce Gooch,et al.  Non-photorealistic rendering , 2001 .

[74]  M. F. Tomé,et al.  GENSMAC: a computational marker and cell method for free surface flows in general domains , 1994 .

[75]  Philip J. Willis,et al.  A Physically Based Colour Model , 1991, Comput. Graph. Forum.

[76]  Joe DeMaio,et al.  Introduction to Color , 1997 .

[77]  Marie-Paule Cani,et al.  Interactive animation of ocean waves , 2002, SCA '02.

[78]  Lillian F. Schwartz,et al.  The Computer Artist's Handbook: Concepts, Techniques, and Applications , 1992 .

[79]  Sunil K. Agrawal,et al.  Optimization of Dynamic Systems , 1999 .

[80]  Roy Hall,et al.  Illumination and Color in Computer Generated Imagery , 1988, Monographs in Visual Communication.

[81]  Pat Hanrahan,et al.  Modeling and rendering of metallic patinas , 1996, SIGGRAPH.

[82]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[83]  John Fleming,et al.  The Visual Arts: A History , 1982 .

[84]  Hwan-Gue Cho,et al.  A model based technique for realistic oriental painting , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[85]  Elizabeth Gilmore Holt,et al.  Theories of Modern Art , 1970 .

[86]  Hwan-Gue Cho,et al.  Interactive Rendering Technique for Realistic Oriental Painting , 2003, WSCG.

[87]  David E. Breen,et al.  Cloth modeling and animation , 2000 .

[88]  Mathieu Desbrun,et al.  Smoothed particles: a new paradigm for animating highly deformable bodies , 1996 .

[89]  Irfan A. Essa,et al.  Image and video based painterly animation , 2004, NPAR '04.

[90]  Alexei Sourin,et al.  Functionally based virtual computer art , 2001, I3D '01.

[91]  Jessica K. Hodgins,et al.  Dynamic simulation of splashing fluids , 1995, Proceedings Computer Animation'95.

[92]  Markus H. Gross,et al.  Interactive 3D Painting on Point-Sampled Objects , 2004, PBG.

[93]  Hiroo Iwata,et al.  Artificial reality with force-feedback: development of desktop virtual space with compact master manipulator , 1990, SIGGRAPH.

[94]  Elaine Cohen,et al.  Painting textures with a haptic interface , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[95]  Frank Van Reeth,et al.  Real-time watercolor painting on a distributed paper model , 2004, Proceedings Computer Graphics International, 2004..

[96]  William V. Baxter,et al.  A viscous paint model for interactive applications , 2004, SIGGRAPH '04.

[97]  Ming C. Lin,et al.  Haptic display of interaction between textured models , 2004, IEEE Visualization 2004.

[98]  HanrahanPat,et al.  Direct WYSIWYG painting and texturing on 3D shapes , 1990 .

[99]  Colin Ware,et al.  Bat brushes: on the uses of six position and orientation parameters in a paint program , 1989, CHI '89.

[100]  W. Buxton,et al.  A study in two-handed input , 1986, CHI '86.

[101]  Elaine Cohen,et al.  Direct haptic rendering of sculptured models , 1997, SI3D.

[102]  Markus H. Gross,et al.  Particle-based fluid simulation for interactive applications , 2003, SCA '03.

[103]  John P. Lewis,et al.  Texture synthesis for digital painting , 1984, SIGGRAPH.

[104]  Christopher A. Maines,et al.  Color Science in the Examination of Museum Objects , 2002 .

[105]  William Buxton,et al.  A taxonomy of see-through tools , 1994, CHI '94.

[106]  Gary W. Meyer,et al.  Wavelength selection for synthetic image generation , 1988, Comput. Vis. Graph. Image Process..

[107]  Michael Griebel,et al.  Numerical Simulation in Fluid Dynamics: A Practical Introduction , 1997 .

[108]  Ming C. Lin,et al.  Sensation preserving simplification for haptic rendering , 2003, SIGGRAPH Courses.

[109]  Ergun Akleman,et al.  Two methods for creating chinese painting , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[110]  Jim X. Chen,et al.  Toward Interactive-Rate Simulation of Fluids with Moving Obstacles Using Navier-Stokes Equations , 1995, CVGIP Graph. Model. Image Process..

[111]  Ronald Fedkiw,et al.  Animation and rendering of complex water surfaces , 2002, ACM Trans. Graph..

[112]  Nelson Siu-Hang Chu,et al.  An efficient brush model for physically-based 3D painting , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[113]  Patrick Callet,et al.  Pertinent Data for Modelling Pigmented Materials in Realistic Rendering , 1996, Comput. Graph. Forum.

[114]  John J. Bloomer.  Practical Fluid Mechanics for Engineering Applications , 1999 .

[115]  Eric Neufeld,et al.  Simulating wax crayons , 2003, 11th Pacific Conference onComputer Graphics and Applications, 2003. Proceedings..

[116]  Lucy Y. Pao,et al.  Shock and vortex visualization using a combined visual/haptic interface , 2000, Proceedings Visualization 2000. VIS 2000 (Cat. No.00CH37145).

[117]  R. LeVeque.  Numerical methods for conservation laws , 1990 .

[118]  Richard Shoup.  SuperPaint: An Early Frame Buffer Graphics System , 2001, IEEE Ann. Hist. Comput..

[119]  Qing Zhang,et al.  Simple cellular automaton-based simulation of ink behaviour and its application to Suibokuga-like 3D rendering of trees , 1999, Comput. Animat. Virtual Worlds.

[120]  Yunhe Pan,et al.  A Solid Model Based Virtual Hairy Brush , 2002, Comput. Graph. Forum.

[121]  HesselinkLambertus,et al.  Representation and Display of Vector Field Topology in Fluid Flow Data Sets , 1989 .

[122]  Peter Litwinowicz,et al.  Processing images and video for an impressionist effect , 1997, SIGGRAPH.

[123]  Siu Chi Hsu,et al.  Skeletal strokes , 1993, UIST '93.

[124]  Ivan E. Sutherland,et al.  Sketchpad a Man-Machine Graphical Communication System , 1899, Outstanding Dissertations in the Computer Sciences.

[125]  J. Shewchuk.  An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[126]  Mark D. Fairchild,et al.  Full-Spectral Color Calculations in Realistic Image Synthesis , 1999, IEEE Computer Graphics and Applications.

[127]  Marc Levoy,et al.  3D painting on scanned surfaces , 1995, I3D '95.

[128]  A Acrivos,et al.  Non-Newtonian Fluids , 1985 .

[129]  Eric Neufeld,et al.  A Bidirectional Deposition Model of Wax Crayons , 2005, Comput. Graph. Forum.

[130]  Paul Kabbash,et al.  Human performance using computer input devices in the preferred and non-preferred hands , 1993, INTERCHI.

[131]  Alvy Ray Smith,et al.  PAINT , 1998, Herz Kardiovaskuläre Erkrankungen.

[132]  Lisa J. K. Durbeck,et al.  SCIRun Haptic Display for Scientific Visualization , 1999 .

[133]  Michael Gleicher,et al.  Retargetting motion to new characters , 1998, SIGGRAPH.

[134]  Andrew Witkin,et al.  Physically Based Modeling: Principles and Practice , 1997 .

[135]  Greg Turk,et al.  Melting and flowing , 2002, SCA '02.

[136]  Ming C. Lin,et al.  ArtNova: touch-enabled 3D model design , 2005, SIGGRAPH Courses.

[137]  D. Duncan.  The colour of pigment mixtures , 1940 .

[138]  Abigail Sellen,et al.  Two-handed input in a compound task , 1994, CHI Conference Companion.

[139]  Jintae Lee,et al.  Diffusion rendering of black ink paintings using new paper and ink models , 2001, Comput. Graph..

[140]  Qinglian Guo,et al.  Modeling the Diffuse Paintings of 'Sumie' , 1991, Modeling in Computer Graphics.

[141]  Siu Chi Hsu,et al.  Drawing and animation using skeletal strokes , 1994, SIGGRAPH.

[142]  Vincent Hayward,et al.  Haptic interfaces and devices , 2004 .

[143]  B. Hapke.  Theory of reflectance and emittance spectroscopy , 1993 .

[144]  Stefan Schlechtweg,et al.  Non-photorealistic computer graphics: modeling, rendering, and animation , 2002 .

[145]  William V. Baxter,et al.  A versatile interactive 3D brush model , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..