Physically-based modeling techniques for interactive digital painting

In this dissertation I present a novel, physically-based approach to digital painting. With the interactive simulation techniques I present, digital painters can work with digital brushes and paints whose behavior is similar to real ones. Using this physically-based approach, a digital painting system can provide artists with a versatile and expressive creative tool, while at the same time providing a more natural style of interaction enabled by the emulation of real-world implements. I introduce several specific modeling techniques for digital painting. First, I present a physically-based, 3D, deformable, virtual brush model based on non-linear quasi-static constrained energy minimization. The brush dynamics are computed using a skeletal physical model, which then determines the motion of a more complex geometric model. I also present three different models for capturing the dynamic behavior of viscous paint media, each offering a different trade-off between speed and fidelity—from 2D heuristics, to 3D partial differential equations. Accurate modeling of the optical behavior of paint mixtures and glazes is also important, and for this I present a real-time, physically-based rendering technique, based on the Kubelka-Munk equations and an eight-sample color space. Finally, I present techniques for modeling the haptic response of brushes in an artist's hand, and demonstrate that all these techniques can be combined to provide the digital painter with an interactive, virtual painting system with a working style similar to real-world painting.

[1]  Ming C. Lin,et al.  Haptic display of interaction between textured models , 2004, IEEE Visualization 2004.

[2]  HanrahanPat,et al.  Direct WYSIWYG painting and texturing on 3D shapes , 1990 .

[3]  Colin Ware,et al.  Bat brushes: on the uses of six position and orientation parameters in a paint program , 1989, CHI '89.

[4]  W. Buxton,et al.  A study in two-handed input , 1986, CHI '86.

[5]  Elaine Cohen,et al.  Direct haptic rendering of sculptured models , 1997, SI3D.

[6]  Markus H. Gross,et al.  Particle-based fluid simulation for interactive applications , 2003, SCA '03.

[7]  John M. Hollerbach,et al.  Haptic manipulation of virtual mechanisms from mechanical CAD designs , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[8]  Aaron Hertzmann,et al.  Painterly rendering with curved brush strokes of multiple sizes , 1998, SIGGRAPH.

[9]  Der-Lor Way,et al.  Physical-based Model of Ink Diffusion in Chinese Paintings , 2003, WSCG.

[10]  A. Chorin Numerical solution of the Navier-Stokes equations , 1968 .

[11]  F. Harlow,et al.  Numerical Calculation of Time‐Dependent Viscous Incompressible Flow of Fluid with Free Surface , 1965 .

[12]  D. Duncan The colour of pigment mixtures , 1940 .

[13]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[14]  David Salesin,et al.  Rendering parametric surfaces in pen and ink , 1996, SIGGRAPH.

[15]  Horace Ho-Shing Ip,et al.  Virtual brush: a model-based synthesis of Chinese calligraphy , 2000, Comput. Graph..

[16]  William V. Baxter,et al.  DAB: Interactive Haptic Painting with 3D Virtual Brushes , 2001, SIGGRAPH Courses.

[17]  Edwin Earl Catmull,et al.  A subdivision algorithm for computer display of curved surfaces. , 1974 .

[18]  David H. Small,et al.  Simulating watercolor by modeling diffusion, pigment, and paper fibers , 1991, Electronic Imaging.

[19]  Jintae Lee,et al.  Diffusion rendering of black ink paintings using new paper and ink models , 2001, Comput. Graph..

[20]  Qinglian Guo,et al.  Modeling the Diffuse Paintings of 'Sumie' , 1991, Modeling in Computer Graphics.

[21]  James J. Troy,et al.  Six degree-of-freedom haptic rendering using voxel sampling , 1999, SIGGRAPH.

[22]  D. B. Judd,et al.  Color in Business, Science, and Industry , 1953 .

[23]  D. Kershaw The incomplete Cholesky—conjugate gradient method for the iterative solution of systems of linear equations , 1978 .

[24]  Dieter W. Fellner,et al.  The circle-brush algorithm , 1988, TOGS.

[25]  P. Kubelka Ein Beitrag zur Optik der Farban striche , 1931 .

[26]  B. Hapke Theory of reflectance and emittance spectroscopy , 1993 .

[27]  Guillaume Poirier,et al.  Human Skin Modelling and Rendering , 2004 .

[28]  Stefan Schlechtweg,et al.  Non-photorealistic computer graphics: modeling, rendering, and animation , 2002 .

[29]  William V. Baxter,et al.  A versatile interactive 3D brush model , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..

[30]  Richard Shoup SuperPaint: An Early Frame Buffer Graphics System , 2001, IEEE Ann. Hist. Comput..

[31]  Roy Hall,et al.  Illumination and Color in Computer Generated Imagery , 1988, Monographs in Visual Communication.

[32]  W. D. Wright The Measurement of Colour , 1945 .

[33]  Simon Schofield Non-photorealistic rendering: a critical examination and proposed system. , 1994 .

[34]  Steve Strassmann,et al.  Hairy brushes , 1986, SIGGRAPH.

[35]  Yunhe Pan,et al.  A Solid Model Based Virtual Hairy Brush , 2002, Comput. Graph. Forum.

[36]  Norishige Chiba,et al.  Simple cellular automaton-based simulation of ink behaviour and its application to suibokuga-like 3d rendering of trees , 1999 .

[37]  Abigail Sellen,et al.  Two-handed input in a compound task , 1994, CHI 1994.

[38]  Hwan-Gue Cho,et al.  Interactive Rendering Technique for Realistic Oriental Painting , 2003, WSCG.

[39]  Lambertus Hesselink,et al.  Representation and display of vector field topology in fluid flow data sets , 1989, Computer.

[40]  Richard Greene,et al.  The drawing prism: a versatile graphic input device , 1985, SIGGRAPH.

[41]  Mathieu Desbrun,et al.  Smoothed particles: a new paradigm for animating highly deformable bodies , 1996 .

[42]  Ming Ouhyoung,et al.  On the effects of haptic display in brush and ink simulation for Chinese painting and calligraphy , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[43]  Markus H. Gross,et al.  Interactive 3D Painting on Point-Sampled Objects , 2004, PBG.

[44]  William V. Baxter,et al.  A viscous paint model for interactive applications , 2004, Comput. Animat. Virtual Worlds.

[45]  Peter Litwinowicz,et al.  Processing images and video for an impressionist effect , 1997, SIGGRAPH.

[46]  Siu Chi Hsu,et al.  Skeletal strokes , 1993, UIST '93.

[47]  Ivan E. Sutherland,et al.  Sketchpad a Man-Machine Graphical Communication System , 1899, Outstanding Dissertations in the Computer Sciences.

[48]  J. Shewchuk An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[49]  Paul Kabbash,et al.  Human performance using computer input devices in the preferred and non-preferred hands , 1993, INTERCHI.

[50]  Lisa J. K. Durbeck,et al.  SCIRun Haptic Display for Scientific Visualization , 1999 .

[51]  Michael Gleicher,et al.  Retargetting motion to new characters , 1998, SIGGRAPH.

[52]  Barbara J. Meier Painterly rendering for animation , 1996, SIGGRAPH.

[53]  Hiroo Iwata,et al.  Artificial reality with force-feedback: development of desktop virtual space with compact master manipulator , 1990, SIGGRAPH.

[54]  A Acrivos,et al.  Non-Newtonian Fluids , 1985 .

[55]  Marie-Paule Cani,et al.  Interactive animation of ocean waves , 2002, SCA '02.

[56]  Elaine Cohen,et al.  Painting textures with a haptic interface , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[57]  Seibert Q. Duntley,et al.  The Optical Properties of Diffusing Materials , 1942 .

[58]  R. M. Evans,et al.  An Introduction to Color , 1948 .

[59]  Greg Turk,et al.  Melting and flowing , 2002, SCA '02.

[60]  Lillian F. Schwartz,et al.  The Computer Artist's Handbook: Concepts, Techniques, and Applications , 1992 .

[61]  Turner Whitted,et al.  Anti-aliased line drawing using brush extrusion , 1983, SIGGRAPH.

[62]  Aaron Hertzmann,et al.  Paint by relaxation , 2001, Proceedings. Computer Graphics International 2001.

[63]  Ming C. Lin,et al.  inTouch: interactive multiresolution modeling and 3D painting with a haptic interface , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[64]  Ergun Akleman,et al.  Two methods for creating chinese painting , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[65]  Jos Stam,et al.  Stable fluids , 1999, SIGGRAPH.

[66]  Ronald Fedkiw,et al.  Visual simulation of smoke , 2001, SIGGRAPH.

[67]  Jim X. Chen,et al.  Toward Interactive-Rate Simulation of Fluids with Moving Obstacles Using Navier-Stokes Equations , 1995, CVGIP Graph. Model. Image Process..

[68]  Eric Neufeld,et al.  A Bidirectional Deposition Model of Wax Crayons , 2005, Comput. Graph. Forum.

[69]  Ronald Fedkiw,et al.  Animation and rendering of complex water surfaces , 2002, ACM Trans. Graph..

[70]  Gary W. Meyer,et al.  Modeling pigmented materials for realistic image synthesis , 1992, TOGS.

[71]  Malcolm Tunde Cockshott Wet and sticky : a novel model for computer-based painting , 1991 .

[72]  Sunil K. Agrawal,et al.  Optimization of Dynamic Systems , 1999 .

[73]  John L. Sibert,et al.  Charcoal sketching: returning control to the artist , 1988, TOGS.

[74]  Gene H. Golub,et al.  Matrix computations , 1983 .

[75]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[76]  Nelson Siu-Hang Chu,et al.  An efficient brush model for physically-based 3D painting , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[77]  Qinglian Guo,et al.  "Nijimi" rendering algorithm for creating quality black ink paintings , 2003, Proceedings Computer Graphics International 2003.

[78]  Xiaoyuan Tu,et al.  Artificial Animals for Computer Animation: Biomechanics, Locomotion, Perception, and Behavior , 1999, Lecture Notes in Computer Science.

[79]  Siu Chi Hsu,et al.  Drawing and animation using skeletal strokes , 1994, SIGGRAPH.

[80]  Vincent Hayward,et al.  Haptic interfaces and devices , 2004 .

[81]  Mark D. Fairchild,et al.  Full-Spectral Color Calculations in Realistic Image Synthesis , 1999, IEEE Computer Graphics and Applications.

[82]  Marc Levoy,et al.  3D painting on scanned surfaces , 1995, I3D '95.

[83]  Hiroo Iwata,et al.  Volume haptization , 1993, Proceedings of 1993 IEEE Research Properties in Virtual Reality Symposium.

[84]  Andrew P. Witkin,et al.  Spacetime constraints , 1988, SIGGRAPH.

[85]  Ricardo S. Avila,et al.  A haptic interaction method for volume visualization , 1996, Proceedings of Seventh Annual IEEE Visualization '96.

[86]  Tosiyasu L. Kunii,et al.  A diffusion model for computer animation of diffuse ink painting , 1995, Proceedings Computer Animation'95.

[87]  Binh Pham Expressive brush strokes , 1991, CVGIP Graph. Model. Image Process..

[88]  Jintae Lee Simulating Oriental Black-Ink Painting , 1999, IEEE Computer Graphics and Applications.

[89]  John P. Lewis,et al.  Texture synthesis for digital painting , 1984, SIGGRAPH.

[90]  Christopher A. Maines,et al.  Color Science in the Examination of Museum Objects , 2002 .

[91]  William Buxton,et al.  A taxonomy of see-through tools , 1994, CHI '94.

[92]  Gary W. Meyer,et al.  Wavelength selection for synthetic image generation , 1988, Comput. Vis. Graph. Image Process..

[93]  Michael Griebel,et al.  Numerical Simulation in Fluid Dynamics: A Practical Introduction , 1997 .

[94]  John Kenneth Salisbury,et al.  Haptic rendering: programming touch interaction with virtual objects , 1995, I3D '95.

[95]  Aaron Hertzmann,et al.  Fast paint texture , 2002, NPAR '02.

[96]  Yunhe Pan,et al.  Advanced Design for a Realistic Virtual Brush , 2003, Comput. Graph. Forum.

[97]  Mario Costa Sousa,et al.  Computer-generated graphite pencil materials and rendering , 1999 .

[98]  Gavin S. P. Miller,et al.  Rapid, stable fluid dynamics for computer graphics , 1990, SIGGRAPH.

[99]  J. Edward Colgate,et al.  Factors affecting the Z-Width of a haptic display , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[100]  David Salesin,et al.  Computer-generated pen-and-ink illustration , 1994, SIGGRAPH.

[101]  Ming C. Lin,et al.  ArtNova: touch-enabled 3D model design , 2002, Proceedings IEEE Virtual Reality 2002.

[102]  Lance Williams,et al.  3D paint , 1990, I3D '90.

[103]  C. W. Hirt,et al.  Volume of fluid (VOF) method for the dynamics of free boundaries , 1981 .

[104]  Bruce Gooch,et al.  Non-photorealistic rendering , 2001 .

[105]  M. F. Tomé,et al.  GENSMAC: a computational marker and cell method for free surface flows in general domains , 1994 .

[106]  Philip J. Willis,et al.  A Physically Based Colour Model , 1991, Comput. Graph. Forum.

[107]  Patrick Callet,et al.  Pertinent Data for Modelling Pigmented Materials in Realistic Rendering , 1996, Comput. Graph. Forum.

[108]  T. Van Laerhoven,et al.  Real-time watercolor painting on a distributed paper model , 2004 .

[109]  John J. Bloomer Practical Fluid Mechanics for Engineering Applications , 1999 .

[110]  Eric Neufeld,et al.  Simulating wax crayons , 2003, 11th Pacific Conference onComputer Graphics and Applications, 2003. Proceedings..

[111]  James F. O'Brien,et al.  A method for animating viscoelastic fluids , 2004, SIGGRAPH 2004.

[112]  R. LeVeque Numerical methods for conservation laws , 1990 .

[113]  Dore Ashton,et al.  Picasso on Art: A Selection of Views , 1972 .

[114]  David England,et al.  Modelling the Texture of Paint , 1992, Comput. Graph. Forum.

[115]  Oussama Khatib,et al.  The haptic display of complex graphical environments , 1997, SIGGRAPH.

[116]  Nelson Siu-Hang Chu,et al.  Real-time painting with an expressive virtual Chinese brush , 2004, IEEE Computer Graphics and Applications.

[117]  Brian Wyvill,et al.  Rendering cracks in Batik , 2004, NPAR '04.

[118]  Alvy Ray Smith Digital Paint Systems: An Anecdotal and Historical Overview , 2001, IEEE Ann. Hist. Comput..

[119]  Paul Johnson,et al.  Art: A New History , 2003 .

[120]  David E. Breen,et al.  Cloth modeling and animation , 2000 .

[121]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[122]  W. F. Riley,et al.  Engineering Mechanics: Statics , 1993 .

[123]  John Fleming,et al.  The Visual Arts: A History , 1984 .

[124]  Hwan-Gue Cho,et al.  A model based technique for realistic oriental painting , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[125]  Elizabeth Gilmore Holt,et al.  Theories of Modern Art , 1970 .

[126]  Irfan A. Essa,et al.  Image and video based painterly animation , 2004, NPAR '04.

[127]  Alexei Sourin,et al.  Functionally based virtual computer art , 2001, I3D '01.

[128]  Jessica K. Hodgins,et al.  Dynamic simulation of splashing fluids , 1995, Proceedings Computer Animation'95.

[129]  Lucy Y. Pao,et al.  Shock and vortex visualization using a combined visual/Haptic interface , 2000 .