Artistic Augmentation of Photographs with Droplets

Artistic augmentation of photographs with water droplets aims at generating aesthetic yet realistic images, and thus differs from traditional augmented reality in two aspects. One difference lies in the adoption of a new image as the environment map in order to render reflected or refracted effects on the surface of inserted water droplets. The other difference is in modeling of water droplets including hanging droplets and resting droplets. These differences raise two research challenges: 1) how to adjust the brightness and colors of the new environment map to maintain visual consistency between the new environment map and the original input image; 2) how to model hanging and resting droplets aesthetically. This paper proposes a framework that addresses these two challenges and demonstrates the effectiveness of our framework by generating example augmented images.

[1]  Kazufumi Kaneda,et al.  A volume-preserving approach for modeling and animating water flows generated by metaballs , 2002, The Visual Computer.

[2]  Yannick Hold-Geoffroy,et al.  Deep Outdoor Illumination Estimation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Zhang Tin.  Computer Generation of 3D Inscriptions from 2D Images of Chinese Calligraphy , 2014 .

[4]  Erik Reinhard,et al.  Image-based material editing , 2005, SIGGRAPH '05.

[5]  Yannick Hold-Geoffroy,et al.  A Perceptual Measure for Deep Single Image Camera Calibration , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Erik Reinhard,et al.  Image-based material editing , 2005, SIGGRAPH '05.

[7]  Andrew Gardner,et al.  Photorealistic rendering of mixed reality scenes , 2015, Comput. Graph. Forum.

[8]  Edward H. Adelson,et al.  Ground truth dataset and baseline evaluations for intrinsic image algorithms , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[9]  D. Hamlyn,et al.  The Psychology of Perception: A Philosophical Examination of Gestalt Theory and Derivative Theories of Perception , 2017 .

[10]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[11]  David A. Forsyth,et al.  Rendering synthetic objects into legacy photographs , 2011, ACM Trans. Graph..

[12]  Qunsheng Peng,et al.  On-line Illumination Estimation of Outdoor Scenes Based on Area Selection for Augmented Reality , 2011, 2011 12th International Conference on Computer-Aided Design and Computer Graphics.

[13]  Paul Graham,et al.  A single-shot light probe , 2012, SIGGRAPH '12.

[14]  Kazufumi Kaneda,et al.  Animation of water droplets moving down a surface , 1999, Comput. Animat. Virtual Worlds.

[15]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Qunsheng Peng,et al.  A practical approach for real-time illumination estimation of outdoor videos , 2012, Comput. Graph..

[17]  L. M. M.-T..  Spherical Harmonics: an Elementary Treatise on Harmonic Functions, with Applications , 1928, Nature.

[18]  Erik Reinhard,et al.  Compositing images through light source detection , 2010, Comput. Graph..

[19]  K. Schroën,et al.  Microfluidic EDGE emulsification: the importance of interface interactions on droplet formation and pressure stability , 2016, Scientific Reports.

[20]  T. MacRobert.  Spherical harmonics : an elementary treatise on harmonic functions , 1927 .

[21]  Lawrence B. Wolff,et al.  Using polarization to separate reflection components , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Brian Wyvill,et al.  Robust iso-surface tracking for interactive character skinning , 2014, ACM Trans. Graph..

[23]  Pat Hanrahan,et al.  An efficient representation for irradiance environment maps , 2001, SIGGRAPH.

[24]  Alexei A. Efros,et al.  What Do the Sun and the Sky Tell Us About the Camera? , 2010, International Journal of Computer Vision.

[25]  Alexei A. Efros,et al.  Estimating natural illumination from a single outdoor image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[26]  Ihsan Rabbi.  Augmented Reality Tracking Techniques: A Systematic Literature Review Protocol , 2012 .

[27]  P. Hanrahan,et al.  On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[28]  Shin-Tson Wu,et al.  Effects of gravity on the shape of liquid droplets , 2010 .

[29]  Fabio Pellacini,et al.  envyLight: an interface for editing natural illumination , 2010, ACM Trans. Graph..

[30]  Xavier Granier,et al.  Online Tracking of Outdoor Lighting Variations for Augmented Reality with Moving Cameras , 2012, IEEE Transactions on Visualization and Computer Graphics.

[31]  Alexei A. Efros,et al.  Estimating the Natural Illumination Conditions from a Single Outdoor Image , 2012, International Journal of Computer Vision.

[32]  Sylvain Paris,et al.  User-assisted intrinsic images , 2009, ACM Trans. Graph..

[33]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[34]  Hwan-Gue Cho,et al.  A new water droplet model using metaball in the gravitational field , 1999, Comput. Graph..

[35]  James F. Blinn,et al.  A Generalization of Algebraic Surface Drawing , 1982, TOGS.

[36]  Francesco Banterle,et al.  Inverse tone mapping , 2006, GRAPHITE '06.

[37]  André Gagalowicz,et al.  Image-based rendering of diffuse, specular and glossy surfaces from a single image , 2001, SIGGRAPH.

[38]  James E. Miller,et al.  Modelling And Rendering Liquids In Motion , 1999 .

[39]  K. Kaneda,et al.  Animation of Water Droplets on a Glass Plate , 1993 .

[40]  Alexei A. Efros,et al.  Webcam clip art: appearance and illuminant transfer from time-lapse sequences , 2009, ACM Trans. Graph..

[41]  A EfrosAlexei,et al.  Webcam clip art , 2009 .

[42]  Luc Van Gool,et al.  What is Around the Camera? , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[43]  Sehat Ullah,et al.  A Survey on Augmented Reality Challenges and Tracking , 2016 .

[44]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[45]  Qunsheng Peng,et al.  Lighting Simulation of Augmented Outdoor Scene Based on a Legacy Photograph , 2013, Comput. Graph. Forum.

[46]  G. Faris,et al.  Droplet shape analysis and permeability studies in droplet lipid bilayers. , 2012, Langmuir : the ACS journal of surfaces and colloids.

[47]  Kalyan Sunkavalli,et al.  Automatic Scene Inference for 3D Object Compositing , 2014, ACM Trans. Graph..

[48]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[49]  Alexei A. Efros,et al.  Understanding and Recreating Visual Appearance Under Natural Illumination , 2011 .

[50]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[51]  Qunsheng Peng,et al.  Light source estimation of outdoor scenes for mixed reality , 2009, The Visual Computer.

[52]  Alexei A. Efros,et al.  Webcam clip art: appearance and illuminant transfer from time-lapse sequences , 2009, SIGGRAPH 2009.

[53]  Song Wang,et al.  Restoration of Brick and Stone Relief from Single Rubbing Images , 2012, IEEE Transactions on Visualization and Computer Graphics.

[54]  Stephen H. Westin,et al.  A global illumination solution for general reflectance distributions , 1991, SIGGRAPH.

[55]  D. W. F. van Krevelen,et al.  A Survey of Augmented Reality Technologies, Applications and Limitations , 2010, Int. J. Virtual Real..

[56]  Erik Reinhard,et al.  Real-time color blending of rendered and captured video , 2004 .

[57]  Karol Myszkowski,et al.  High Dynamic Range Imaging and Low Dynamic Range Expansion for Generating HDR Content , 2009, Eurographics.

[58]  J. Michalsky,et al.  All-weather model for sky luminance distribution—Preliminary configuration and validation , 1993 .

[59]  Ersin Yumer,et al.  Learning to predict indoor illumination from a single image , 2017, ACM Trans. Graph..

[60]  Pierre Poulin,et al.  Simulating the Flow of Liquid Droplets , 1998, Graphics Interface.

[61]  Jessica K. Hodgins,et al.  Dynamic simulation of splashing fluids , 1995, Proceedings Computer Animation'95.

[62]  Andrew Jones,et al.  Direct HDR capture of the sun and sky , 2006, SIGGRAPH Courses.