Interactive Light Field Editing and Compositing

Light fields can be used to represent an object’s appearance with a high degree of realism. However, unlike their geometric counterparts, these image-based representations lack user control for editing and manipulating them. We present a system that allows a user to interactively edit and composite light fields. We introduce five basic operators, alpha-compositing, apply, multiplication, warping, and projection, which may be used in combination. To describe how multiple operators and light fields are combined together, we introduce a compositing language similar to those used for combining shader programs. To enable real-time compositing, we have developed a framework for the programmable graphics hardware that implements the five basic operators and provides mechanisms for users to develop combinations of these. We built an interactive system using this framework and demonstrate its use for editing and authoring new light fields.

[1]  Raymond Fielding,et al.  The technique of special effects cinematography , 1965 .

[2]  Robert L. Cook,et al.  Shade trees , 1984, SIGGRAPH.

[3]  Tom Duff,et al.  Compositing digital images , 1984, SIGGRAPH.

[4]  Ken Perlin,et al.  An image synthesizer , 1988 .

[5]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[6]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[7]  Hans-Peter Seidel,et al.  Light Field Techniques for Reflections and Refractions , 1999, Rendering Techniques.

[8]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[9]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[10]  Frédo Durand,et al.  A gentle introduction to bilateral filtering and its applications , 2007, SIGGRAPH Courses.

[11]  Pat Hanrahan,et al.  A real-time procedural shading system for programmable graphics hardware , 2001, SIGGRAPH.

[12]  Baining Guo,et al.  Feature-based light field morphing , 2002, ACM Trans. Graph..

[13]  Adam W. Bargteil,et al.  Introducing Vision-Realistic Rendering , 2002 .

[14]  Harry Shum,et al.  Pop-up light field: An interactive image-based modeling and rendering system , 2004, TOGS.

[15]  Marc Levoy,et al.  Using plane + parallax for calibrating dense camera arrays , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[16]  Brian A. Barsky.  Vision-realistic rendering: simulation of the scanned foveal image from wavefront data of human subjects , 2004, APGV '04.

[17]  Kiriakos N. Kutulakos,et al.  Plenoptic Image Editing , 2004, International Journal of Computer Vision.

[18]  Billy Chen,et al.  Light Source Interpolation for Sparsely Sampled Reflectance Fields , 2005 .

[19]  Harry Shum,et al.  Interactive deformation of light fields , 2005, I3D '05.

[20]  Leonard McMillan,et al.  Real-time reflection mapping with parallax , 2005, I3D '05.

[21]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..