LightShop: interactive light field manipulation and rendering

Light fields can be used to represent an object's appearance with a high degree of realism. However, unlike their geometric counterparts, these image-based representations lack user control for manipulating them. We present a system that allows a user to interactively manipulate, composite and render multiple light fields. LightShop is a modular system consisting of three parts: 1) a set of functions that allow a user to model a scene containing multiple light fields, 2) a ray-shading language that describes how an image should be constructed from a set of light fields, and 3) a real-time light field rendering system in OpenGL that can plug into existing 3D engines as a GLSL shader. We show applications in digital photography and we demonstrate how to integrate light fields into a modern space-flight game using LightShop.

[1]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[2]  J. Todd.  Book Review: Digital image processing (second edition). By R. C. Gonzalez and P. Wintz, Addison-Wesley, 1987. 503 pp. Price: £29.95. (ISBN 0-201-11026-1) , 1988 .

[3]  Kiriakos N. Kutulakos,et al.  Plenoptic Image Editing , 2004, International Journal of Computer Vision.

[4]  Wei-Chao Chen,et al.  Light field mapping: efficient representation and hardware rendering of surface light fields , 2002, SIGGRAPH.

[5]  Marc Levoy,et al.  Using plane + parallax for calibrating dense camera arrays , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[6]  Donald S. Fussell,et al.  Uniformly Sampled Light Fields , 1998, Rendering Techniques.

[7]  Harry Shum,et al.  Interactive deformation of light fields , 2005, I3D '05.

[8]  Leonard McMillan,et al.  Real-time reflection mapping with parallax , 2005, I3D '05.

[9]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[10]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[11]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[12]  Dave Shreiner,et al.  OpenGL(R) Programming Guide: The Official Guide to Learning OpenGL(R), Version 2.1 , 2007 .

[13]  Ken Perlin,et al.  An image synthesizer , 1988 .

[14]  Tom Davis,et al.  OpenGL(R) Programming Guide: The Official Guide to Learning OpenGL(R), Version 2 (5th Edition) (OpenGL) , 2005 .

[15]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[16]  Steve D. Upstill,et al.  The renderman companion - a programmer's guide to realistic computer graphics , 1990 .

[17]  Adam W. Bargteil,et al.  Introducing Vision-Realistic Rendering , 2002 .

[18]  Frédo Durand,et al.  A gentle introduction to bilateral filtering and its applications , 2007, SIGGRAPH Courses.

[19]  Wojciech Matusik,et al.  Opacity light fields: interactive rendering of surface light fields with view-dependent opacity , 2003, I3D '03.

[20]  William K. Pratt,et al.  Digital image processing, 2nd Edition , 1991, A Wiley-Interscience publication.

[21]  Hans-Peter Seidel,et al.  Light Field Techniques for Reflections and Refractions , 1999, Rendering Techniques.

[22]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[23]  Tom Duff,et al.  Compositing digital images , 1984, SIGGRAPH.

[24]  Tom Davis,et al.  Opengl programming guide: the official guide to learning opengl , 1993 .

[25]  Harry Shum,et al.  Pop-up light field: An interactive image-based modeling and rendering system , 2004, TOGS.

[26]  Brian A. Barsky.  Vision-realistic rendering: simulation of the scanned foveal image from wavefront data of human subjects , 2004, APGV '04.

[27]  Baining Guo,et al.  Feature-based light field morphing , 2002, ACM Trans. Graph..

[28]  Marc Levoy,et al.  Novel methods for manipulating and combining light fields , 2006 .

[29]  Andrew Jones,et al.  Relighting human locomotion with flowed reflectance fields , 2006, EGSR '06.

[30]  Dipl.-Ing,et al.  Real-time Rendering , 2022 .