Plenoptic Image Editing

This paper presents a new class of interactive image editing operations designed to maintain consistency between multiple images of a physical 3D scene. The distinguishing feature of these operations is that edits to any one image propagate automatically to all other images as if the (unknown) 3D scene had itself been modified. The modified scene can then be viewed interactively from any other camera viewpoint and under different scene illuminations. The approach is useful first as a power-assist that enables a user to quickly modify many images by editing just a few, and second as a means for constructing and editing image-based scene representations by manipulating a set of photographs. The approach works by extending operations like image painting, scissoring, and morphing so that they alter a scene's generalized plenoptic function in a physically-consistent way, thereby affecting scene appearance from all viewpoints simultaneously. A key element in realizing these operations is a new volumetric decomposition technique for reconstructing an scene's plenoptic function from an incomplete set of camera viewpoints.

[1]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[2]  Barthold Lichtenbelt,et al.  Introduction to volume rendering , 1998 .

[3]  Gregory M. Nielson,et al.  Scattered data modeling , 1993, IEEE Computer Graphics and Applications.

[4]  Michael Gleicher,et al.  This document was created with FrameMaker 4.0.4 Image Snapping , 2022 .

[5]  Stephen A. Benton,et al.  Survey Of Holographic Stereograms , 1983, Optics & Photonics.

[6]  John K. Tsotsos,et al.  Active object recognition , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Olivier D. Faugeras,et al.  Variational principles, surface evolution, PDEs, level set methods, and the stereo problem , 1998, IEEE Trans. Image Process..

[8]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[9]  William A. Barrett,et al.  Intelligent scissors for image composition , 1995, SIGGRAPH.

[10]  Steven M. Seitz,et al.  Physically-valid view synthesis by image interpolation , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[11]  Kiriakos N. Kutulakos.  Shape from the light field boundary , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  R. Woodham,et al.  Photometric Stereo: Lambertian Reflectance and Light Sources with Unknown Direction and Strength , 1991 .

[13]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[14]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[15]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[16]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[17]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[18]  Amnon Shashua,et al.  Novel View Synthesis by Cascading Trilinear Tensors , 1998, IEEE Trans. Vis. Comput. Graph..

[19]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1992, SIGGRAPH.

[20]  Stephen H. Westin,et al.  A global illumination solution for general reflectance distributions , 1991, SIGGRAPH.

[21]  HanrahanPat,et al.  Direct WYSIWYG painting and texturing on 3D shapes , 1990 .

[22]  Peter Schröder,et al.  Spherical wavelets: efficiently representing functions on the sphere , 1995, SIGGRAPH.

[23]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[24]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[25]  Robert T. Collins,et al.  A space-sweep approach to true multi-image matching , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.

[27]  David Salesin,et al.  Wavelets for computer graphics: a primer. 2 , 1995, IEEE Computer Graphics and Applications.

[28]  Richard Szeliski,et al.  3-D Scene Data Recovery Using Omnidirectional Multibaseline Stereo , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Takeo Kanade,et al.  Constructing virtual worlds using dense stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[30]  A. Shashua.  Geometry and Photometry in 3D Visual Recognition , 1992 .

[31]  Alan L. Yuille,et al.  Learning Object Representation form Lighting Variations , 1996, Object Representation in Computer Vision.

[32]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[33]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[34]  Ramesh C. Jain,et al.  Reality modeling and visualization from multiple video sequences , 1996, IEEE Computer Graphics and Applications.

[35]  Amnon Shashua,et al.  Novel view synthesis in tensor space , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  P. Belhumeur,et al.  Learning Object Representations from LightingVariationsR , 1996 .

[37]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[39]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.