Binary-space-partitioned images for resolving image-based visibility

We propose a novel 2D representation for 3D visibility sorting, the binary-space-partitioned image (BSPI), to accelerate real-time image-based rendering. BSPI is an efficient 2D realization of a 3D BSP tree, which is commonly used in computer graphics for time-critical visibility sorting. Since the overall structure of a BSP tree is encoded in a BSPI, traversing a BSPI is comparable to traversing the corresponding BSP tree. BSPI performs visibility sorting efficiently and accurately in the 2D image space by warping the reference image triangle-by-triangle instead of pixel-by-pixel. Multiple BSPIs can be combined to solve "disocclusion," when an occluded portion of the scene becomes visible at a novel viewpoint. Our method is highly automatic, including a tensor voting preprocessing step that generates candidate image partition lines for BSPIs, filters the noisy input data by rejecting outliers, and interpolates missing information. Our system has been applied to a variety of real data, including stereo, motion, and range images.

[1]  Carlo H. Séquin,et al.  Management of large amounts of data in interactive building walkthroughs , 1992, I3D '92.

[2]  Carlo H. Séquin,et al.  Adaptive display algorithm for interactive frame rates during visualization of complex virtual environments , 1993, SIGGRAPH.

[3]  Jr. Leonard McMillan,et al.  An Image-Based Approach to Three-Dimensional Computer Graphics , 1997 .

[4]  Long Quan,et al.  Image interpolation by joint view triangulation , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[5]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[6]  Craig Gotsman,et al.  Output‐SensitiveVisibility Algorithms for Dynamic Scenes with Applications to Virtual Reality , 1996, Comput. Graph. Forum.

[7]  Anselmo Lastra,et al.  LDI tree: a hierarchical representation for image-based rendering , 1999, SIGGRAPH.

[8]  Edwin Earl Catmull,et al.  A subdivision algorithm for computer display of curved surfaces. , 1974 .

[9]  Steven K. Feiner,et al.  Near real-time shadow generation using BSP trees , 1989, SIGGRAPH '89.

[10]  Hayder Radha,et al.  Binary space partitioning tree representation of images , 1991, J. Vis. Commun. Image Represent..

[11]  L. Guibas,et al.  Kinetic vertical decomposition trees , 1999 .

[12]  Ned Greene,et al.  Hierarchical polygon tiling with coverage masks , 1996, SIGGRAPH.

[13]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[14]  D. P. Anderson.  Hidden Line Elimination in Projected Grid Surfaces , 1982, TOGS.

[15]  William R. Mark,et al.  Memory access patterns of occlusion-compatible 3D image warping , 1997, HWWS '97.

[16]  Gavin S. P. Miller,et al.  Hierarchical Z-buffer visibility , 1993, SIGGRAPH.

[17]  Han-Ming Chen,et al.  The feudal priority algorithm on hidden-surface removal , 1996, SIGGRAPH.

[18]  Zbigniew Michalewicz,et al.  Near-Optimal Construction of Partitioning Trees by Evolutionary Techniques , 1995 .

[19]  Guoping Qiu,et al.  Color image coding, indexing and retrieval using binary space partitioning tree , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[20]  Gordon W. Romney,et al.  Half-tone perspective drawings by computer , 1899, AFIPS '67 (Fall).

[21]  Chi-Wing Fu,et al.  Computing Visibility for Triangulated Panoramas , 1999, Rendering Techniques.

[22]  Enric Torres,et al.  Optimization of the Binary Space Partition Algorithm (BSP) for the Visualization of Dynamic Scenes , 1990, Eurographics.

[23]  Donald P. Greenberg,et al.  A visible polygon reconstruction algorithm , 1981, SIGGRAPH '81.

[24]  Mi-Suen Lee,et al.  A Computational Framework for Segmentation and Grouping , 2000 .

[25]  Gérard G. Medioni,et al.  Accurate motion flow estimation with discontinuities , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[26]  Kalpathi R. Subramanian,et al.  Converting Discrete Images to Partitioning Trees , 1997, IEEE Trans. Vis. Comput. Graph..

[27]  Gary A. Crocker.  Invisibility coherence for faster scan-line hidden surface algorithms , 1984, SIGGRAPH.

[28]  Edward H. Adelson,et al.  Layered representation for motion analysis , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Long Quan,et al.  Image-based rendering by joint view triangulation , 2003, IEEE Trans. Circuits Syst. Video Technol..

[30]  Ketan Mulmuley,et al.  An efficient algorithm for hidden surface removal , 1989, SIGGRAPH.

[31]  John M. Snyder,et al.  Visibility sorting and compositing without splitting for image layer decompositions , 1998, SIGGRAPH.

[32]  Chi-Keung Tang,et al.  A Computational Framework for Feature Extraction and Segmentation , 2000 .

[33]  John Amanatides,et al.  Merging BSP trees yields polyhedral set operations , 1990, SIGGRAPH 1990.

[34]  Michael McKenna.  Worst-case optimal hidden-surface removal , 1987, TOGS.

[35]  Richard Szeliski,et al.  A layered approach to stereo reconstruction , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[36]  Leonidas J. Guibas,et al.  Kinetic binary space partitions for intersecting segments and disjoint triangles , 1998, SODA '98.

[37]  Henry Fuchs,et al.  Near real-time shaded display of rigid objects , 1983, SIGGRAPH.

[38]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[39]  Olivier D. Faugeras,et al.  What can two images tell us about a third one? , 1994, ECCV.

[40]  John Edward Warnock,et al.  A hidden surface algorithm for computer generated halftone pictures , 1969 .

[41]  Henry Fuchs,et al.  On visible surface generation by a priori tree structures , 1980, SIGGRAPH '80.

[42]  Andrew S. Glassner,et al.  Space subdivision for fast ray tracing , 1984, IEEE Computer Graphics and Applications.

[43]  Frederick P. Brooks,et al.  Towards image realism with interactive update rates in complex virtual building environments , 1990, I3D '90.

[44]  John Amanatides,et al.  Merging BSP trees yields polyhedral set operations , 1990, SIGGRAPH.

[45]  Mi-Suen Lee,et al.  N-Dimensional Tensor Voting and Application to Epipolar Geometry Estimation , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Murray Hill.  Constructing Good Partitioning Trees , 1996 .

[47]  Bruce F. Naylor.  Partitioning tree image representation and generation from 3D geometric models , 1992 .

[48]  Carlo H. Séquin,et al.  Visibility preprocessing for interactive walkthroughs , 1991, SIGGRAPH.

[49]  W. Jack Bouknight,et al.  An algorithm for producing half-tone computer graphics presentations with shadows and movable light sources , 1899, AFIPS '70 (Spring).

[50]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[51]  Hayder Radha,et al.  Image compression using binary space partitioning trees , 1996, IEEE Trans. Image Process..

[52]  Mark de Berg,et al.  Perfect Binary Space Partitions , 1993, Comput. Geom..

[53]  Dinesh Manocha,et al.  Visibility culling using hierarchical occlusion maps , 1997, SIGGRAPH.

[54]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[55]  Mi-Suen Lee,et al.  Inferring segmented surface description from stereo data , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[56]  Hayder Radha,et al.  Multiresolution image compression with BSP trees and multilevel BTC , 1995, Proceedings., International Conference on Image Processing.

[57]  Steven K. Feiner,et al.  Fast object-precision shadow generation for area light sources using BSP trees , 1992, I3D '92.

[58]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[59]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[60]  Gernot Schaufler,et al.  Efficient displacement Mapping by Image Warping , 1999, Rendering Techniques.

[61]  F. Frances Yao,et al.  Efficient binary space partitions for hidden-surface removal and solid modeling , 1990, Discret. Comput. Geom..

[62]  Seth J. Teller,et al.  Global visibility algorithms for illumination computations , 1993, SIGGRAPH.

[63]  Gary S. Watkins,et al.  A real time visible surface algorithm , 1970 .

[64]  John Snyder,et al.  Rendering with coherent layers , 1997, SIGGRAPH.

[65]  Gérard G. Medioni,et al.  First order tensor voting, and application to 3-D scale analysis , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[66]  W. Jack Bouknight,et al.  A procedure for generation of three-dimensional half-toned computer graphics presentations , 1970, CACM.

[67]  Martin E. Newell,et al.  A solution to the hidden surface problem , 1972, ACM Annual Conference.