VOIDGA: A View-Approximation Oriented Image Database Generation Approach

Figure 1.Comparison between the generated views (left and right) and the ground truth (middle) for the asteroid impact dataset yA31 at cycle time 29945. The orange and blue surfaces are contours of the temperature (0:2 eV) and water density field (0:002 g/cm3), respectively. The views have been approximated using only 24 depth images with a resolution of 5122 pixels, whereas the original triangulation has around 3 million triangles. Our approach first computes the position of each depth image pixel in world coordinates, and then renders the resulting geometry either via a point cloud (left) or a surface triangulation (right). The used depth images have been chosen with VOIDGA to bound the maximum approximation error for the current view.In this work, we propose a novel view-approximation oriented image database generation approach (VOIDGA) that enables the adequate generation of arbitrary views. Our approach utilizes Depth Image Based Rendering (DIBR) techniques to derive novel views based on a set of depth images. In contrast to approaches that store a huge amount of images to cover a wide range of possible view directions, VOIDGA identifies and stores only those images that significantly contribute to the overall view-approximation quality while bounding the resulting approximation error. This further reduces the size of image databases and the number of images that need to be processed by DIBR algorithms. We demonstrate VOIDGA on several challenging real-world examples, and compare our approximations against ground truth renderings using two image-based metrics.

[1]  B. S. Manjunath,et al.  Improving the quality of depth image based rendering for 3D Video systems , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[2]  Peter H. N. de With,et al.  Free-viewpoint depth image based rendering , 2010, J. Vis. Commun. Image Represent..

[3]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[4]  Voicu Popescu,et al.  Animated Depth Images for Interactive Remote Visualization of Time-Varying Data Sets , 2014, IEEE Transactions on Visualization and Computer Graphics.

[5]  Chunhua Shen,et al.  Depth and surface normal estimation from monocular images using regression on deep features and hierarchical CRFs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Matthias Zwicker,et al.  Surface splatting , 2001, SIGGRAPH.

[7]  Gunther H. Weber,et al.  Nested Tracking Graphs , 2017, Comput. Graph. Forum.

[8]  Marcus A. Magnor,et al.  Perception-motivated interpolation of image sequences , 2008, APGV.

[9]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[10]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting , 2010 .

[11]  David H. Rogers,et al.  Visualization and Analysis of Threats from Asteroid Ocean Impacts , 2016 .

[12]  Waleed H. Abdulla,et al.  A new error metric for geometric shape distortion using depth values from orthographic projections , 2012, IVCNZ '12.

[13]  Aljoscha Smolic,et al.  Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems , 2008, 2008 15th IEEE International Conference on Image Processing.

[14]  Harry Shum,et al.  Review of image-based rendering techniques , 2000, Visual Communications and Image Processing.

[15]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[16]  Dirk A. Lorenz,et al.  Image Sequence Interpolation Based on Optical Flow, Segmentation, and Optimal Control , 2012, IEEE Transactions on Image Processing.

[17]  Ian D. Reid,et al.  Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Oscar C. Au,et al.  An overview of free viewpoint depth-image-based rendering (DIBR) , 2010 .

[19]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[20]  Bernd Hamann,et al.  A Query-Based Framework for Searching, Sorting, and Exploring Data Ensembles , 2020, Computing in Science & Engineering.

[21]  James P. Ahrens,et al.  An Image-Based Approach to Extreme Scale in Situ Visualization and Analysis , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[22]  Horst Bischof,et al.  Optical Flow Guided TV-L1 Video Interpolation and Restoration , 2011, EMMCVPR.

[24]  Christian Lipski Virtual video camera: a system for free viewpoint video of arbitrary dynamic scenes , 2013 .

[25]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[26]  Michael W. Marcellin,et al.  View Compensated Compression of Volume Rendered Images for Remote Visualization , 2009, IEEE Transactions on Image Processing.

[27]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[28]  M. Levoy,et al.  Fast volume rendering using a shear-warp factorization of the viewing transformation , 1994, SIGGRAPH.

[29]  Marc Pollefeys,et al.  Direction matters: Depth estimation with a surface normal classifier , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Jens Ogniewski,et al.  High-Quality Real-Time Depth-Image-Based-Rendering , 2017, SIGRAD.

[31]  Manuel Menezes de Oliveira Neto Image-Based Modeling and Rendering Techniques: A Survey , 2002, RITA.

[32]  Murat Yakar,et al.  Importance of digital close-range photogrammetry in documentation of cultural heritage , 2007 .

[33]  James P. Ahrens,et al.  ADR visualization: A generalized framework for ranking large-scale scientific data using Analysis-Driven Refinement , 2014, 2014 IEEE 4th Symposium on Large Data Analysis and Visualization (LDAV).

[34]  Mirella M. Moro,et al.  Modelo Temporal de Versões , 2002, RITA.

[35]  B. Hamann,et al.  Viscous Fingering: A Topological Visual Analytic Approach , 2017 .

[36]  Daniel Cremers,et al.  Spatial and Temporal Interpolation of Multi-view Image Sequences , 2014, GCPR.

[37]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[38]  Jason Lee,et al.  Using High-Speed WANs and Network Data Caches to Enable Remote and Distributed Visualization , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[39]  Andrew Gardner,et al.  Capturing reality for computer graphics applications , 2015, SIGGRAPH Asia Courses.

[40]  Jun Li,et al.  A Two-Streamed Network for Estimating Fine-Scaled Depth Maps from Single RGB Images , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[41]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[42]  Thomas Ertl,et al.  Space-time volumetric depth images for in-situ visualization , 2014, 2014 IEEE 4th Symposium on Large Data Analysis and Visualization (LDAV).

[43]  Aljoscha Smolic,et al.  View Synthesis for Advanced 3D Video Systems , 2008, EURASIP J. Image Video Process..

[44]  Hui Xu,et al.  Stylized rendering of 3D scanned real world environments , 2004, NPAR '04.