3-D Cinematography with Approximate or No Geometry

3-D cinematography is a new step towards full immersive video, allowing complete control of the viewpoint during playback both in space and time.One major challenge towards this goal is precise scene reconstruction, either implicit or explicit.While some approaches exist which are able to generate a convincing geometry proxy, they are bound to many constraints, e.g., accurate camera calibration and synchronized cameras.This chapter is about methods to remedy some of these constraints.

[1]  Hans-Peter Seidel,et al.  Image-based reconstruction of spatial appearance and geometric detail , 2003, TOGS.

[2]  Robert T. Collins,et al.  A space-sweep approach to true multi-image matching , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Carlo Tomasi,et al.  Color edge detection with the compass operator , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[4]  Michael S. Landy,et al.  Computational models of visual processing , 1991 .

[5]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[6]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[7]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[8]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[9]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[10]  Hans Wallach.  Über visuell wahrgenommene Bewegungsrichtung , 1935 .

[11]  W. Reichardt,et al.  Autocorrelation, a principle for the evaluation of sensory information by the central nervous system , 1961 .

[12]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[13]  Maneesh Agrawala,et al.  Using Photographs to Enhance Videos of a Static Scene , 2007, Rendering Techniques.

[14]  M. Wertheimer.  Laws of organization in perceptual forms. , 1938 .

[15]  Xiaojun Wu,et al.  Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video , 2004, Comput. Vis. Image Underst..

[16]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Marcus A. Magnor,et al.  View and Time Interpolation in Image Space , 2008, Comput. Graph. Forum.

[18]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[19]  Hans-Peter Seidel,et al.  Spatio-Temporal Registration Techniques for Relightable 3D Video , 2007, 2007 IEEE International Conference on Image Processing.

[20]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[21]  Horst Bischof,et al.  A Duality Based Algorithm for TV- L 1-Optical-Flow Image Registration , 2007, MICCAI.

[22]  M. Wertheimer,et al.  A source book of Gestalt psychology. , 1939 .

[23]  Anita Sellent,et al.  Floating Textures , 2008, Comput. Graph. Forum.

[24]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[25]  M SeitzSteven,et al.  A Theory of Shape by Space Carving , 2000 .

[26]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[27]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[29]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[30]  Martin Jägersand,et al.  Dynamic Textures for Image‐based Rendering of Fine‐Scale 3D Structure and Animation of Non‐rigid Motion , 2002, Comput. Graph. Forum.

[31]  Hans-Peter Seidel,et al.  Rapid Animation of Laser-scanned Humans , 2007, 2007 IEEE Virtual Reality Conference.

[32]  Jeffrey L. Posdamer,et al.  Surface measurement by space-encoded projected beam systems , 1982, Comput. Graph. Image Process..

[33]  Alan L. Yuille,et al.  Scaling Theorems for Zero Crossings , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[35]  N. Qian,et al.  A Physiological Model for Motion-Stereo Integration and a Unified Explanation of Pulfrich-like Phenomena , 1997, Vision Research.

[36]  Marcus A. Magnor,et al.  Perception-motivated interpolation of image sequences , 2008, APGV.

[37]  Ruigang Yang,et al.  Multi-resolution real-time stereo on commodity graphics hardware , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[38]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[39]  Hans-Peter Seidel,et al.  Marker-less Deformable Mesh Tracking for Human Shape and Motion Capture , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Leonard McMillan,et al.  A new reconstruction filter for undersampled light fields , 2003 .

[41]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[42]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[43]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH 2005.

[44]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Marcus A. Magnor,et al.  Filtered Blending: A new, minimal Reconstruction Filter for Ghosting-Free Projective Texturing with Multiple Images , 2007, VMV.

[46]  Takeshi Naemura,et al.  Real-Time Video-Based Modeling and Rendering of 3D Scenes , 2002, IEEE Computer Graphics and Applications.

[47]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[48]  Mark Segal,et al.  Fast shadows and lighting effects using texture mapping , 1992, SIGGRAPH.

[49]  Adrian Hilton,et al.  Surface Capture for Performance-Based Animation , 2007, IEEE Computer Graphics and Applications.

[50]  Steven J. Gortler,et al.  Geometry images , 2002, SIGGRAPH.

[51]  R. Y. Tsai,et al.  An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision , 1986, CVPR 1986.

[52]  D. Heeger,et al.  Motion Opponency in Visual Cortex , 1999, The Journal of Neuroscience.

[53]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[54]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[55]  Edilson de Aguiar,et al.  Reconstructing Human Shape and Motion from Multi-View Video , 2006 .

[56]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[57]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.

[58]  Markus H. Gross,et al.  3D Video Billboard Clouds , 2007, Comput. Graph. Forum.

[59]  Wojciech Matusik,et al.  3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes , 2004, ACM Trans. Graph..

[60]  Edmond Boyer,et al.  Exact polyhedral visual hulls , 2003, BMVC.

[61]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, ACM Trans. Graph..

[62]  Jan-Michael Frahm,et al.  Real-Time Plane-Sweeping Stereo with Multiple Sweeping Directions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[64]  Douglas Lanman,et al.  Surround Structured Lighting for Full Object Scanning , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[65]  Toshiaki Fujii,et al.  Free viewpoint TV system based on ray-space representation , 2002, SPIE ITCom.

[66]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[67]  Hans-Peter Seidel,et al.  Hardware‐Accelerated Rendering of Photo Hulls , 2004, Comput. Graph. Forum.

[68]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.