Vision field capture for advanced 3DTV applications

The seven-dimensional plenoptic function provides a full description of the visual information for the real world. In this paper, we present a novel concept called vision field, which simplifies the seven-dimensional plenoptic function into its three subspaces, namely, view, light, time. Based on this concept, we found that most previous 3D capture systems can be related to the vision field capture. This paper first gives a brief survey on the previous 3D capture systems, categorizes them from the vision field perspective. Then, we introduce a system which is able to capture the vision field. A Multi-View-Multi-Lighting (MVML) capture system is built to obtain the multiview images of the 3D scenes or objects under different steerable light conditions. Finally, we show how the vision field capture can be used for advanced 3DTV applications.

[1]  H. Opower Multiple view geometry in computer vision , 2002 .

[2]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, ACM Trans. Graph..

[3]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[4]  Li Zhang,et al.  Shape and motion under varying illumination: unifying structure from motion, photometric stereo, and multiview stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[5]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[6]  Qionghai Dai,et al.  Vision field capturing and its applications in 3DTV , 2010, 28th Picture Coding Symposium.

[7]  Toshiaki Fujii,et al.  Free-Viewpoint TV , 2011, IEEE Signal Processing Magazine.

[8]  Paul A. Beardsley,et al.  Image-based 3D photography using opacity hulls , 2002, ACM Trans. Graph..

[9]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[10]  Toshiaki Fujii,et al.  Ray space coding for 3D visual communication , 1996 .

[11]  Qionghai Dai,et al.  Dynamic Shape Capture via Periodical-Illumination Optical Flow Estimation and Multi-view Photometric Stereo , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[12]  P. Debevec,et al.  Image-based modeling, rendering, and lighting , 2002, IEEE Computer Graphics and Applications.

[13]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[14]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, SIGGRAPH 2005.

[15]  Qionghai Dai,et al.  Fusing Multiview and Photometric Stereo for 3D Reconstruction under Uncalibrated Illumination , 2011, IEEE Transactions on Visualization and Computer Graphics.

[16]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.

[17]  Andrew Jones,et al.  Relighting human locomotion with flowed reflectance fields , 2006, EGSR '06.

[18]  DaiQionghai,et al.  A Point-Cloud-Based Multiview Stereo Algorithm for Free-Viewpoint Video , 2010 .

[19]  Qionghai Dai,et al.  Continuous depth estimation for multi-view stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Masayuki Tanimoto Overview of FTV (free-viewpoint television) , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[21]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[22]  Jiang Li,et al.  A real-time interactive multi-view video system , 2005, MULTIMEDIA '05.