Image-Based Geometrically-Correct Photorealistic Scene/Object Modeling (IBPhM): A Review

There are emerging interests from both computer vision and computer graphics communities in obtaining photorealistic modeling of a scene or an object from real images. This paper presents a tentative review of the computer vision techniques used in such modeling which guarantee the generated views to be geometrically correct. The topics covered include mosaicking for building environment maps, CAD-like modeling for building 3D geometric models together with texture maps extracted from real images, image-based rendering for synthesizing new views from uncalibrated images, and techniques for modeling the appearance variation of a scene or an object under different illumination conditions. Major issues and difficulties are addressed.

[1]  Amnon Shashua,et al.  Algebraic Functions For Recognition , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Harry Shum,et al.  Construction and refinement of panoramic mosaics with global and local alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[3]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[4]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[5]  Quang-Tuan Luong.  Matrice Fondamentale et Calibration Visuelle sur l''''Environnement - Vers une plus grande autonomie , 1992 .

[6]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[7]  Hiroshi Murase,et al.  Dimensionality of illumination in appearance matching , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[8]  Zhengyou Zhang.  Motion and Structure From Two Perspective Views : From Essential Parameters to Euclidean Motion Via Fundamental Matrix , 1997 .

[9]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[10]  Olivier D. Faugeras,et al.  From projective to Euclidean reconstruction , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Thierry Viéville,et al.  Canonical Representations for the Geometries of Multiple Projective Views , 1996, Comput. Vis. Image Underst..

[12]  Stéphane Laveau.  Géométrie d'un système de N caméras : théorie, estimation et applications. (Geometry of a System of n Cameras. Theory. Estimation. Applications) , 1996 .

[13]  Sing Bing Kang,et al.  Survey of image-based rendering techniques , 1998, Electronic Imaging.

[14]  Yasushi Yagi,et al.  Omnidirectional imaging with hyperboloidal projection , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[15]  A. Shashua.  Geometry and Photometry in 3D Visual Recognition , 1992 .

[16]  Yasushi Yagi,et al.  Panorama scene analysis with conic projection , 1990, EEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications.

[17]  Han Wang,et al.  Recent Developments in Computer Vision , 1995, Lecture Notes in Computer Science.

[18]  Václav Hlavác,et al.  Rendering real-world objects using view interpolation , 1995, Proceedings of IEEE International Conference on Computer Vision.

[19]  Zhengyou Zhang,et al.  Motion of a Stereo Rig: Strong Weak and Self Calibration , 1995, ACCV.

[20]  Roger Mohr,et al.  Understanding Positioning from Multiple Images , 1995, Artif. Intell..

[21]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[22]  Rachid Deriche,et al.  Using geometric corners to build a 2D mosaic from a set of images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Andrew Zisserman,et al.  Robust parameterization and computation of the trifocal tensor , 1997, Image Vis. Comput..

[24]  Thomas S. Huang,et al.  Motion and structure from feature correspondences: a review , 1994, Proc. IEEE.

[25]  Andrea J. van Doorn,et al.  Bidirectional Reflection Distribution Function Expressed in Terms of Surface Scattering Modes , 1996, ECCV.

[26]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Shree K. Nayar,et al.  Catadioptric omnidirectional camera , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Richard I. Hartley,et al.  An algorithm for self calibration from several views , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Zhengyou Zhang.  Motion and structure from two perspective views: from essential parameters to Euclidean motion through the fundamental matrix , 1997 .

[30]  Lawrence B. Wolff,et al.  Generalizing Lambert's Law for Smooth Surfaces , 1996, ECCV.

[31]  I. Reid,et al.  Metric calibration of a stereo rig , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[32]  Olivier Faugeras,et al.  Motion of an uncalibrated stereo rig: self-calibration and metric reconstruction , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[33]  Yuri M. Bayakovsky,et al.  Russia: computer graphics education takes off in the 1990s , 1996, COMG.

[34]  Gang Xu,et al.  Epipolar Geometry in Stereo, Motion and Object Recognition , 1996, Computational Imaging and Vision.

[35]  Olivier Faugeras,et al.  3D Dynamic Scene Analysis: A Stereo Based Approach , 1992 .

[36]  Richard I. Hartley,et al.  Projective Reconstruction and Invariants from Multiple Images , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Stéphane Laveau.  Geometrie d'un systeme a n cameras. Theorie. Estimation. Applications , 1996 .

[38]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[39]  Shmuel Peleg,et al.  Panoramic mosaics by manifold projection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Olivier Faugeras,et al.  3D Dynamic Scene Analysis , 1992 .

[41]  Zhengyou Zhang,et al.  Euclidean structure from uncalibrated images using fuzzy domain knowledge: application to facial images synthesis , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[42]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[43]  Zhengyou Zhang,et al.  Modeling geometric structure and illumination variation of a scene from real images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[44]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[45]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[46]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[47]  Russell A. Epstein,et al.  5/spl plusmn/2 eigenimages suffice: an empirical investigation of low-dimensional lighting models , 1995, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[48]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[49]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[50]  Gregory D. Hager,et al.  Real-time tracking of image regions with changes in geometry and illumination , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[51]  Jake K. Aggarwal,et al.  On the computation of motion from sequences of images-A review , 1988, Proc. IEEE.

[52]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[53]  S. P. Mudur,et al.  Three-dimensional computer vision: a geometric viewpoint , 1993 .