Is appearance-based structure from motion viable?

Most structure from motion work assumes that tracking is a separate issue from recovery of structure and motion. This paper argues that in certain cases, it helps to use the original images to refine the tracking in the course of estimating structure and motion. We illustrate this by presenting two algorithms: one for appearance-based structure from complete rotation and another for 3-D face modeling from three images. We come to the following conclusion: use appearance prediction only if the gross shape or type of shape deformation is known. Otherwise, one is very likely better off just using the currently known tracks.

[1]  Peter F. Sturm,et al.  Critical motion sequences for monocular self-calibration and uncalibrated Euclidean reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[3]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[4]  S. Ullman,et al.  The interpretation of visual motion , 1977 .

[5]  Thomas S. Huang,et al.  Uniqueness and Estimation of Three-Dimensional Motion Parameters of Rigid Objects with Curved Surfaces , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Anders Heyden,et al.  Reconstruction from image sequences by means of relative depths , 1995, Proceedings of IEEE International Conference on Computer Vision.

[7]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[8]  Hajime Terasaki,et al.  A portable three-dimensional digitizer , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[9]  Richard I. Hartley,et al.  Projective Reconstruction and Invariants from Multiple Images , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Richard Szeliski,et al.  Hierarchical spline-based image registration , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Andrew W. Fitzgibbon,et al.  Automatic 3D Model Construction for Turn-Table Sequences , 1998, SMILE.

[12]  Richard Szeliski,et al.  Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares , 1994, J. Vis. Commun. Image Represent..

[13]  Amnon Shashua,et al.  Projective Structure from Uncalibrated Images: Structure From Motion and Recognition , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[15]  Luc Van Gool,et al.  Euclidean 3D Reconstruction from Image Sequences with Variable Focal Lenghts , 1996, ECCV.

[16]  Shahriar Negahdaripour,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence , 2004 .

[17]  Reinhard Koch,et al.  Dynamic 3-D Scene Analysis Through Synthesis Feedback Control , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[20]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[21]  A. Heyden,et al.  Euclidean reconstruction from constant intrinsic parameters , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[22]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[23]  Richard Szeliski,et al.  Shape from rotation , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.