Frame Decimation for Structure and Motion

A frame decimation scheme is proposed that makes automatic extraction of Structure and Motion (SaM) from handheld sequences more practical. Decimation of the number of frames used for the actual SaM calculations keeps the size of the problem manageable, regardless of the input frame rate. The proposed preprocessor is based upon global motion estimation between frames and a sharpness measure. With these tools, shot boundary detection is first performed followed by the removal of redundant frames. The frame decimation makes it feasible to feed the system with a high frame rate, which in turn avoids loss of connectivity due to matching difficulties. A high input frame rate also enables robust automatic detection of shot boundaries. The development of the preprocessor was prompted by experience with a number of test sequences, acquired directly from a handheld camera. The preprocessor was tested on this material together with a SaM algorithm. The scheme is conceptually simple and still has clear benefits.

[1]  Long Quan,et al.  Relative 3D Reconstruction Using Multiple Uncalibrated Images , 1995, Int. J. Robotics Res..

[2]  Olivier D. Faugeras,et al.  Relative 3D positioning and 3D convex hull computation from a weakly calibrated stereo pair , 1995, Image Vis. Comput..

[3]  David W. Murray,et al.  A unifying framework for structure and motion recovery from image sequences , 1995, Proceedings of IEEE International Conference on Computer Vision.

[4]  Peter F. Sturm,et al.  A Factorization Based Algorithm for Multi-Image Projective Structure and Motion , 1996, ECCV.

[5]  Amnon Shashua,et al.  Trilinearity in Visual Recognition by Alignment , 1994, ECCV.

[6]  Anders Heyden,et al.  Euclidean reconstruction from image sequences with varying and unknown focal length and principal point , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Richard I. Hartley,et al.  Euclidean Reconstruction from Uncalibrated Views , 1993, Applications of Invariance in Computer Vision.

[8]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[9]  Naoya Ohta,et al.  Accuracy bounds and optimal computation of homography for image mosaicing applications , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[10]  Long Quan,et al.  Relative 3D Reconstruction Using Multiple Uncalibrated Images , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[12]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[13]  Pascal Fua,et al.  Reconstructing complex surfaces from multiple stereo views , 1995, Proceedings of IEEE International Conference on Computer Vision.

[14]  Andrew Zisserman,et al.  Applications of Invariance in Computer Vision , 1993, Lecture Notes in Computer Science.

[15]  Luc Van Gool,et al.  Automatic 3D model building from video sequences , 1997, Eur. Trans. Telecommun..

[16]  KanadeTakeo,et al.  Shape and motion from image streams under orthography , 1992 .

[17]  Wayne H. Wolf,et al.  Key frame selection by motion analysis , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[18]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[19]  Bill Triggs,et al.  Plane+Parallax, Tensors and Factorization , 2000, ECCV.

[20]  Keith J. Hanna,et al.  Combining stereo and motion analysis for direct estimation of scene structure , 1993, 1993 (4th) International Conference on Computer Vision.

[21]  Harpreet S. Sawhney,et al.  Robust Video Mosaicing through Topology Inference and Local to Global Alignment , 1998, ECCV.

[22]  David Nistér,et al.  Reconstruction from Uncalibrated Sequences with a Hierarchy of Trifocal Tensors , 2000, ECCV.

[23]  Roberto Cipolla,et al.  3D Model Acquisition from Uncalibrated Images , 1998, MVA.

[24]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[25]  P. Anandan,et al.  Direct Recovery of Planar-Parallax from Multiple Frames , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Andrew Zisserman,et al.  Automated mosaicing with super-resolution zoom , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[27]  William H. Press,et al.  Numerical recipes in C , 2002 .

[28]  Reinhard Koch,et al.  Realistic 3-D scene modeling from uncalibrated image sequences , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[29]  Shmuel Peleg,et al.  Panoramic mosaics by manifold projection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.