Efficient key frames selection for panorama generation from video

A video sequence consists of several hundred frames, and as a result, creating a panoramic image from these frames is a very time-consuming process. Consecutive frames have large overlap areas that do not provide much information. Therefore, some key frames must be extracted for better performance. There are a number of methods for key-frame selection that match all frames in a video sequence. We present a novel and efficient method to select key frames from video for creating a large panoramic mosaic without matching all frames. Consecutive frames are transformed and projected onto the common mosaic surface and the position of each corner of the next frame is predicted with a distinct Kalman filter on this surface. The overlap area between each predicted frame and its previous key frame is used as the criterion to select the next key frame. This method uses video information to reduce features and align frames with repeated structures more accurately. We show that this approach is an efficient preprocessing step and substantially reduces the time required to construct panorama from a video sequence.

[1]  Shmuel Peleg,et al.  Seamless Image Stitching in the Gradient Domain , 2004, ECCV.

[2]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[3]  Jun-Wei Hsieh,et al.  Fast stitching algorithm for moving object detection and mosaic construction , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[4]  R. Singer.  Estimating Optimal Tracking Filter Performance for Manned Maneuvering Targets , 1970, IEEE Transactions on Aerospace and Electronic Systems.

[5]  Sung-Jea Ko,et al.  Robust digital image stabilization using the Kalman filter , 2009, IEEE Transactions on Consumer Electronics.

[6]  Denis Simakov,et al.  Space-time scene manifolds , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[7]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Mahmood Fathy,et al.  Key frames selection into panoramic mosaics , 2009, 2009 7th International Conference on Information, Communications and Signal Processing (ICICS).

[9]  Richard Szeliski,et al.  Systems and Experiment Paper: Construction of Panoramic Image Mosaics with Global and Local Alignment , 2000, International Journal of Computer Vision.

[10]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[11]  Richard Szeliski,et al.  Video mosaics for virtual environments , 1996, IEEE Computer Graphics and Applications.

[12]  Yong-In Yoon,et al.  An efficient method to build panoramic image mosaics , 2003, Pattern Recognit. Lett..

[13]  Shmuel Peleg,et al.  Panoramic mosaics by manifold projection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Zhigang Zhu,et al.  Fast Construction of Dynamic and Multi-Resolution 360 ° Panoramas from Video Sequences ( Revised-September 14 , 2005 ) , 2005 .

[15]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[16]  J. Santos-Victor,et al.  Underwater mosaicing and trajectory reconstruction using global alignment , 2001, MTS/IEEE Oceans 2001. An Ocean Odyssey. Conference Proceedings (IEEE Cat. No.01CH37295).

[17]  Mohammad H. Mahoor,et al.  Fast image blending using watersheds and graph cuts , 2009, Image Vis. Comput..

[18]  Javier Civera,et al.  Drift-Free Real-Time Sequential Mosaicing , 2009, International Journal of Computer Vision.

[19]  Andrea Fusiello,et al.  High resolution video mosaicing with global alignment , 2004, CVPR 2004.

[20]  Richard Szeliski,et al.  Construction of Panoramic Image Mosaics with Global and Local Alignment , 2001 .

[21]  Allen R. Hanson,et al.  Fast construction of dynamic and multi-resolution 360 degrees panoramas from video sequences , 2006, Image Vis. Comput..

[22]  Greg Welch,et al.  Welch & Bishop , An Introduction to the Kalman Filter 2 1 The Discrete Kalman Filter In 1960 , 1994 .

[23]  David Salesin,et al.  Panoramic video textures , 2005, ACM Trans. Graph..

[24]  Richard Szeliski,et al.  Efficiently registering video into panoramic mosaics , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[25]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[26]  Harpreet S. Sawhney,et al.  Robust Video Mosaicing through Topology Inference and Local to Global Alignment , 1998, ECCV.

[27]  Ki-Sang Hong,et al.  Real-time mosaic using sequential graph , 2006, J. Electronic Imaging.

[28]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[29]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[30]  Tao Mei,et al.  Efficient video mosaicing based on motion analysis , 2005, IEEE International Conference on Image Processing 2005.

[31]  Shmuel Peleg,et al.  Mosaicing on Adaptive Manifolds , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  R. Chellappa,et al.  Fast 3D stabilization and mosaic construction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  X. R. Li,et al.  Survey of maneuvering target tracking. Part I. Dynamic models , 2003 .

[34]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[35]  Philip F. McLauchlan,et al.  Image mosaicing using sequential bundle adjustment , 2002, Image Vis. Comput..