Efficient Video Stitching Based on Fast

In computer vision, video stitching is a very chal- lenging problem. In this paper, we proposed an efficient and effective wide-view video stitching method based on fast struc- ture deformation that is capable of simultaneously achieving quality stitching and computational efficiency. For a group of synchronized frames, firstly, an effective double-seam selection scheme is designed to search two distinct but structurally corre- sponding seams in the two original images. The seam location of the previous frame is further considered to preserve the inter- frame consistency. Secondly, along the double seams, 1-D feature detection and matching is performed to capture the structural relationship between the two adjacent views. Thirdly, after fea- ture matching, we propose an efficient algorithm to linearly propagate the deformation vectors to eliminate structure mis- alignment. At last, image intensity misalignment is corrected by rapid gradient fusion based on the successive over relaxation iteration (SORI) solver. A principled solution to the initializa- tion of the SORI significantly reduced the number of iterations required. We have compared favorably our method with seven state-of-the-art image and video stitching algorithms as well as traditional ones. Experimental results show that our method out- performs the existing ones compared in terms of overall stitching quality and computational efficiency.

[1]  Aljoscha Smolic,et al.  High-resolution video mosaicing , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[2]  Nam Ik Cho,et al.  Panorama Mosaic Optimization for Mobile Camera Systems , 2007, IEEE Transactions on Consumer Electronics.

[3]  Ling Shao,et al.  Weakly-Supervised Cross-Domain Dictionary Learning for Visual Recognition , 2014, International Journal of Computer Vision.

[4]  Dani Lischinski,et al.  Gradient Domain High Dynamic Range Compression , 2023 .

[5]  Yizhou Yu,et al.  Feature matching and deformation for texture synthesis , 2004, SIGGRAPH 2004.

[6]  Josechu J. Guerrero,et al.  Monocular 3-D Gait Tracking in Surveillance Scenes , 2014, IEEE Transactions on Cybernetics.

[7]  Luc Van Gool,et al.  Efficient Non-Maximum Suppression , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[8]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Valerio Pascucci,et al.  Panorama weaving , 2012, ACM Trans. Graph..

[10]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[11]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[12]  Sridha Sridharan,et al.  In the Pursuit of Effective Affective Computing: The Relationship Between Features and Registration , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Shmuel Peleg,et al.  Seamless Image Stitching in the Gradient Domain , 2004, ECCV.

[14]  Shmuel Peleg,et al.  Elimination of seams from photomosaics , 1981, Computer Graphics and Image Processing.

[15]  Jerry L. Prince,et al.  Image registration based on boundary mapping , 1996, IEEE Trans. Medical Imaging.

[16]  Aseem Agarwala Efficient gradient-domain compositing using quadtrees , 2007, SIGGRAPH 2007.

[17]  Wei Xu,et al.  Angle consistency for registration between catadioptric omni-images and orthorectified aerial images , 2013, IET Image Process..

[18]  Hefeng Wu,et al.  Hierarchical Ensemble of Background Models for PTZ-Based Video Surveillance , 2015, IEEE Transactions on Cybernetics.

[19]  Edward H. Adelson,et al.  PYRAMID METHODS IN IMAGE PROCESSING. , 1984 .

[20]  Jitendra Malik,et al.  Detecting and localizing edges composed of steps, peaks and roofs , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[21]  Yasuyuki Matsushita,et al.  Smoothly varying affine stitching , 2011, CVPR 2011.

[22]  Richard Szeliski,et al.  Eliminating ghosting and exposure artifacts in image mosaics , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[23]  Richard Szeliski,et al.  Efficiently registering video into panoramic mosaics , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[24]  John Hart,et al.  Textureshop: texture synthesis as a photograph editing tool , 2004, SIGGRAPH 2004.

[25]  Xiaolin Chen,et al.  Stitching Video from Webcams , 2008, ISVC.

[26]  Morten Bro-Nielsen,et al.  Fast Fluid Registration of Medical Images , 1996, VBC.

[27]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[28]  H. Barlow Vision: A computational investigation into the human representation and processing of visual information: David Marr. San Francisco: W. H. Freeman, 1982. pp. xvi + 397 , 1983 .

[29]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[30]  Charles Poynton,et al.  Digital Video and HDTV Algorithms and Interfaces , 2012 .

[31]  Yue Yu,et al.  A new optimal seam selection method for airborne image stitching , 2009, 2009 IEEE International Workshop on Imaging Systems and Techniques.

[32]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[33]  Harpreet S. Sawhney,et al.  Robust Video Mosaicing through Topology Inference and Local to Global Alignment , 1998, ECCV.

[34]  Jiun-Hung Chen,et al.  Panoramic appearance-based recognition of video contents using matching graphs , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[35]  Jeremy R. Cooperstock,et al.  Toward Dynamic Image Mosaic Generation With Robustness to Parallax , 2012, IEEE Transactions on Image Processing.

[36]  Eli Shechtman,et al.  Image melding , 2012, ACM Trans. Graph..

[37]  Shmuel Peleg,et al.  Seamless image stitching by minimizing false edges , 2006, IEEE Transactions on Image Processing.

[38]  Ayman Kaheel,et al.  Fast stitching of videos captured from freely moving devices by exploiting temporal redundancy , 2010, 2010 IEEE International Conference on Image Processing.

[39]  Frank M. Candocia Jointly registering images in domain and range by piecewise linear comparametric analysis , 2003, IEEE Trans. Image Process..

[40]  Chi-Keung Tang,et al.  Image Stitching Using Structure Deformation , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Ji Wu,et al.  Towards efficient registration of medical images , 2007, Comput. Medical Imaging Graph..

[42]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[43]  Andrew P. Bradley,et al.  Towards Optimal Image Stitching for Virtual Microscopy , 2005, Digital Image Computing: Techniques and Applications (DICTA'05).

[44]  Gregory Dudek,et al.  Image stitching with dynamic elements , 2009, Image Vis. Comput..

[45]  Ling Shao,et al.  Learning Object-to-Class Kernels for Scene Classification , 2014, IEEE Transactions on Image Processing.

[46]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[47]  Chi-Keung Tang,et al.  Eliminating structure and intensity misalignment in image stitching , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[48]  Ayman Kaheel,et al.  Improved optimal seam selection blending for fast video stitching of videos captured from freely moving devices , 2011, 2011 18th IEEE International Conference on Image Processing.