Feedback stitching for gigapixel video

Abstract. Methods of stitching static panoramas are unsuitable for video-rate stitching from camera arrays, because these methods are too computationally intensive for real-time operation and do not take advantage of prior knowledge of camera positions or the coherence between successive frames of a video sequence. We propose feedback stitching, which embeds the stitching process in a feedback loop, so that as new frames are captured, any new stitching errors occurring in the video sequence are analyzed and corrected as the sequence progresses. These algorithms are suitable for multiscale cameras, a camera array technology proven to be capable of gigapixel snapshot and video imaging, to allow for real-time compensation of any registration or parallax errors.

[1]  Fan Zhang,et al.  Parallax-Tolerant Image Stitching , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  A. E. Conrady.  The Five Aberrations of Lens-Systems. (Second Paper.) , 1919 .

[3]  Richard Szeliski,et al.  Efficiently registering video into panoramic mosaics , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[4]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[5]  Duane C. Brown,et al.  Close-Range Camera Calibration , 1971 .

[6]  Jun-Wei Hsieh,et al.  Fast stitching algorithm for moving object detection and mosaic construction , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[7]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[8]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[9]  David J. Brady,et al.  Engineering a gigapixel monocentric multiscale camera , 2012 .

[10]  John A. Antoniades,et al.  Autonomous real-time ground ubiquitous surveillance-imaging system (ARGUS-IS) , 2008, SPIE Defense + Commercial Sensing.

[11]  Chi-Keung Tang,et al.  Image Stitching Using Structure Deformation , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Jeremy R. Cooperstock,et al.  Toward Dynamic Image Mosaic Generation With Robustness to Parallax , 2012, IEEE Transactions on Image Processing.

[13]  Shree K. Nayar,et al.  Gigapixel Computational Imaging , 2011, 2011 IEEE International Conference on Computational Photography (ICCP).

[14]  A. E. Conrady.  The Five Aberrations of Lens-Systems , 1918 .

[15]  Yasuyuki Matsushita,et al.  Smoothly varying affine stitching , 2011, CVPR 2011.

[16]  Pengfei Xu,et al.  Wide baseline image mosaicing by integrating MSER and Hessian-Affine , 2011, 2011 4th International Congress on Image and Signal Processing.

[17]  Michael F. Cohen,et al.  Capturing and viewing gigapixel images , 2007, ACM Trans. Graph..

[18]  Cordelia Schmid,et al.  3D Object Modeling and Recognition Using Local Affine-Invariant Image Descriptors and Multi-View Spatial Constraints , 2006, International Journal of Computer Vision.

[19]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[20]  Dean Brown,et al.  Decentering distortion of lenses , 1966 .

[21]  Xiaoqiang Wang,et al.  An effective video stitching method , 2010, 2010 International Conference On Computer Design and Applications.

[22]  B. S. Manjunath,et al.  Panoramic video capturing and compressed domain virtual camera control , 2001, MULTIMEDIA '01.

[23]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[24]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[25]  Jason P. de Villiers,et al.  Real-time photogrammetric stitching of high resolution video on COTS hardware , 2009 .

[26]  Shaoen Wu,et al.  Real-time video surveillance for large scenes , 2011, 2011 International Conference on Wireless Communications and Signal Processing (WCSP).

[27]  Nathan Hagen,et al.  Multiscale lens design. , 2009, Optics express.

[28]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[29]  Daniel L Marks,et al.  Design and scaling of monocentric multiscale imagers. , 2012, Applied optics.

[30]  David J. Brady,et al.  Multiscale gigapixel photography , 2012, Nature.

[31]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..