Real-time Stereo-Image Stitching using GPU-based Belief Propagation

Image stitching or mosaicing is a challenging vision problem, especially when considering aspects like high definition content, real-time, and proper compensation of the parallax error of objects at different distances to the camera system. Today many approaches to image stitching exist, most of them deal with medium resolution images, offline processing, or are restricted to objects at similar distance. Our approach is based on calculation of disparities between corresponding points in stereo images by employing standard belief propagation. Instead of computing depth-maps like in previous approaches, we compute stitch-maps modifying cost functions in the underlying Markov random field model, which makes further projection steps dispensable and thereby overall computation more efficient. Our GPU-based implementation is real-time capable, allowing to stitch high definition images at constant frame rates. We show exemplary results from our system to demonstrate the quality of the merged images.

[1]  Camillo J. Taylor,et al.  Graph Cuts via $\ell_1$ Norm Minimization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Xiaolin Chen,et al.  Stitching Video from Webcams , 2008, ISVC.

[3]  Ming Ouhyoung,et al.  VideoVR: A Real-Time System for Automatically Constructing Panoramic Images from Video Clips , 1998, CAPTECH.

[4]  S. Selvi,et al.  Registration and mosaicing for images obtained from UAV , 2004, 2004 International Conference on Signal Processing and Communications, 2004. SPCOM '04..

[5]  Yoichi Muraoka,et al.  Real-time image mosaicing from a video sequence , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[6]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Gerhard Lakemeyer,et al.  Exploring artificial intelligence in the new millennium , 2003 .

[8]  Alexander Ferworn,et al.  Panoramic Spherical Video - The Space Ball , 2003, ICCSA.

[9]  Gunilla Borgefors,et al.  Distance transformations in digital images , 1986, Comput. Vis. Graph. Image Process..

[10]  William T. Freeman,et al.  Understanding belief propagation and its generalizations , 2003 .

[11]  William T. Freeman,et al.  Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  Yair Weiss,et al.  Interpreting Images by Propagating Bayesian Beliefs , 1996, NIPS.

[13]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[14]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[15]  Naokazu Yokoya,et al.  High-resolution video mosaicing for documents and photos by estimating camera motion , 2004, IS&T/SPIE Electronic Imaging.

[16]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[17]  Alan Brunton,et al.  Belief Propagation on the GPU for Stereo Vision , 2006, The 3rd Canadian Conference on Computer and Robot Vision (CRV'06).

[18]  Alan Brunton,et al.  Belief Propagation for Panorama Generation , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[19]  Michael F. Cohen,et al.  Capturing and viewing gigapixel images , 2007, ACM Trans. Graph..

[20]  Edward H. Adelson,et al.  PYRAMID METHODS IN IMAGE PROCESSING. , 1984 .

[21]  Jeremy R. Cooperstock,et al.  Depth-based Image Mosaicing for Both Static and Dynamic Scenes , 2008 .

[22]  Daniel Cremers,et al.  A Convex Formulation of Continuous Multi-label Problems , 2008, ECCV.

[23]  C. Kambhamettu,et al.  GPU implementation of belief propagation using CUDA for cloud tracking and reconstruction , 2008, 2008 IAPR Workshop on Pattern Recognition in Remote Sensing (PRRS 2008).

[24]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[25]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[26]  Michael Leitner,et al.  A Platform for Audiovisual Telepresence Using Model- and Data-Based Wave-Field Synthesis , 2008 .

[27]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.