Stereoscopic Image Stitching via Disparity-Constrained Warping and Blending

As a significant branch of virtual reality, stereoscopic image stitching aims to generating wide perspectives and natural-looking scenes. Existing 2D image stitching methods cannot be successfully applied to the stereoscopic images without considering the disparity consistency of stereoscopic images. To address this issue, this paper presents a stereoscopic image stitching method based on disparity-constrained warping and blending, which could avoid visual distortion and preserve disparity consistency. First, a point-line-driven homography based disparity minimization method is designed to pre-align the left and right images and reduce vertical disparity. Afterwards, a multi-constraint warping is proposed to further align the left and right images, where the initial disparity map is introduced to control the consistency of disparities. Finally, a disparity consistency seam-cutting and blending method is presented to determine the optimal seam and conduct stereoscopic image stitching. Experimental results demonstrate that the proposed method achieves competitive performance compared with other state-of-the-art methods.

[1]  Fan Zhang,et al.  Parallax-Tolerant Image Stitching , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[3]  Tianzhu Xiang,et al.  Locally warping-based image stitching by imposing line constraints , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[4]  Jianjun Lei,et al.  Depth Sensation Enhancement for Multiple Virtual View Rendering , 2015, IEEE Transactions on Multimedia.

[5]  Jing Li,et al.  Parallax-Tolerant Image Stitching Based on Robust Elastic Warping , 2018, IEEE Transactions on Multimedia.

[6]  Sharath Pankanti,et al.  Adaptive as-natural-as-possible image stitching , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Michael S. Brown,et al.  As-Projective-As-Possible Image Stitching with Moving DLT , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Jianjun Lei,et al.  Visual Attention Prediction for Stereoscopic Video by Multi-Module Fully Convolutional Network , 2019, IEEE Transactions on Image Processing.

[9]  Jian Sun,et al.  Dual-Feature Warping-Based Motion Model Estimation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[10]  Hong Qiao,et al.  Stitching contaminated images , 2016, Neurocomputing.

[11]  Yael Pritch,et al.  Megastereo: Constructing High-Resolution Stereo Panoramas , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Bin Xu,et al.  Wide-angle image stitching using multi-homography warping , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[13]  Wei Jiang,et al.  Video stitching with spatial-temporal content-preserving warping , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Fan Zhang,et al.  Casual stereoscopic panorama stitching , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Yifang Xu,et al.  Quasi-Homography Warps in Image Stitching , 2017, IEEE Transactions on Multimedia.

[16]  Yael Pritch,et al.  Omnistereo: Panoramic Stereo Imaging , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Richard Szeliski,et al.  Seamless Image Stitching of Scenes with Large Motions and Exposure Differences , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[18]  Yao Wang,et al.  Color-Guided Depth Recovery From RGB-D Data Using an Adaptive Autoregressive Model , 2014, IEEE Transactions on Image Processing.

[19]  Nan Li,et al.  Perception-based energy functions in seam-cutting , 2017, ArXiv.

[20]  Hao Tang,et al.  Content-Based 3-D Mosaics for Representing Videos of Dynamic Urban Scenes , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Dong-Qing Zhang,et al.  Multi-objective content preserving warping for image stitching , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[22]  Yasuyuki Matsushita,et al.  Smoothly varying affine stitching , 2011, CVPR 2011.

[23]  Fabio Bellavia,et al.  Dissecting and Reassembling Color Correction Algorithms for Image Stitching , 2018, IEEE Transactions on Image Processing.

[24]  Yu-Sheng Chen,et al.  Natural Image Stitching with the Global Similarity Prior , 2016, ECCV.

[25]  Eric Dubois,et al.  Stereoscopic cameras for the real-time acquisition of panoramic 3D images and videos , 2013, Electronic Imaging.

[26]  Kyungdon Joo,et al.  Line meets as-projective-as-possible image stitching with moving DLT , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[27]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[28]  Michael Gleicher,et al.  Content-preserving warps for 3D video stabilization , 2009, ACM Trans. Graph..

[29]  Xiaowu Chen,et al.  Copy and Paste: Temporally Consistent Stereoscopic Video Blending , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Kun Zhou,et al.  2D shape deformation using nonlinear least squares optimization , 2006, The Visual Computer.

[31]  Shiguang Liu,et al.  Shape-optimizing hybrid warping for image stitching , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[32]  Jianjun Lei,et al.  Stereoscopic Image Stitching Based on a Hybrid Warping Model , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Robert J. Woodham,et al.  Combining Line and Point Correspondences for Homography Estimation , 2008, ISVC.

[34]  Weisi Lin,et al.  Saliency Detection in the Compressed Domain for Adaptive Image Retargeting , 2012, IEEE Transactions on Image Processing.

[35]  Raj Rao Nadakuditi,et al.  Robust photometric stereo using learned image and gradient dictionaries , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[36]  Antoni Buades,et al.  Reliable Multiscale and Multiwindow Stereo Matching , 2015, SIAM J. Imaging Sci..

[37]  Jianjun Lei,et al.  Fast Mode Decision Based on Grayscale Similarity and Inter-View Correlation for Depth Map Coding in 3D-HEVC , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[38]  Stefan Roth,et al.  Registering Images to Untextured Geometry Using Average Shading Gradients , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[39]  Xuelong Li,et al.  Shape-Preserving Object Depth Control for Stereoscopic Images , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[40]  Michael S. Brown,et al.  Constructing image panoramas using dual-homography warping , 2011, CVPR 2011.

[41]  Jean-Philippe Pons,et al.  Seamless image-based texture atlases using multi-band blending , 2008, 2008 19th International Conference on Pattern Recognition.

[42]  Matthew A. Brown,et al.  Recognising panoramas , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[43]  Yoichi Sato,et al.  Shape-Preserving Half-Projective Warps for Image Stitching , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  智一 吉田,et al.  Efficient Graph-Based Image Segmentationを用いた圃場図自動作成手法の検討 , 2014 .

[45]  Kun Li,et al.  Graph-Based Segmentation for RGB-D Data Using 3-D Geometry Enhanced Superpixels , 2015, IEEE Transactions on Cybernetics.

[46]  Ju Liu,et al.  A Novel Distortion Model and Lagrangian Multiplier for Depth Maps Coding , 2014, IEEE Transactions on Circuits and Systems for Video Technology.