A retargeting method for stereoscopic 3D video

We propose a disparity-constrained retargeting method for stereoscopic 3D video, which simultaneously resizes a binocular video to a new aspect ratio and remaps the depth to the perceptual comfort zone. First, we model distortion energies to prevent important video contents from deforming. Then, to maintain depth mapping stability, we model disparity variation energies to constraint the disparity range both in spatial and temporal domains. The last component of our method is a non-uniform, pixel-wise warp to the target resolution based on these energy models. Using this method, we can process the original stereoscopic video to generate new, high-perceptual-quality versions at different display resolutions. For evaluation, we conduct a user study; we also discuss the performance of our method.

[1]  ITU-T Rec. P.910 (04/2008) Subjective video quality assessment methods for multimedia applications , 2009 .

[2]  Shuen-Huei Guan,et al.  Stereoscopic 3D experience optimization using cropping and warping , 2011, SA '11.

[3]  Mtm Marc Lambooij,et al.  Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review , 2009 .

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  Ralph R. Martin,et al.  A response time model for abrupt changes in binocular disparity , 2014, The Visual Computer.

[6]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[7]  Xing Xie,et al.  A visual attention model for adapting images on small displays , 2003, Multimedia Systems.

[8]  Wu-chi Feng,et al.  Enabling warping on stereoscopic images , 2012, ACM Trans. Graph..

[9]  M. Gross,et al.  Nonlinear disparity mapping for stereoscopic 3D , 2010, ACM Trans. Graph..

[10]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[12]  Christel Chamaret,et al.  Video retargeting for stereoscopic content under 3D viewing constraints , 2012, Electronic Imaging.

[13]  Chinchen Chang,et al.  Cooperative Resizing Technique for Stereo Image Pairs , 2022 .

[14]  Yung-Yu Chuang,et al.  Content-Aware Display Adaptation and Interactive Editing for Stereoscopic Images , 2011, IEEE Transactions on Multimedia.

[15]  Yung-Yu Chuang,et al.  3D cinematography principles and their applications to stereoscopic media processing , 2011, ACM Multimedia.

[16]  Rynson W. H. Lau,et al.  Depth Mapping for Stereoscopic Videos , 2013, International Journal of Computer Vision.

[17]  Ralph R. Martin,et al.  Changing Perspective in Stereoscopic Images , 2013, IEEE Transactions on Visualization and Computer Graphics.

[18]  Ariel Shamir,et al.  Improved seam carving for video retargeting , 2008, ACM Trans. Graph..

[19]  Michael F. Cohen,et al.  A Viewer-Centric Editor for 3D Movies , 2011, IEEE Computer Graphics and Applications.

[20]  Markus H. Gross,et al.  A system for retargeting of streaming video , 2009, ACM Trans. Graph..

[21]  Daniel Cohen-Or,et al.  Feature-aware texturing , 2006, EGSR '06.

[22]  Olga Sorkine-Hornung,et al.  Visual media retargeting , 2009, SIGGRAPH ASIA Courses.

[23]  Shai Avidan,et al.  Stereo Seam Carving a Geometrically Consistent Approach , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Hans-Peter Seidel,et al.  Modeling and optimizing eye vergence response to stereoscopic cuts , 2014, ACM Trans. Graph..

[25]  Daniel Cohen-Or,et al.  Non-homogeneous Content-driven Video-retargeting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[26]  Marcel P. Lucassen,et al.  Visual comfort of binocular and 3D displays , 2001, IS&T/SPIE Electronic Imaging.

[27]  Yung-Yu Chuang,et al.  Scene warping: Layer-based stereoscopic image resizing , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Wen-Huang Cheng,et al.  Perspective-aware warping for seamless stereoscopic image cloning , 2012, ACM Trans. Graph..

[29]  Bernard Mendiburu,et al.  3D Movie Making: Stereoscopic Digital Cinema from Script to Screen , 2009 .

[30]  Rafael Grompone von Gioi,et al.  LSD: a Line Segment Detector , 2012, Image Process. Line.

[31]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[32]  Yun Zhang,et al.  StereoPasting: Interactive Composition in Stereoscopic Images , 2013, IEEE Transactions on Visualization and Computer Graphics.

[33]  S. Avidan,et al.  Seam carving for content-aware image resizing , 2007, SIGGRAPH 2007.

[34]  Olga Sorkine-Hornung,et al.  Optimized scale-and-stretch for image resizing , 2008, SIGGRAPH Asia '08.