Hybrid shift map for video retargeting

We propose a new method for video retargeting, which can generate spatial-temporal consistent video. The new measure called spatial-temporal naturality preserves the motion in the source video without any motion analysis in contrast to other methods that need motion estimation. This advantage prevents the retargeted video from degenerating due to the propagation of the errors in motion analysis. It allows the proposed method to be applied on challenging videos with complex camera and object motion. To improve the efficiency of the retargeting process, we retarget video using a 3D shift map in low resolution and refine it using an incremental 2D shift map in higher resolution. This new hierarchical framework, denoted as hybrid shift map, can produce satisfactory retargeting results while significantly improving the computational efficiency.

[1]  Ariel Shamir,et al.  Seam Carving for Content-Aware Image Resizing , 2007, ACM Trans. Graph..

[2]  Pradeep Sen,et al.  Video Carving , 2008, Eurographics.

[3]  Sung-Jea Ko,et al.  Wavelet based seam carving for content-aware image resizing , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[4]  Hua Huang,et al.  Real-time content-aware image resizing , 2009, Science in China Series F: Information Sciences.

[5]  Leo Grady,et al.  A multilevel banded graph cuts method for fast image segmentation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Daniel Cohen-Or,et al.  Feature-aware texturing , 2006, EGSR '06.

[7]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[8]  Ariel Shamir,et al.  Improved seam carving for video retargeting , 2008, SIGGRAPH 2008.

[9]  Xing Xie,et al.  Browsing large pictures under limited display sizes , 2006, IEEE Transactions on Multimedia.

[10]  Yael Pritch,et al.  Shift-map image editing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[11]  Ariel Shamir,et al.  Cropping Scaling Seam carving Warping Multi-operator , 2009 .

[12]  David Salesin,et al.  Gaze-based interaction for semi-automatic photo cropping , 2006, CHI.

[13]  Shang-Hong Lai,et al.  Fast structure-preserving image retargeting , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Michael Gleicher,et al.  Video retargeting: automating pan and scan , 2006, MM '06.

[15]  O. Sorkine,et al.  Optimized scale-and-stretch for image resizing , 2008, SIGGRAPH 2008.

[16]  Chang-Su Kim,et al.  Adaptive image and video retargeting technique based on Fourier analysis , 2009, CVPR.

[17]  Hermann Ney,et al.  Pan, zoom, scan — Time-coherent, trained automatic video cropping , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Jiaya Jia,et al.  Active Window Oriented Dynamic Video Retargeting , 2007 .

[19]  Denis Simakov,et al.  Summarizing visual data using bidirectional similarity , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Daniel Cohen-Or,et al.  Non-homogeneous Content-driven Video-retargeting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[21]  Hans-Peter Seidel,et al.  Motion-aware temporal coherence for video resizing , 2009, ACM Trans. Graph..