Automatic stereoscopic 3D video reframing

3D displays have various aspect ratios (e.g., 16:9, 4:3, and 3:2). Watching 3D videos with the wrong aspect ratio decreases the quality of the viewing experience. We have developed a smart reframing solution that uses a visual attention model for stereoscopic 3D video to identify the prominent visual regions of every stereoscopic frame. Our method uses several saliency indicators such as depth, edges, brightness, color, and movement. Additionally, our method provides a dynamic cropping window that slides smoothly from frame to frame.

[1]  Heinz Hügli,et al.  Computing visual attention from scene depth , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[2]  S. Engel,et al.  Colour tuning in human visual cortex measured with functional magnetic resonance imaging , 1997, Nature.

[3]  Patrick Le Callet,et al.  A coherent computational approach to model bottom-up visual attention , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Stephen J. Sangwine,et al.  Hypercomplex Fourier Transforms of Color Images , 2007, IEEE Trans. Image Process..

[5]  Aljoscha Smolic,et al.  Nonlinear disparity mapping for stereoscopic 3D , 2010, SIGGRAPH 2010.

[6]  Ken Chen,et al.  Stereoscopic Visual Attention Model for 3D Video , 2010, MMM.

[7]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[8]  Christel Chamaret,et al.  Spatio-temporal combination of saliency maps and eye-tracking assessment of different strategies , 2010, 2010 IEEE International Conference on Image Processing.

[9]  Liming Zhang,et al.  Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[11]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[12]  Aljoscha Smolic,et al.  Nonlinear disparity mapping for stereoscopic 3D , 2010, ACM Trans. Graph..

[13]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[14]  O. Meur,et al.  Predicting visual fixations on video based on low-level visual features , 2007, Vision Research.

[15]  Atsuto Maki,et al.  A computational model of depth-based attention , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[16]  Marcus Barkowsky,et al.  The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives , 2011, IEEE Transactions on Broadcasting.

[17]  Christel Chamaret,et al.  Attention-based video reframing: Validation using eye-tracking , 2008, 2008 19th International Conference on Pattern Recognition.