Spherical Coordinates Transform-Based Motion Model for Panoramic Video Coding

Panoramic video is produced and displayed as if it is supported on sphere, but it is usually mapped to two-dimensional (2D) plane for compression purpose. The mapping from sphere to 2D plane inevitably causes deformation of video content, which shall be taken into account in panoramic video coding. In this paper, we propose a new motion model based on spherical coordinates transform to explicitly address the deformation problem. Different from the traditional 2D translational motion model, our model assumes a 3D translational motion, which is characterized by a 2D motion vector and a relative depth parameter. Based on spherical coordinates transform, we can deduce the motion compensation and motion estimation algorithms for our proposed motion model. We then integrate the algorithms into the joint exploration model (JEM) scheme for panoramic video coding, and perform experiments on some typical equirectangular projection format video sequences. Experimental results show that our proposed method leads to up to 10.7% and on average 2.6% BD-rate reduction compared to the JEM anchor on our test sequences, with no increase of decoding time.

[1]  Andrew Chi-Sing Leung,et al.  The Rhombic Dodecahedron Map: An Efficient Scheme for Encoding Panoramic Video , 2009, IEEE Transactions on Multimedia.

[2]  Miska M. Hannuksela,et al.  Fisheye video coding using elastic motion compensated reference frames , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[3]  Y. Ohtani,et al.  Panorama video coding for user-driven interactive video application , 2009, 2009 IEEE 13th International Symposium on Consumer Electronics.

[4]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[5]  Detlev Marpe,et al.  Block Merging for Quadtree-Based Partitioning in HEVC , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  André Kaup,et al.  Motion estimation for fisheye video sequences combining perspective projection with camera calibration information , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[7]  Kenneth Rose,et al.  Rotational motion model for temporal prediction in 360 video coding , 2017, 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP).

[8]  Mark R. Pickering,et al.  Video Coding Using Elastic Motion Model and Larger Blocks , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Li Li,et al.  A new motion model for panoramic video coding , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[10]  Ned Greene,et al.  Environment Mapping and Other Applications of World Projections , 1986, IEEE Computer Graphics and Applications.

[11]  Zhu Li,et al.  Projection based advanced motion model for cubic mapping for 360-degree video , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[12]  Matthias Narroschke,et al.  Extending HEVC by an affine motion model , 2013, 2013 Picture Coding Symposium (PCS).

[13]  Dong Liu,et al.  An Efficient Four-Parameter Affine Motion Model for Video Coding , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Madhukar Budagavi,et al.  Motion estimation and compensation for fisheye warped video , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[16]  Michael T. Orchard,et al.  Overlapped block motion compensation: an estimation-theoretic approach , 1994, IEEE Trans. Image Process..

[17]  Zehdreh Allen-Lafayette,et al.  Flattening the Earth, Two Thousand Years of Map Projections , 1998 .