Adaptive data-driven parallelization of multi-view video coding on multi-core processor

Multi-view video coding (MVC) comprises rich 3D information and is widely used in new visual media, such as 3DTV and free viewpoint TV (FTV). However, even with mainstream computer manufacturers migrating to multi-core processors, the huge computational requirement of MVC currently prohibits its wide use in consumer markets. In this paper, we demonstrate the design and implementation of the first parallel MVC system on Cell Broadband Engine™ processor which is a state-of-the-art multi-core processor. We propose a task-dispatching algorithm which is adaptive data-driven on the frame level for MVC, and implement a parallel multi-view video decoder with modified H.264/AVC codec on real machine. This approach provides scalable speedup (up to 16 times on sixteen cores) through proper local store management, utilization of code locality and SIMD improvement. Decoding speed, speedup and utilization rate of cores are expressed in experimental results.

[1]  H. Peter Hofstee,et al.  Introduction to the Cell multiprocessor , 2005, IBM J. Res. Dev..

[2]  Masayuki Tanimoto,et al.  FTV (free viewpoint television) creating ray-based image engineering , 2005, IEEE International Conference on Image Processing 2005.

[3]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[4]  Aljoscha Smolic,et al.  Interactive 3-D Video Representation and Coding Technologies , 2005, Proceedings of the IEEE.

[5]  Edward J. Delp,et al.  Parallel implementation of an MPEG-1 encoder: faster than real time , 1995, Electronic Imaging.

[6]  B. Flachs,et al.  A streaming processing unit for a CELL processor , 2005, ISSCC. 2005 IEEE International Digest of Technical Papers. Solid-State Circuits Conference, 2005..

[7]  Anthony Vetro,et al.  Extensions of H.264/AVC for Multiview Video Compression , 2006, 2006 International Conference on Image Processing.

[8]  Erik B. van der Tol,et al.  Mapping of MPEG-4 decoding on a flexible architecture platform , 2001, IS&T/SPIE Electronic Imaging.

[9]  Erik B. van der Tol,et al.  Mapping of H.264 decoding on a multiprocessor architecture , 2003, IS&T/SPIE Electronic Imaging.

[10]  Spatial and temporal data parallelization of the H.261 video coding algorithm , 2001, IEEE Trans. Circuits Syst. Video Technol..

[11]  Neil A. Dodgson,et al.  Autostereoscopic 3D displays , 2005, Computer.

[12]  Mei Yu,et al.  Parallel Process of Hyper-Space-Based Multiview Video Compression , 2006, 2006 International Conference on Image Processing.

[13]  Ashwini K. Nanda,et al.  Cell/B.E. blades: Building blocks for scalable, real-time, interactive, and digital media servers , 2007, IBM J. Res. Dev..

[14]  Lifeng Sun,et al.  Spatial and Temporal Data Parallelization of Multi-view Video Encoding Algorithm , 2007, 2007 IEEE 9th Workshop on Multimedia Signal Processing.

[15]  Hui Wang,et al.  A novel HDTV video decoder and decentralized control scheme , 2001, IEEE Trans. Consumer Electron..

[16]  Martin Hopkins,et al.  A novel SIMD architecture for the cell heterogeneous chip-multiprocessor , 2005, 2005 IEEE Hot Chips XVII Symposium (HCS).