Manipulation and Compositing of MC-DCT Compressed Video

Many advanced video applications require manipulations of compressed video signals. Popular video manipulation functions include overlap (opaque or semitransparent), translation, scaling, linear filtering, rotation, and pixel multiplication. We propose algorithms to manipulate compressed video in the compressed domain. Specifically, we focus on compression algorithms using the discrete cosine transform (DCT) with or without motion compensation (MC). Such compression systems include JPEG, motion JPEG, MPEG, and H.261. We derive a complete set of algorithms for all aforementioned manipulation functions in the transform domain, in which video signals are represented by quantized transform coefficients. Due to a much lower data rate and the elimination of decompression/compression conversion, the transform-domain approach has great potential in reducing the computational complexity. The actual computational speedup depends on the specific manipulation functions and the compression characteristics of the input video, such as the compression rate and the nonzero motion vector percentage. The proposed techniques can be applied to general orthogonal transforms, such as the discrete trigonometric transform. For compression systems incorporating MC (such as MPEG), we propose a new decoding algorithm to reconstruct the video in the transform domain and then perform the desired manipulations in the transform domain. The same technique can be applied to efficient video transcoding (e.g., from MPEG to JPEG) with minimal decoding. >

[1]  Shih-Fu Chang,et al.  New algorithms for processing images in the transform-compressed domain , 1995, Other Conferences.

[2]  B. Chitprasert,et al.  Discrete cosine transform filtering , 1990 .

[3]  N. Ahmed,et al.  Discrete Cosine Transform , 1996 .

[4]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[5]  Kou-Hu Tzou,et al.  HDTV And The Emerging Broadband ISDN Network , 1988, Other Conferences.

[6]  Stephen A. Martucci,et al.  Symmetric convolution and the discrete sine and cosine transforms , 1993, IEEE Trans. Signal Process..

[7]  Shih-Fu Chang,et al.  Compositing motion-compensated video within the network , 1992, CCRV.

[8]  Thomas Ertl,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 2014 .

[9]  Tore Fjällbrant,et al.  A direct computation of DCT coefficients for a signal block taken from two adjacent blocks , 1991, IEEE Trans. Signal Process..

[10]  Carl F. R. Weiman,et al.  Continuous anti-aliased rotation and zoom of raster images , 1980, SIGGRAPH '80.

[11]  J. D. Robbins,et al.  Motion-compensated television coding: Part I , 1979, The Bell System Technical Journal.

[12]  Wen-Hsiung Chen,et al.  A Fast Computational Algorithm for the Discrete Cosine Transform , 1977, IEEE Trans. Commun..

[13]  Byeong Gi Lee,et al.  Transform domain filtering based on pipelining structure , 1992, IEEE Trans. Signal Process..

[14]  Shih-Fu Chang,et al.  Error accumulation of repetitive image coding , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[15]  M. Kunt,et al.  Second-generation image-coding techniques , 1985, Proceedings of the IEEE.

[16]  Shih-Fu Chang,et al.  A new approach to decoding and compositing motion-compensated DCT-based images , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[17]  K. R. Rao,et al.  Discrete cosine transform filtering , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[18]  John W. Woods,et al.  Video Post-Production with Compressed Images , 1994 .

[19]  R.K. Jurgen.  The challenges of digital HDTV , 1991, IEEE Spectrum.

[20]  Tom Duff,et al.  Compositing 3-D rendered images , 1985, SIGGRAPH.

[21]  Tom Duff,et al.  Compositing digital images , 1984, SIGGRAPH.

[22]  Lawrence A. Rowe,et al.  Algorithms for manipulating compressed images , 1993, IEEE Computer Graphics and Applications.

[23]  K. L. Ho,et al.  A new two-dimensional fast cosine transform algorithm , 1991, IEEE Trans. Signal Process..

[24]  Jorma Rissanen,et al.  Compression of Black-White Images with Arithmetic Coding , 1981, IEEE Trans. Commun..

[25]  Didier Le Gall,et al.  MPEG: a video compression standard for multimedia applications , 1991, CACM.

[26]  R. J. Clarke,et al.  Lowpass filtering in the cosine transform domain , 1980 .

[27]  Ming Lei Liou,et al.  Overview of the p×64 kbit/s video coding standard , 1991, CACM.

[28]  R. R. Clarke.  Transform coding of images , 1985 .

[29]  K WallaceGregory.  The JPEG still picture compression standard , 1991 .

[30]  P. Venkat Rangan,et al.  Communication architectures and algorithms for media mixing in multimedia conferences , 1993, TNET.

[31]  Anil K. Jain,et al.  Displacement Measurement and Its Application in Interframe Image Coding , 1981, IEEE Trans. Commun..

[32]  M. J. Narasimha,et al.  On the Computation of the Discrete Cosine Transform , 1978, IEEE Trans. Commun..