A software-only videocodec using pixelwise conditional differential replenishment and perceptual enhancements

Designing a videocodec involves a four-way tradeoff among computational complexity, data rate, picture quality, and latency. Rapid advancement in very large-scale integration technology has provided CPUs with enough power to accommodate a software-only videocodec. Accordingly, computational complexity has resurfaced as a major element in this tradeoff. With a view toward significantly reducing computational complexity relative to standards-based videocodecs, we introduce a pixelwise conditional differential replenishment scheme to compress video via perception-sensitive decomposition of difference frames into a facsimile map and an intensity vector. Our schemes, which apply techniques from facsimile, are transform free. Some of them also involve no motion compensation and hence are completely free of block-based artifacts and particularly computationally economical. The fusion of our facsimile-based video-coding schemes and spatio-temporal perceptual-coding techniques facilitates powerful software-only video conferencing on today's medium- and highend personal computers. Indeed, assuming that a frame-capture driver has been provided, our motion-compensation-free approach has yielded a software-only, full-duplex, full-color videoconferencing system that conveys high-quality, CIF/Q-NTSC-sized video at 30 frames per second on 200-MHz Pentium PCs sending less than 300 Kbps in each direction. We also present new spatio-temporal compression techniques for perceptual coding of video. These techniques, motivated by the classical psychological experiments that led to formulation of the Weber-Fechner law, allow videocodec systems to capitalize on properties of the human visual system. Some of our spatiotemporal perceptual techniques not only apply to our proprietary pixelwise conditional differential replenishment schemes that we describe for video conferencing but also can readily be incorporated into today's popular video standards.

[1]  J. Robson,et al.  Application of fourier analysis to the visibility of gratings , 1968, The Journal of physiology.

[2]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[3]  V. John Mathews,et al.  A new model of perceptual threshold functions for application in image compression systems , 1995, Proceedings DCC '95 Data Compression Conference.

[4]  F. W. Mounts.  A video encoding system with conditional picture-element replenishment , 1969 .

[5]  Toby Berger,et al.  Video compression using fax techniques , 1996, Proceedings of Data Compression Conference - DCC '96.

[6]  Sarah A. Rajala,et al.  Subband/VQ coding of color images with perceptually optimal bit allocation , 1994, IEEE Trans. Circuits Syst. Video Technol..

[7]  Toby Berger,et al.  High-speed software implementation of Huffman coding , 1998, Proceedings DCC '98 Data Compression Conference (Cat. No.98TB100225).

[8]  E. Boring.  Sensation and Perception. (Scientific Books: Sensation and Perception in the History of Experimental Psychology) , 1943 .

[9]  David L. Neuhoff,et al.  Perceptual coding of images for halftone display , 1994, IEEE Trans. Image Process..

[10]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[11]  Robert J. Safranek,et al.  Signal compression based on models of human perception , 1993, Proc. IEEE.

[12]  Y.-J. Chiu.  A perceptual preprocessor to segment video for motion estimation , 1998, Proceedings DCC '98 Data Compression Conference (Cat. No.98TB100225).

[13]  Ja-Ling Wu,et al.  Novel real-time software-based video coding algorithms , 1993 .

[14]  James C. Candy,et al.  Transmitting television as clusters of frame-to-frame differences , 1971 .

[15]  M. Kunt,et al.  Block coding of graphics: A tutorial review , 1980, Proceedings of the IEEE.

[16]  Atul Puri,et al.  Motion-compensated video coding with adaptive perceptual quantization , 1991, IEEE Trans. Circuits Syst. Video Technol..

[17]  Kannan Ramchandran,et al.  Rate-distortion optimal fast thresholding with complete JPEG/MPEG decoder compatibility , 1994, IEEE Trans. Image Process..

[18]  James C. Candy,et al.  Interframe coding of videotelephone pictures , 1972 .

[19]  B. Prasada,et al.  Adaptive quantization of picture signals using spatial masking , 1977, Proceedings of the IEEE.

[20]  Nasser M. Nasrabadi,et al.  Image coding using vector quantization: a review , 1988, IEEE Trans. Commun..

[21]  William E. Glenn.  Digital Image Compression Based on Visual Perception and Scene Properties , 1993 .

[22]  Arun N. Netravali,et al.  Digital Video: An introduction to MPEG-2 , 1996 .

[23]  Chun-Hsien Chou,et al.  A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile , 1995, IEEE Trans. Circuits Syst. Video Technol..