Human Vision Models for Perceptually Optimized Image Processing – A Review

By taking into account the properties and limitations of the human visual system (HVS), images can be more efficiently compressed, colors more accurately reproduced, prints better rendered, to mention a few major advantages. To achieve these goals it is necessary to build a computational model of the HVS. In this paper we give an introduction to the general issue of HVS-modeling and review the specific applications of visual quality assessment and HVS-based image compression, which are closely related. On one hand, these two examples demonstrate the common structure of HVS-models, on the other hand they also show how application-specific constraints influence model design. Recent vision models from these application areas are reviewed and summarized in a table for direct comparison. Keywords— Human Visual System (HVS), Color Perception, Quality Assessment, Image Compression

[1]  D. Dacey,et al.  This paper was presented at a colloquium entitled ‘ ‘ Vision : From Photon to Perception , ’ ’ organized by , 1998 .

[2]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[3]  David J. Sakrison,et al.  The effects of a visual fidelity criterion of the encoding of images , 1974, IEEE Trans. Inf. Theory.

[4]  Per Lindh,et al.  Efficient spatio-temporal decomposition for perceptual processing of video sequences , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[5]  C.-C. Jay Kuo,et al.  A Haar Wavelet Approach to Compressed Image Quality Measurement , 2000, J. Vis. Commun. Image Represent..

[6]  C. van den Branden Lambrecht.  Color moving pictures quality metric , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[7]  S L Guth,et al.  Model for color vision and light adaptation. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[8]  Murat Kunt,et al.  Quality assessment of motion rendition in video coding , 1999, IEEE Trans. Circuits Syst. Video Technol..

[9]  Ralph E. Jacobson,et al.  An Evaluation of Image Quality Metrics , 1995 .

[10]  Jerome R. Cox,et al.  Experimental evaluation of psychophysical distortion metrics for JPEG-encoded images , 1993, Electronic Imaging.

[11]  John D. Villasenor,et al.  Visibility of wavelet quantization noise , 1997, IEEE Transactions on Image Processing.

[12]  Albert J. Ahumada,et al.  Image quality: a multidimensional problem , 1993 .

[13]  Bernd Girod,et al.  The Information Theoretical Significance of Spatial and Temporal Masking in Video Signals , 1989, Photonics West - Lasers and Applications in Science and Engineering.

[14]  Anthony M. Norcia,et al.  Modelfest: year one results and plans for future years , 2000, Electronic Imaging.

[15]  Murat Kunt,et al.  Integer wavelet transform for embedded lossy to lossless image compression , 2001, IEEE Trans. Image Process..

[16]  Robert Eriksson,et al.  Spatiotemporal discrimination model predicts temporal masking functions , 1998, Electronic Imaging.

[17]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[18]  Joan L. Mitchell,et al.  JPEG: Still Image Data Compression Standard , 1992 .

[19]  O. Schade.  Optical and photoelectric analog of the eye. , 1956, Journal of the Optical Society of America.

[20]  J. Reichel,et al.  Wavelet compression of color images with reference to the HVS , 1999 .

[21]  Stefan Winkler,et al.  Computing isotropic local contrast from oriented pyramid decompositions , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[22]  Paul S. Fisher,et al.  Image quality measures and their performance , 1995, IEEE Trans. Commun..

[23]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[24]  Jeffrey B. Mulligan,et al.  Design and performance of a digital video quality metric , 1999, Electronic Imaging.

[25]  Wei Wu,et al.  Contrast gain control for color image quality , 1998, Electronic Imaging.

[26]  Andrew P. Bradley,et al.  A wavelet visible difference predictor , 1999, IEEE Trans. Image Process..

[27]  Alexander I. Drukarev.  Compression-related properties of color spaces , 1997, Electronic Imaging.

[28]  Joel Pokorny,et al.  Responses of macaque ganglion cells and human observers to compound periodic waveforms , 1993, Vision Research.

[29]  C. Enroth-Cugell,et al.  Chapter 9 Visual adaptation and retinal gain controls , 1984 .

[30]  Stefan Winkler,et al.  Perceptual distortion metric for digital color video , 1999, Electronic Imaging.

[31]  Stefan Winkler,et al.  Quality metric design: a closer look , 2000, Electronic Imaging.

[32]  Hermann Grassmann,et al.  Zur Theorie der Farbenmischung , 1853 .

[33]  Stefan Winkler,et al.  Issues in vision modeling for perceptual video quality assessment , 1999, Signal Process..

[34]  E. Peli.  In search of a contrast metric: Matching the perceived contrast of gabor patches at different phases and bandwidths , 1997, Vision Research.

[35]  R. L. Valois,et al.  A multi-stage color model , 1993, Vision Research.

[36]  Stanley A. Klein,et al.  Image quality and image compression: a psychophysicist's viewpoint , 1993 .

[37]  D. Jameson,et al.  Some quantitative aspects of an opponent-colors theory. II. Brightness, saturation, and hue in normal and dichromatic vision. , 1955, Journal of the Optical Society of America.

[38]  V. Ralph Algazi,et al.  Objective picture quality scale (PQS) for image coding , 1998, IEEE Trans. Commun..

[39]  Jean-Bernard Martens,et al.  Image quality prediction in a multidimensional perceptual space , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[40]  Hans Marmolin,et al.  Subjective MSE Measures , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[41]  Anastasios N. Venetsanopoulos,et al.  A perceptual model for JPEG applications based on block classification, texture masking, and luminance masking , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[42]  D. Baylor,et al.  Spectral sensitivity of cones of the monkey Macaca fascicularis. , 1987, The Journal of physiology.

[43]  Patrick C. Teo,et al.  Perceptual image distortion , 1994, Proceedings of 1st International Conference on Image Processing.

[44]  E. Peli.  Contrast in complex images. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[45]  Julien Reichel,et al.  Opponent Color, Human Vision and Wavelets for Image Compression , 1999, Color Imaging Conference.

[46]  D. H. Kelly.  Spatiotemporal variation of chromatic and achromatic contrast thresholds. , 1983, Journal of the Optical Society of America.

[47]  Robert J. Safranek,et al.  Perceptual coding of images , 1993, Electronic Imaging.

[48]  D. G. Albrecht,et al.  Motion selectivity and the contrast-response function of simple cells in the visual cortex , 1991, Visual Neuroscience.

[49]  A. Stockman,et al.  The spectral sensitivity of the human short-wavelength sensitive cones derived from thresholds and color matches , 1999, Vision Research.

[50]  D. Hubel,et al.  Segregation of form, color, movement, and depth: anatomy, physiology, and perception. , 1988, Science.

[51]  Peter G. J. Barten,et al.  Contrast sensitivity of the human eye and its e ects on image quality , 1999 .

[52]  Stefan Winkler,et al.  A perceptual distortion metric for digital color images , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[53]  Margaret H. Pinson,et al.  Validating Objective Measures of MPEG Video Quality , 1998 .

[54]  P. Lennie,et al.  Chromatic mechanisms in lateral geniculate nucleus of macaque. , 1984, The Journal of physiology.

[55]  D. Amnon Silverstein,et al.  Relevance of human vision to JPEG-DCT compression , 1992, Electronic Imaging.

[56]  N. Graham,et al.  Normalization: contrast-gain control in simple (Fourier) and complex (non-Fourier) pathways of pattern vision , 2000, Vision Research.

[57]  Xin Tong,et al.  Video quality evaluation using ST-CIELAB , 1999, Electronic Imaging.

[58]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[59]  Jeremy Nathans,et al.  Absorption spectra of human cone pigments , 1992, Nature.

[60]  S. Wu,et al.  Synaptic transmission in the outer retina. , 1994, Annual review of physiology.

[61]  Jesús Malo,et al.  Subjective image fidelity metric based on bit allocation of the human visual system in the DCT domain , 1997, Image Vis. Comput..

[62]  R. Hess,et al.  Estimating multiple temporal mechanisms in human vision , 1998, Vision Research.

[63]  J A Solomon,et al.  Model of visual contrast gain control and pattern masking. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[64]  Anil C. Kokaram,et al.  Perceptual distortion measure for edgelike artifacts in image sequences , 1998, Electronic Imaging.

[65]  I. Daubechies,et al.  Factoring wavelet transforms into lifting steps , 1998 .

[66]  Norman B. Nill,et al.  A Visual Model Weighted Cosine Transform for Image Compression and Quality Assessment , 1985, IEEE Trans. Commun..

[67]  Sarah A. Rajala,et al.  Subband/VQ Coding of Color Images Using a Separable Diamond Decomposition , 1994, J. Vis. Commun. Image Represent..

[68]  Jeffrey Lubin,et al.  A VISUAL DISCRIMINATION MODEL FOR IMAGING SYSTEM DESIGN AND EVALUATION , 1995 .

[69]  Andrew B. Watson,et al.  The cortex transform: rapid computation of simulated neural images , 1987 .

[70]  Robert J. Safranek,et al.  Image and video compression : A review , 1997 .

[71]  R. J. Safranek,et al.  A perceptually tuned sub-band image coder with image dependent quantization and post-quantization data compression , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[72]  C.-C. Jay Kuo,et al.  Wavelet image compression with optimized perceptual quality , 1998, Optics & Photonics.

[73]  D. Heeger.  Half-squaring in responses of cat striate cells , 1992, Visual Neuroscience.

[74]  Andreas E. Savakis,et al.  Evaluation of image appeal in consumer photography , 2000, Electronic Imaging.

[75]  M A García-Pérez.  The perceived image: efficient modelling of visual inhomogeneity. , 1992, Spatial vision.

[76]  Frederic Truchetet,et al.  High-quality still color image compression , 2000 .

[77]  M. G. Albanesi,et al.  Human vision model and wavelets for high-quality image compression , 1995 .

[78]  Touradj Ebrahimi,et al.  A study of JPEG 2000 still image coding versus other standards , 2000, 2000 10th European Signal Processing Conference.

[79]  Andrew B. Watson,et al.  Image quality and entropy masking , 1997, Electronic Imaging.

[80]  Hocine Cherifi,et al.  A comparison of image quality models and metrics based on human visual sensitivity , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[81]  M. J. Nadenau,et al.  Compression of color images with wavelets under consideration of the HVS , 1999 .

[82]  Gary W. Meyer,et al.  Comparison of two image quality models , 1998, Electronic Imaging.

[83]  Zhaoping Li,et al.  Understanding Retinal Color Coding from First Principles , 1992, Neural Computation.

[84]  Jaj Jacques Roufs,et al.  PERCEPTUAL IMAGE QUALITY: CONCEPT AND MEASUREMENT , 1992 .

[85]  K. Mullen.  The contrast sensitivity of human colour vision to red‐green and blue‐yellow chromatic gratings. , 1985, The Journal of physiology.

[86]  Scott Daly,et al.  Engineering observations from spatiovelocity and spatiotemporal visual models , 1998, Electronic Imaging.

[87]  A. Stockman,et al.  The spectral sensitivities of the middle- and long-wavelength-sensitive cones derived from measurements in observers of known genotype , 2000, Vision Research.

[88]  G. Buchsbaum,et al.  Trichromacy, opponent colours coding and optimum colour information transmission in the retina , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[89]  Roger S. Gaborski,et al.  Comparative study of wavelet and discrete cosine transform (DCT) decompositions with equivalent quantization and encoding strategies for medical images , 1995, Medical Imaging.

[90]  Albert J. Ahumada,et al.  Computational image quality metrics: A review , 1993 .

[91]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[92]  Andrew B. Watson,et al.  DCT quantization matrices visually optimized for individual images , 1993, Electronic Imaging.

[93]  D.J. Granrath,et al.  The role of human visual models in image processing , 1981, Proceedings of the IEEE.

[94]  J. Pokorny,et al.  Spectral sensitivity of the foveal cone photopigments between 400 and 500 nm , 1975, Vision Research.

[95]  E. M. Granger,et al.  Visual chromaticity-modulation transfer function , 1973 .

[96]  A B Watson,et al.  Efficiency of a model human image code. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[97]  M. Ghanbari,et al.  An objective measurement tool for MPEG video quality , 1998, Signal Process..

[98]  Stanley A. Klein,et al.  Seven models of masking , 1997, Electronic Imaging.

[99]  Margaret H. Pinson,et al.  Spatial-temporal distortion metric for in-service quality monitoring of any digital video system , 1999, Optics East.

[100]  Robert L. Stevenson,et al.  Human Visual System Based Wavelet Decomposition for Image Compression , 1995, J. Vis. Commun. Image Represent..

[101]  Huib de Ridder,et al.  Naturalness and image quality: chroma and hue variation in color images of natural scenes , 1995, Electronic Imaging.

[102]  Andrew B. Watson,et al.  Toward a perceptual video-quality metric , 1998, Electronic Imaging.

[103]  J. Daugman.  Two-dimensional spectral analysis of cortical receptive field profiles , 1980, Vision Research.

[104]  Michael A. Webster,et al.  Human colour perception and its adaptation , 1996 .

[105]  D. Ruderman,et al.  Statistics of cone responses to natural images: implications for visual coding , 1998 .

[106]  O. Faugeras.  Digital color image processing within the framework of a human visual model , 1979 .

[107]  Julien Reichel,et al.  Image-compression-related contrast-masking measurements , 2000, Electronic Imaging.

[108]  David S. Taubman,et al.  High performance scalable image compression with EBCOT , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[109]  M. A. Bouman,et al.  Spatiotemporal chromaticity discrimination. , 1969, Journal of the Optical Society of America.

[110]  A. M. Rohaly,et al.  Object detection in natural backgrounds predicted by discrimination performance and models , 1997, Vision Research.

[111]  Reginald L. Lagendijk,et al.  Optimization of JPEG color image coding using a human visual system model , 1996, Electronic Imaging.

[112]  Andrew B. Watson,et al.  Perceptual optimization of DCT color quantization matrices , 1994, Proceedings of 1st International Conference on Image Processing.

[113]  Walter Makous,et al.  Spatiotemporal separability in contrast sensitivity , 1994, Vision Research.

[114]  Olivier Verscheure,et al.  Perceptual quality measure using a spatiotemporal model of the human visual system , 1996, Electronic Imaging.

[115]  R. M. Boynton.  Human color vision , 1979 .

[116]  Robert Eriksson,et al.  Modeling the perception of digital images: a performance study , 1998, Electronic Imaging.

[117]  Jean-Bernard Martens,et al.  Subjective quality assessment of compressed images , 1997, Signal Process..

[118]  Stefan Winkler,et al.  Video Quality Experts Group: current results and future directions , 2000, Visual Communications and Image Processing.

[119]  W. Ehrenstein,et al.  Psychophysical Methods , 1999 .

[120]  D. Heeger.  Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[121]  William F. Schreiber,et al.  Fundamentals of Electronic Imaging Systems , 1986 .

[122]  A B Watson,et al.  Perceptual-components architecture for digital video. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[123]  Shuichi Matsumoto,et al.  Picture quality assessment system by three-layered bottom-up noise weighting considering human visual perception , 1999 .

[124]  C. Lambrecht.  Perceptual models and architectures for video coding applications , 1996 .

[125]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[126]  Zigmantas L. Budrikis,et al.  Detail perception after scene changes in television image presentations , 1965, IEEE Trans. Inf. Theory.

[127]  M. Farah,et al.  A functional MRI study of mental image generation , 1997, Neuropsychologia.

[128]  Charles F. Hall.  The Application Of Human Visual System Models To Digital Color Image Compression , 1986, Other Conferences.

[129]  Z. L. Budrikis,et al.  Picture Quality Prediction Based on a Visual Model , 1982, IEEE Trans. Commun..

[130]  Paul Bao,et al.  Wavelet transform image coding based on fuzzy visual perception modeling , 1998, Defense, Security, and Sensing.

[131]  R. F. Hess,et al.  Temporal properties of human visual filters: number, shapes and spatial covariation , 1992, Vision Research.

[132]  Michael P. Eckert.  Lossy compression using wavelets, block DCT, and lapped orthogonal transforms optimized with a perceptual model , 1997, Medical Imaging.

[133]  B. Wandell.  Foundations of vision , 1995 .

[134]  Gary W. Meyer,et al.  Visual difference metric for realistic image synthesis , 1999, Electronic Imaging.

[135]  Joel Pokorny,et al.  Responses to pulses and sinusoids in macaque ganglion cells , 1994, Vision Research.

[136]  Huib de Ridder,et al.  Perceptually optimal color reproduction , 1998, Electronic Imaging.