Effect of content features on short-term video quality in the visual periphery

The area outside our central field of vision, also referred to as the visual periphery, captures most information in a visual scene, although much less sensitive than the central Fovea. Vision studies in the past have stated that there is reduced sensitivity of texture, color, motion and flicker (temporal harmonic) perception in this area, that bears an interesting application in the domain of quality perception. In this work, we particularly analyze the perceived subjective quality of videos containing H.264/AVC transmission impairments, incident at various degrees of retinal eccentricities of observers. We relate the perceived drop in quality, to five basic types of features that are important from a perceptive standpoint: texture, color, flicker, motion trajectory distortions and also the semantic importance of the underlying regions. We are able to observe that the perceived drop in quality across the visual periphery, is closely related to the Cortical Magnification fall-off characteristics of the V1 cortical region. Additionally, we see that while object importance and low frequency spatial distortions are important indicators of quality in the central foveal region, temporal flicker and color distortions are the most important determinants of quality in the periphery. We therefore conclude that, although users are more forgiving of distortions they viewed peripherally, they are nevertheless not totally blind towards it: the effects of flicker and color distortions being particularly important.

[1]  Sanjit K. Mitra,et al.  H.264 coding artifacts and their relation to perceived annoyance , 2006, 2006 14th European Signal Processing Conference.

[2]  Ingrid Heynderickx,et al.  Perceivable artifacts in compressed video and their relation to video quality , 2009, Signal Process. Image Commun..

[3]  S. Dumoulin,et al.  The Relationship between Cortical Magnification Factor and Population Receptive Field Size in Human Visual Cortex: Constancies in Cortical Architecture , 2011, The Journal of Neuroscience.

[4]  U. Tulunay-Keesey,et al.  Threshold and suprathreshold spatiotemporal response throughout adulthood. , 1988, Journal of the Optical Society of America. A, Optics and image science.

[5]  Ulrich Engelke,et al.  Modelling saliency awareness for objective video quality assessment , 2010, 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX).

[6]  Sanjit K. Mitra,et al.  Modeling subjectively perceived annoyance of H.264/AVC video as a function of perceived artifact strength , 2010, Signal Process..

[7]  A. Hendrickson,et al.  Human photoreceptor topography , 1990, The Journal of comparative neurology.

[8]  M. Iwasaki,et al.  Relation between superficial capillaries and foveal structures in the human retina. , 1986, Investigative ophthalmology & visual science.

[9]  T. Nealey,et al.  Magnocellular and parvocellular contributions to responses in the middle temporal visual area (MT) of the macaque monkey , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[10]  J. Rovamo,et al.  Temporal contrast sensitivity and cortical magnification , 1982, Vision Research.

[11]  R. Scobey,et al.  Detection of image displacement by phasic cells in peripheral visual fields of the monkey , 1976, Vision Research.

[12]  Patrick Le Callet,et al.  Robust approach for color image quality assessment , 2003, Visual Communications and Image Processing.

[13]  Marcus Barkowsky,et al.  Does H.265 based peri and para-foveal quality flicker disrupt natural viewing patterns? , 2015, 2015 International Conference on Systems, Signals and Image Processing (IWSSIP).

[14]  Rhea T Eskew,et al.  Chromatic detection and discrimination in the periphery: a postreceptoral loss of color sensitivity. , 2003, Visual neuroscience.

[15]  Kai Zeng,et al.  Characterizing perceptual artifacts in compressed video streams , 2014, Electronic Imaging.

[16]  Patrick Le Callet,et al.  Role of HEVC coding artifacts on gaze prediction in interactive video streaming systems , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[17]  R. Hess,et al.  Temporal frequency filters in the human peripheral visual field , 1992, Vision Research.

[18]  K. Gegenfurtner,et al.  Color perception in the intermediate periphery of the visual field. , 2009, Journal of vision.

[19]  Stefan Winkler,et al.  Perceptual distortion metric for digital color video , 1999, Electronic Imaging.

[20]  Peng Wang,et al.  Region-classification-based rate control for flicker suppression of I-frames in HEVC , 2013, 2013 IEEE International Conference on Image Processing.

[21]  W Seiple,et al.  Duration Thresholds for Target Detection and Identification in the Peripheral Visual Field , 2001, Optometry and vision science : official publication of the American Academy of Optometry.

[22]  Kedarnath P Vilankar,et al.  Local masking in natural images: a database and analysis. , 2014, Journal of vision.

[23]  Marcus Barkowsky,et al.  Role of spatio-temporal distortions in the visual periphery in disrupting natural attention deployment , 2016, HVEI.

[24]  Junle Wang,et al.  Quantifying the relationship between visual salience and visual importance , 2010, Electronic Imaging.

[25]  Hans-Peter Seidel,et al.  Visible difference predicator for high dynamic range images , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[26]  Murat Kunt,et al.  Quality assessment of motion rendition in video coding , 1999, IEEE Trans. Circuits Syst. Video Technol..

[27]  P Gouras,et al.  Opponent‐colour cells in different layers of foveal striate cortex , 1974, The Journal of physiology.

[28]  Eero P. Simoncelli,et al.  Metamers of the ventral stream , 2011, Nature Neuroscience.

[29]  Matthias Bethge,et al.  Testing models of peripheral encoding using metamerism in an oddity paradigm. , 2016, Journal of vision.