Gaze-contingent video compression with targeted gaze containment performance

A delay compensation algorithm is presented for a gaze-contingent video compression system (GCS) with a robust targeted gaze containment (TGC) performance. The TGC parameter allows varying compression levels of a gaze-contingent video stream by controlling its perceptual quality. The delay compensation model is based on the Kalman filter framework that models the human visual system with eye position and velocity data. The model predicts future eye position and constructs a high-quality coded region of interest (ROI) designed to contain a targeted number of gaze samples while reducing perceptual quality in the periphery of that region. Several model parameterization schemes were tested with 21 subjects using a delay range of 0.02 to 2 s and a TGC of 60 to 90%. The results indicate that the model was able to achieve TGC levels with compression of 1.4 to 2.3 times for TGC=90% and compression of 1.8 to 2.5 for TGC=60%. The lowest compression values were recorded for high delays, while the highest compression values were reported during small delays.

[1]  Bruce H. McCormick,et al.  Preattentive considerations for gaze-contingent image processing , 1995, Electronic Imaging.

[2]  Oleg V. Komogortsev,et al.  Kalman Filtering in the Design of Eye-Gaze-Guided Computer Interfaces , 2007, HCI.

[3]  Bruce H. McCormick,et al.  Simple multiresolution approach for representing multiple regions of interest (ROIs) , 1995, Other Conferences.

[4]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[5]  Lester C. Loschky,et al.  How late can you update gaze-contingent multiresolutional displays without detection? , 2007, TOMCCAP.

[6]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[7]  Wilson S. Geisler,et al.  Implementation of a foveated image coding system for image bandwidth reduction , 1996, Electronic Imaging.

[8]  Laurent Itti,et al.  Causal saliency effects during natural vision , 2006, ETRA.

[9]  Lester C. Loschky,et al.  Gaze-Contingent Multiresolutional Displays: An Integrative Review , 2003, Hum. Factors.

[10]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[11]  Wilson S. Geisler,et al.  Real-time foveated multiresolution system for low-bandwidth video communication , 1998, Electronic Imaging.

[12]  Andrew T. Duchowski,et al.  Eye Tracking Methodology: Theory and Practice , 2003, Springer London.

[13]  Laurent Itti,et al.  Computational mechanisms for gaze direction in interactive visual environments , 2006, ETRA.

[14]  Andrew T. Duchowski,et al.  Hybrid image-/model-based gaze-contingent rendering , 2007, TAP.

[15]  Lester C. Loschky,et al.  User performance with gaze contingent multiresolutional displays , 2000, ETRA.

[16]  Oleg V. Komogortsev,et al.  Eye movement prediction by Kalman filter with integrated linear horizontal oculomotor plant mechanical model , 2008, ETRA.

[17]  Bernice E. Rogowitz,et al.  Human Vision and Electronic Imaging II , 1997 .

[18]  Markus Kohler,et al.  Using the Kalman Filter to track Human Interactive Motion - Modelling and Initialization of the Kalm , 1997 .

[19]  P. A. P. Moran,et al.  An introduction to probability theory , 1968 .

[20]  Andrew T. Duchowski.  Acuity-matching resolution degradation through wavelet coefficient scaling , 2000, IEEE Trans. Image Process..

[21]  N. Shimizu.  [Neurology of eye movements]. , 2000, Rinsho shinkeigaku = Clinical neurology.

[22]  Bruce H. McCormick,et al.  Gaze-contingent video resolution degradation , 1998, Electronic Imaging.

[23]  Andrew T. Duchowski,et al.  Gaze-Contingent Displays: A Review , 2004, Cyberpsychology Behav. Soc. Netw..

[24]  Arzu Çöltekin,et al.  Foveated gaze-contingent displays for peripheral LOD management, 3D visualization, and stereo imaging , 2007, TOMCCAP.

[25]  Ernst Niebur,et al.  Variable-Resolution Displays: A Theoretical, Practical, and Behavioral Evaluation , 2002, Hum. Factors.

[26]  Marios S. Pattichis,et al.  Foveated video compression with optimal rate control , 2001, IEEE Trans. Image Process..

[27]  Jordi Ribas-Corbera,et al.  As plain as the noise on your face: Adaptive video compression using face detection and visual eccentricity models , 2001, J. Electronic Imaging.

[28]  Oleg V. Komogortsev,et al.  Hybrid scheme for perceptual object window design with joint scene analysis and eye-gaze tracking for media encoding based on perceptual attention , 2006, J. Electronic Imaging.

[29]  Oleg V. Komogortsev,et al.  Predictive perceptual compression for real time video communication , 2004, MULTIMEDIA '04.

[30]  Oleg V. Komogortsev,et al.  Perceptual attention focus prediction for multiple viewers in case of multimedia perceptual compression with feedback delay , 2006, ETRA.

[31]  Guanrong Chen,et al.  Introduction to random signals and applied Kalman filtering, 2nd edn. Robert Grover Brown and Patrick Y. C. Hwang, Wiley, New York, 1992. ISBN 0‐471‐52573‐1, 512 pp., $62.95. , 1992 .

[32]  D. E. Irwin,et al.  Visual Memory Within and Across Fixations , 1992 .

[33]  Oleg V. Komogortsev,et al.  Eye movement prediction by oculomotor plant Kalman filter with brainstem control , 2009 .

[34]  Zhou Wang,et al.  Embedded foveation image coding , 2001, IEEE Trans. Image Process..

[35]  D. Robinson,et al.  The upper limit of human smooth pursuit velocity , 1985, Vision Research.

[36]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.

[37]  Oleg V. Komogortsev,et al.  Perceptual multimedia compression based on the predictive Kalman filter eye movement modeling , 2007, Electronic Imaging.

[38]  Wilson S. Geisler,et al.  Retinally reconstructed images (RRIs): digital images having a resolution match with the human eye , 1998, Electronic Imaging.

[39]  Oleg V. Komogortsev,et al.  Predictive real-time perceptual compression based on eye-gaze-position analysis , 2008, TOMCCAP.