Saliency-based multifoveated MPEG compression

Most current foveation strategies are limited to foveating sequences based on a direct measurement or an implicit assumption of the gaze direction. Such approaches often fail in unconstrained environments or when necessary equipment is absent. Alternatively, a computational model of visual attention may be used to predict visually salient locations. We describe such a neurobiological model of attention and its specific application to foveated video compression. The algorithm is demonstrated to be successful in foveating to regions of human interest in a variety of video segments, including synthetic as well as natural scenes, and also gives good compression ratios.

[1]  Gunilla Borgefors,et al.  Distance transformations in digital images , 1986, Comput. Vis. Graph. Image Process..

[2]  Alex Pentland,et al.  Probabilistic visual learning for object detection , 1995, Proceedings of IEEE International Conference on Computer Vision.

[3]  C. Chabris,et al.  Gorillas in Our Midst: Sustained Inattentional Blindness for Dynamic Events , 1999, Perception.

[4]  M. Posner,et al.  Components of visual orienting , 1984 .

[5]  J. Rovamo,et al.  An estimation and application of the human cortical magnification factor , 2004, Experimental Brain Research.

[6]  F. Campbell,et al.  Optical quality of the human eye , 1966, The Journal of physiology.

[7]  Wim H. Hesselink,et al.  A General Algorithm for Computing Distance Transforms in Linear Time , 2000, ISMM.

[8]  Gunilla Borgefors.  Another comment on "a note on 'distance transformations in digital images'" , 1991, CVGIP Image Underst..

[9]  C D Frith,et al.  Space-based and object-based visual attention: shared and specific neural domains. , 1997, Brain : a journal of neurology.

[10]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[11]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[12]  Anthony A. Wasilewski,et al.  Robust, sensor-independent target detection and recognition based on computational models of human vision , 1998 .

[13]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[14]  Z W Pylyshyn,et al.  Tracking multiple independent targets: evidence for a parallel tracking mechanism. , 1988, Spatial vision.

[15]  Laurent Itti,et al.  A Goal Oriented Attention Guidance Model , 2002, Biologically Motivated Computer Vision.

[16]  Ronald A. Rensink,et al.  On the Failure to Detect Changes in Scenes Across Brief Interruptions , 2000 .

[17]  N. P. Bichot,et al.  Visual selection mediated by location: Feature-based selection of noncontiguous locations , 1999, Perception & psychophysics.

[18]  Claudio M. Privitera,et al.  Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Ronald A. Rensink.  Change detection. , 2002, Annual review of psychology.

[20]  U. Neisser,et al.  Selective looking: Attending to visually specified events , 1975, Cognitive Psychology.