Foveated Image and Video Processing and Search

Abstract In this article, we present an overview of current techniques used for foveating images and video. We provide a brief introduction to relevant aspects of the human visual system and how they motivate the idea of foveated, or variable resolution, images. We explore the idea of foveation as a perceptually lossless compression scheme and how foveation can be integrated into modern video coding algorithms. We also explore foveation as an efficient processing scheme that is useful when visual resources, such as bandwidth and computation, are constrained. We discuss algorithms for performing visual tasks, such as search and detection, with foveated imaging systems. We examine human behavior when performing similar visual tasks to gain insight into how best to design algorithms for these tasks. We look at some of the more promising applications of foveated video, such as teleoperation, as well as open issues and problems, including fixation selection. We then conclude with what we believe are the future trends of this fascinating area of research.