Spatio-temporal analysis of eye fixations data in images

Computer vision algorithms such as image compression, image segmentation, context aware image resizing, image quality assessment and target detection, are designed with the aim of replicating our visual system. Human eye fixations recorded by using an eye tracker are typically used as a criterion for the optimisation of such algorithms. In this paper, we propose a method to analyze the spatio-temporal nature of fixations data for different observers. By studying the correlation matrix constructed based on the fixations data of different observers viewing the same image, it was found that 21 percent of the data can be accounted by one eigenvector. A visual inspection of this vector, shows that it represents the time sequence and locations of the objects in the image that observers deem as salient. The eigenvectors are used as a benchmark for evaluating the spatio-temporal performance of Itti's classic visual saliency model. Based on the results obtained from a comprehensive publicly available dataset, we show that the proposed method can be used as ground truth for evaluating the spatio-temporal performance of saliency models. Furthermore, it can be used to provide salient locations and their time sequence for a real-time image compression application with promising results.