Finding Interesting Images in Albums using Attention

Commercial systems such as Flickr display interesting photos from their collection as an interaction mechanism for sampling the collection. It purely relies on social activity analysis for determining the notion of interestingness. We propose an alternative technique based on content analysis for finding interesting photos in a collection. We use a combination of visual attention models and an interactive feedback mechanism to compute interestingness. A differentiating feature of our approach is the ability to customize the set of interesting photos depending upon the individual interest. Also, we incorporate non-identical duplicate detection as a mechanism to strengthen the surprise factor among the potentially interesting set of candidate photos. We have implemented the system and conducted a user study whose results are promising. This proposed work presents a variant on query by example integrating user relevance feedback to choose “interesting” photos.

[1]  Robert A. Marino,et al.  Characterizing surprise in humans and monkeys , 2010 .

[2]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[3]  Ravi Kumar,et al.  Visualizing tags over time , 2006, TWEB.

[4]  Matthew W. Hoffman,et al.  A probabilistic model of gaze imitation and shared attention , 2006, Neural Networks.

[5]  D.R. Edgington,et al.  Detecting, Tracking and Classifying Animals in Underwater Video , 2005, OCEANS 2006.

[6]  Dirk Walther,et al.  Interactions of visual attention and object recognition : computational modeling, algorithms, and psychophysics. , 2006 .

[7]  Susan T. Dumais,et al.  Searching to eliminate personal information management , 2006, CACM.

[8]  Tien-Tsin Wong,et al.  Real-time surveillance video display with salience , 2005, VSSN@MM.

[9]  D. Deng,et al.  How interesting is this? Finding interest hotspots and ranking images using an MPEG-7 visual attention model , 2005 .

[10]  Luc Van Gool,et al.  Modeling scenes with local descriptors and latent aspects , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[11]  Michal Irani,et al.  Detecting Irregularities in Images and in Video , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[12]  Pietro Perona,et al.  Selective visual attention enables learning and recognition of multiple objects in cluttered scenes , 2005, Comput. Vis. Image Underst..

[13]  Lie Lu,et al.  A generic framework of user attention model and its application in video summarization , 2005, IEEE Trans. Multim..

[14]  Pierre Baldi,et al.  A principled approach to detecting surprising events in video , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Jeremy M. Wolfe,et al.  41.1: Invited Paper: How Might the Rules that Govern Visual Search Constrain the Design of Visual Displays? , 2005 .

[16]  L. Itti,et al.  Modeling the influence of task on attention , 2005, Vision Research.

[17]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[18]  Christof Koch,et al.  Detection and tracking of objects in underwater video , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[19]  Xing Xie,et al.  Attention model based progressive image transmission , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[20]  J. Wolfe,et al.  What attributes guide the deployment of visual attention and how do they do it? , 2004, Nature Reviews Neuroscience.

[21]  Jacek Gwizdka,et al.  Personal information management , 2004, CHI EA '04.

[22]  R. Passingham,et al.  Attention to Intention , 2004, Science.

[23]  Stefan B. Williams,et al.  Reduced SIFT Features For Image Retrieval And Indoor Localisation , 2004 .

[24]  C. Schmid,et al.  A performance evaluation of local descriptors , 2005, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[25]  Shih-Fu Chang,et al.  Detection of non-identical duplicate consumer photographs , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[26]  Alyson G. Wilson,et al.  Detection of Visual Events in Underwater Video Using a Neuromorphic Saliency-based Attention System , 2003 .

[27]  HongJiang Zhang,et al.  Contrast-based image attention analysis by using fuzzy growing , 2003, MULTIMEDIA '03.

[28]  Mohan S. Kankanhalli,et al.  Experience based sampling technique for multimedia analysis , 2003, MULTIMEDIA '03.

[29]  Wei-Ying Ma,et al.  Improving pseudo-relevance feedback in web information retrieval using web page segmentation , 2003, WWW '03.

[30]  Thomas S. Huang,et al.  Relevance feedback in image retrieval: A comprehensive review , 2003, Multimedia Systems.

[31]  Jim Gray,et al.  What next?: A dozen information-technology research goals , 1999, JACM.

[32]  HongJiang Zhang,et al.  A user attention model for video summarization , 2002, MULTIMEDIA '02.

[33]  Laurent Itti,et al.  A Goal Oriented Attention Guidance Model , 2002, Biologically Motivated Computer Vision.

[34]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[35]  J. Luo,et al.  On measuring low-level saliency in photographic images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[36]  G. Taylor,et al.  The natural history of the mind , 1979 .

[37]  Classics in the History of Psychology The Myth of Mental Illness , 2022 .