Classification and annotation of digital photos using optical context data

Other than the pixel information, a digital photo of today has a host of other information regarding the photo shooting event. These information are captured by different sensors present on the camera and are stored as metadata. In this paper we exploit this meta information and derive useful semantics about the digital photo. We also compare our results with classical relevance models used for automatic photo annotation. We create a dataset of digital photos containing all information and report results on it. We also make the dataset available to the community for further experiments.

[1]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[3]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[4]  Nigel Shadbolt,et al.  Image annotation with Photocopain , 2006 .

[5]  A. P. deVries,et al.  Experimental evaluation of a generative probabilistic image retrieval model on 'easy' data , 2003 .

[6]  Dingxing Wang,et al.  Boosting image classification with LDA-based feature combination for digital photograph management , 2005, Pattern Recognit..

[7]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[8]  Andreas Girgensohn,et al.  Temporal event clustering for digital photo collections , 2003, ACM Multimedia.

[9]  Jiebo Luo,et al.  Photo classification by integrating image content and camera metadata , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[10]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  David A. Forsyth,et al.  Learning the semantics of words and pictures , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[12]  Yihong Gong.  Advancing content-based image retrieval by exploiting image color and region features , 1999, Multimedia Systems.

[13]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[14]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Thijs Westerveld,et al.  Experimental result analysis for a generative probabilistic image retrieval model , 2003, SIGIR.

[16]  Jiebo Luo,et al.  Bayesian fusion of camera metadata cues in semantic scene classification , 2004, CVPR 2004.

[17]  Adrian E. Raftery,et al.  How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis , 1998, Comput. J..

[18]  R. Manmatha,et al.  A Model for Learning the Semantics of Pictures , 2003, NIPS.

[19]  Ramesh Jain,et al.  Concept annotation and search space decrement of digital photos using optical context information , 2008, Electronic Imaging.

[20]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .