Exploit camera metadata for enhancing interesting region detection and photo retrieval

Photographs taken by human beings differ from the images that taken by a lifeless device, such as a surveillance camera or a visual sensor on a robot, in that human being intentionally shoot photographs to express his/her feeling or photo-realistically record a scene. This creation process is accomplished by adjusting two factors: the setting of parameters on a camera and the position between the camera and the object which he or she is interested in. In this paper, this procedure is learned using the machine learning technique so that what the interest of the photographer is and what the core content of a photo wants to display can be reversely calculated. A photo retrieval system was built upon the category of interesting regions and the metadata is used for help. The research also explore the argument how local or global feature affects the performance of image retrieval. A novel stochastic segmentation algorithm called region restricted EM algorithm was applied in order to construct the interesting regions. Experimental evaluation on over 7000+ photos taken by 200+ different models of cameras with variety of interests has shown the robustness of our technique.

[1]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[2]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[3]  Sabine Süsstrunk,et al.  Salient Region Detection and Segmentation , 2008, ICVS.

[4]  Jiebo Luo,et al.  Bayesian fusion of camera metadata cues in semantic scene classification , 2004, CVPR 2004.

[5]  Frank Y. Shih,et al.  Automatic seeded region growing for color image segmentation , 2005, Image Vis. Comput..

[6]  Claudio Gutierrez,et al.  Survey of graph database models , 2008, CSUR.

[7]  Rachid Deriche,et al.  A Multiphase Level Set Based Segmentation Framework with Pose Invariant Shape Priors , 2006, ACCV.

[8]  Geoffrey E. Hinton,et al.  SMEM Algorithm for Mixture Models , 1998, Neural Computation.

[9]  Claudio M. Privitera,et al.  Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Carla E. Brodley,et al.  Local versus global features for content-based image retrieval , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[12]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[13]  Ana L. N. Fred,et al.  A New Cluster Isolation Criterion Based on Dissimilarity Increments , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Carlos Vázquez,et al.  Multiregion competition: A level set extension of region competition to multiple region image partitioning , 2006, Comput. Vis. Image Underst..

[16]  Laurent Itti,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Rapid Biologically-inspired Scene Classification Using Features Shared with Visual Attention , 2022 .

[17]  Alan L. Yuille,et al.  Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Jiebo Luo,et al.  A Bayesian network-based framework for semantic image understanding , 2005, Pattern Recognit..

[19]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[20]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[21]  Yong Man Ro,et al.  Semantic Home Photo Categorization , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Rachid Deriche,et al.  Unsupervised Segmentation Incorporating Colour, Texture, and Motion , 2003, CAIP.

[23]  John Wright,et al.  Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Thomas Brox,et al.  Level Set Based Image Segmentation with Multiple Regions , 2004, DAGM-Symposium.

[25]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[26]  Thomas C.M. Lee.  A Minimum Description Length-Based Image Segmentation Procedure, and its Comparison with a Cross-Validation-Based Segmentation Procedure , 2000 .

[27]  Z. Kato.  Bayesian color image segmentation using reversible jump Markov chain Monte Carlo , 1999 .

[28]  Cordelia Schmid,et al.  High-dimensional data clustering , 2006, Comput. Stat. Data Anal..

[29]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..

[31]  Zhihua Zhang,et al.  EM algorithms for Gaussian mixtures with split-and-merge operation , 2003, Pattern Recognition.

[32]  B. S. Manjunath,et al.  Unsupervised Segmentation of Color-Texture Regions in Images and Video , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  LuoJiebo,et al.  A Bayesian network-based framework for semantic image understanding , 2005 .

[34]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[35]  I. Watson,et al.  Ieee Workshop on Content-based Access of Image and Video Libraries Cbaivl-98, June '98 1 Image Retrieval Evaluation , 1998 .

[36]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.