Joint statistical analysis of images and keywords with applications in semantic image enhancement

With the advent of social image-sharing communities, millions of images with associated semantic tags are now available online for free and allow us to exploit this abundant data in new ways. We present a fast non-parametric statistical framework designed to analyze a large data corpus of images and semantic tag pairs and find correspondences between image characteristics and semantic concepts. We learn the relevance of different image characteristics for thousands of keywords from one million annotated images. We demonstrate the framework's effectiveness with three different examples of semantic image enhancement: we adapt the gray-level tone-mapping, emphasize semantically relevant colors, and perform a defocus magnification for an image based on its semantic context. The performance of our algorithms is validated with psychophysical experiments.

[1]  Yizhou Yu,et al.  Data-driven image color theme enhancement , 2010, ACM Trans. Graph..

[2]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[3]  Subhasis Chaudhuri,et al.  Recovery of relative depth from a single observation using an uncalibrated (real-aperture) camera , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Naila Murray,et al.  Towards automatic concept transfer , 2011, NPAR '11.

[5]  Dani Lischinski,et al.  Personalization of image enhancement , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Michael Schroeder,et al.  Usage of DSC meta tags in a general automatic image enhancement system , 2002, IS&T/SPIE Electronic Imaging.

[7]  Antonio Torralba,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 80 Million Tiny Images: a Large Dataset for Non-parametric Object and Scene Recognition , 2022 .

[8]  Vicente Ordonez,et al.  Im2Text: Describing Images Using 1 Million Captioned Photographs , 2011, NIPS.

[9]  Ishwar K. Sethi,et al.  Mining association rules between low-level image features and high-level concepts , 2001, SPIE Defense + Commercial Sensing.

[10]  Robert A. Hummel,et al.  Image Enhancement by Histogram transformation , 1975 .

[11]  Yizhou Yu,et al.  Example-based image color and tone style enhancement , 2011, ACM Trans. Graph..

[12]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[13]  Topi Mäenpää,et al.  The local binary pattern approach to texture analysis - extensions and applications , 2003 .

[14]  M. Degroot,et al.  Probability and Statistics , 2021, Examining an Operational Approach to Teaching Probability.

[15]  Mark J. Huiskes,et al.  The MIR flickr retrieval evaluation , 2008, MIR '08.

[16]  Thomas Deselaers,et al.  Visual and semantic similarity in ImageNet , 2011, CVPR 2011.

[17]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[18]  James Ze Wang,et al.  Automatic image semantic interpretation using social action and tagging data , 2010, Multimedia Tools and Applications.

[19]  Terence Sim,et al.  Defocus map estimation from a single image , 2011, Pattern Recognit..

[20]  Raimondo Schettini,et al.  Content Aware Image Enhancement , 2007, AI*IA.

[21]  Frédo Durand,et al.  Defocus Magnification , 2007, Comput. Graph. Forum.

[22]  Wei-Ying Ma,et al.  Exploring statistical correlations for image retrieval , 2006, Multimedia Systems.

[23]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[24]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  O. Sorkine,et al.  Color harmonization , 2006, SIGGRAPH 2006.

[26]  Vincent Boyer,et al.  Harmonic colorization using proportion contrast , 2010, AFRIGRAPH '10.

[27]  Didier Dacunha-Castelle,et al.  Probability and Statistics: Volume II , 1986 .

[28]  Walter L. Smith.  Probability and Statistics , 1959, Nature.

[29]  Clément Fredembach.  Saliency as compact regions for local image enhancement , 2011, Color Imaging Conference.

[30]  John D. Austin,et al.  Adaptive histogram equalization and its variations , 1987 .

[31]  Giovanni Ramponi,et al.  Image enhancement via adaptive unsharp masking , 2000, IEEE Trans. Image Process..