Large Image Collection Visualization Using Perception-Based Similarity with Color Features

This paper introduces the basic steps to build a similarity-based visualization tool for large image collections. We build the similarity metrics based on human perception. Psychophysical experiments have shown that human observers can recognize the gist of scenes within 100 milliseconds (ms) by comprehending the global properties of an image. Color also plays an important role in human rapid scene recognition. However, previous works often neglect color features. We propose new scene descriptors that preserve the information from coherent color regions, as well as the spatial layouts of scenes. Experiments show that our descriptors outperform existing state-of-the-art approaches. Given the similarity metrics, a hierarchical structure of an image collection can be built in a top-down manner. Representative images are chosen for image clusters and visualized using a force-directed graph.

[1]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[2]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Michele Covell,et al.  Google image swirl: a large-scale content-based image visualization system , 2012, WWW.

[4]  Ramin Zabih,et al.  Histogram refinement for content-based image retrieval , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[5]  David A. Forsyth,et al.  Learning the semantics of words and pictures , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  J. Henderson,et al.  The influence of color on the perception of scene gist. , 2008, Journal of experimental psychology. Human perception and performance.

[7]  Michelle R. Greene,et al.  Recognition of natural scenes from global properties: Seeing the forest without representing the trees , 2009, Cognitive Psychology.

[8]  John Krumm,et al.  Object recognition with color cooccurrence histograms , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[9]  Chong-Wah Ngo,et al.  Evaluating bag-of-visual-words representations in scene classification , 2007, MIR '07.

[10]  Michelle R. Greene,et al.  PSYCHOLOGICAL SCIENCE Research Article The Briefest of Glances The Time Course of Natural Scene Understanding , 2022 .

[11]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[12]  A. Oliva,et al.  From Blobs to Boundary Edges: Evidence for Time- and Spatial-Scale-Dependent Scene Recognition , 1994 .

[13]  Marcel Worring,et al.  Interactive access to large image collections using similarity-based visualization , 2008, J. Vis. Lang. Comput..

[14]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[15]  Huijun Gao,et al.  Feature Combination and the kNN Framework in Object Classification , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[17]  Krista A. Ehinger,et al.  SUN database: Large-scale scene recognition from abbey to zoo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Andrew Zisserman,et al.  Scene Classification Using a Hybrid Generative/Discriminative Approach , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Abel G. Oliva,et al.  Gist of a scene , 2005 .

[21]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  A. Oliva,et al.  Diagnostic Colors Mediate Scene Recognition , 2000, Cognitive Psychology.

[24]  Robert Tibshirani,et al.  Estimating the number of clusters in a data set via the gap statistic , 2000 .

[25]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[26]  Jianping Fan,et al.  Semantic Image Browser: Bridging Information Visualization with Automated Intelligent Image Analysis , 2006, 2006 IEEE Symposium On Visual Analytics Science And Technology.