Exploring the relationship between feature and perceptual visual spaces

The number and size of digital repositories containing visual information (images or videos) is increasing and thereby demanding appropriate ways to represent and search these information spaces. Their visualization often relies on reducing the dimensions of the information space to create a lower-dimensional feature space which, from the point-of-view of the end user, will be viewed and interpreted as a perceptual space. Critically for information visualization, the degree to which the feature and perceptual spaces correspond is still an open research question. In this paper we report the results of three studies which indicate that distance (or dissimilarity) matrices based on low-level visual features, in conjunction with various similarity measures commonly used in current CBIR systems, correlate with human similarity judgments. © 2008 Wiley Periodicals, Inc.

[1]  Pearl Pu,et al.  Opportunistic Search with Semantic Fisheye Views , 2004, WISE.

[2]  Hermann Ney,et al.  FIRE - Flexible Image Retrieval Engine: ImageCLEF 2004 Evaluation , 2004, CLEF.

[3]  Hemalata Iyer,et al.  Theories of cognition and image categorization: What category labels reveal about basic level theory , 2008 .

[4]  Hermann Ney,et al.  The CLEF 2005 Automatic Medical Image Annotation Task , 2006, International Journal of Computer Vision.

[5]  C.F.N. Cowan,et al.  Comparison of techniques for measuring cloud texture in remotely sensed satellite meteorological ima , 1989 .

[6]  Robert A. Wilson,et al.  Book Reviews: The MIT Encyclopedia of the Cognitive Sciences , 2000, CL.

[7]  Kerry Rodden,et al.  Evaluating a visualisation of image similarity as a tool for image browsing , 1999, Proceedings 1999 IEEE Symposium on Information Visualization (InfoVis'99).

[8]  Simone Santini,et al.  In search of information in visual media , 1997, CACM.

[9]  Hermann Ney,et al.  Features for image retrieval: an experimental comparison , 2008, Information Retrieval.

[10]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[11]  Christos Faloutsos,et al.  Efficient and effective Querying by Image Content , 1994, Journal of Intelligent Information Systems.

[12]  Robert R. Korfhage,et al.  A distance and angle similarity measure method , 1999 .

[13]  Carlo Tomasi,et al.  Perceptual metrics for image database navigation , 1999 .

[14]  Thierry Pun,et al.  Assessing agreement between human and machine clusterings of image databases , 1998, Pattern Recognit..

[15]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[16]  Corinne Jörgensen,et al.  Image querying by image professionals: Research Articles , 2005 .

[17]  Anthony Peter Macmillan Coxon,et al.  Sorting Data: Collection and Analysis , 1999 .

[18]  Henning Müller,et al.  Learning Feature Weights from User Behavior in Content-Based Image Retrieval , 2000, MDM/KDD.

[19]  J. M. Zachary,et al.  An Information Theoretic Approach to Content Based Image Retrieval. , 2000 .

[20]  Shih-Fu Chang,et al.  Tools and techniques for color image retrieval , 1996, Electronic Imaging.

[21]  B. Manly.  Multivariate Statistical Methods : A Primer , 1986 .

[22]  Amos Tversky,et al.  Studies of similarity , 1978 .

[23]  Tom E. Bishop,et al.  Blind Image Restoration Using a Block-Stationary Signal Model , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[24]  Hermann Ney,et al.  Sparse Patch-Histograms for Object Classification in Cluttered Images , 2006, DAGM-Symposium.

[25]  Peter Ingwersen,et al.  Information Retrieval Interaction , 1992 .

[26]  S. S. Stevens,et al.  Psychophysics: Introduction to Its Perceptual, Neural and Social Prospects , 1975 .

[27]  David A. Forsyth,et al.  Learning the semantics of words and pictures , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[28]  Hsinchun Chen,et al.  Validating a Geographical Image Retrieval System. , 2000 .

[29]  Corinne Jörgensen,et al.  Attributes of Images in Describing Tasks , 1998, Inf. Process. Manag..

[30]  Brian C. O'Connor,et al.  Modelling what users see when they look at images: a cognitive viewpoint , 2002, J. Documentation.

[31]  A. Tversky.  Features of Similarity , 1977 .

[32]  James Ze Wang,et al.  Content-based image retrieval: approaches and trends of the new age , 2005, MIR '05.

[33]  Yves Van de Peer,et al.  zt: A Sofware Tool for Simple and Partial Mantel Tests , 2002 .

[34]  Dov Dori,et al.  Cognitive image retrieval , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[35]  Hermann Ney,et al.  Deformation Models for Image Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  June Abbas,et al.  User Reactions as Access Mechanism: An Exploration Based on Captions for Images , 1999, J. Am. Soc. Inf. Sci..

[37]  Benjamin B. Bederson,et al.  Does zooming improve image browsing? , 1999, DL '99.

[38]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[39]  Jian Qin.  Semantic similarities between a keyword database and a controlled vocabulary database: an investigation in the antibiotic resistance literature , 2000 .

[40]  Naphtali Rishe,et al.  Content-based image retrieval , 1995, Multimedia Tools and Applications.

[41]  Shih-Fu Chang,et al.  A conceptual framework and empirical research for classifying visual descriptors , 2001 .

[42]  Edward Y. Chang,et al.  Discovery of a perceptual distance function for measuring image similarity , 2003, Multimedia Systems.

[43]  S. Sitharama Iyengar,et al.  Content based image retrieval and information theroy: a general approach , 2001 .

[44]  Jin Zhang,et al.  Developing a new similarity measure from two different perspectives , 2001, Inf. Process. Manag..

[45]  A. Kerne,et al.  Collection understanding [visualization tools in information retrieval] , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..

[46]  Gyuri Dorkó,et al.  Selection of Discriminative Regions and Local Descriptors for Generic Object Class Recognition. (Sélection de régions significatives locales et de leurs descripteurs pour la reconnaissance de classes génériques d'objects) , 2006 .

[47]  Hans Burkhardt,et al.  SIMBA - Search IMages By Appearance , 2001, DAGM-Symposium.

[48]  Kerry Rodden,et al.  Does organisation by similarity assist image browsing? , 2001, CHI.

[49]  Robert R. Korfhage,et al.  DARE: distance and angle retrieval environment: a tale of the two measures , 1999 .

[50]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[51]  Chaomei Chen,et al.  Content-based image visualization , 2000, 2000 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics.

[52]  Robert D. Melara,et al.  The concept of perceptual similarity : from psychophysics to cognitive psychology , 1992 .

[53]  Alberto Del Bimbo,et al.  Visual information retrieval , 1999 .

[54]  Brian C. O'Connor,et al.  What do users see? Exploring the cognitive nature of functional image retrieval , 2005, ASIST.

[55]  Mari Laine-Hernandez,et al.  Image semantics in the description and categorization of journalistic photographs , 2007, ASIST.

[56]  Joachim M. Buhmann,et al.  Empirical evaluation of dissimilarity measures for color and texture , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[57]  Xing Xie,et al.  Effective browsing of web image search results , 2004, MIR '04.

[58]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  Hermann Ney,et al.  Discriminative training for object recognition using image patches , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[60]  Gregory B. Newby,et al.  Cognitive space and information space , 2001, J. Assoc. Inf. Sci. Technol..

[61]  Corinne Jörgensen,et al.  Image querying by image professionals , 2005, J. Assoc. Inf. Sci. Technol..

[62]  Ishwar K. Sethi,et al.  eID: a system for exploration of image databases , 2003, Inf. Process. Manag..

[63]  Dirk Neumann,et al.  Image retrieval and perceptual similarity , 2006, TAP.

[64]  Esther Dyson,et al.  Education and jobs in the digital world , 1997, CACM.

[65]  Hideyuki Tamura,et al.  Textural Features Corresponding to Visual Perception , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[66]  Roger B. Wyatt,et al.  Photo Provocations: Thinking In, With, and About Photographs , 2004 .

[67]  E. Rosch,et al.  Cognition and Categorization , 1980 .

[68]  Wei-Ying Ma,et al.  Hierarchical clustering of WWW image search results using visual, textual and link information , 2004, MULTIMEDIA '04.

[69]  Peter Ingwersen,et al.  Cognitive Perspectives of Information Retrieval Interaction: Elements of a Cognitive IR Theory , 1996, J. Documentation.

[70]  Heung-Kyu Lee,et al.  Re-ranking algorithm using post-retrieval clustering for content-based image retrieval , 2005, Inf. Process. Manag..

[71]  N. Mantel.  The detection of disease clustering and a generalized regression approach. , 1967, Cancer research.

[72]  Simone Santini,et al.  Similarity Measures , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[73]  Daniel Algom,et al.  Psychophysical approaches to cognition. , 1992 .

[74]  Charles A. Bouman,et al.  Perceptual image similarity experiments , 1998, Electronic Imaging.

[75]  Nuno Vasconcelos,et al.  A unifying view of image similarity , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[76]  Kerry Rodden,et al.  A comparison of measures for visualising image similarity , 2000 .

[77]  Peter G. B. Enser,et al.  Visual image retrieval: seeking the alliance of concept-based and content-based paradigms , 2000, J. Inf. Sci..

[78]  Marcus Jerome Pickering,et al.  A comparative study of evidence combination strategies , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[79]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[80]  Abby Goodrum,et al.  Multidimensional scaling of video surrogates , 2001, J. Assoc. Inf. Sci. Technol..