Qualitative Approaches to Semantic Scene Modelling and Retrieval

This paper investigates the use of qualitative and spatially expressive semantic descriptions for image classification. In particular, it addresses the question of how a qualitative representation performs compared to a more quantitative one, using a semantic based symbolic approach. The approach is based on using different qualitative spatial representations applied to local semantic concepts such as grass, sky, water etc in a corpus of natural scenes images, to learn qualitative class descriptions and categorise them into one of six semantically meaningful classes such as coasts, forest etc. Three kinds of qualitative spatial relations, namely Allen’s relations [1] applied on the vertical axis of images, chord representation [9] applied on segmented semantically labelled image regions, and relative size of semantic concepts in each image have been investigated. A number of well-known supervised learning techniques have been applied to determine their usefulness. It is shown that a purely qualitative representation can result into better image description and perform at the similar level or even slightly better, compared to an existing largely quantitative representation [20]. Such qualitative and spatially expressive descriptions may therefore have utility in semantic querying and image retrieval systems.

[1]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[2]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[3]  Jiebo Luo,et al.  Improved scene classification using efficient low-level features and semantic cues , 2004, Pattern Recognit..

[4]  Sankar K. Pal,et al.  A review on image segmentation techniques , 1993, Pattern Recognit..

[5]  Fabian Mörchen,et al.  Discovering Temporal Knowledge in Multivariate Time Series , 2004, GfKl.

[6]  Aude Oliva,et al.  Global semantic classification of scenes using power spectrum templates , 1999 .

[7]  Hanno Walischewski.  Learning regions of interest in postal automation , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[8]  Anil K. Jain,et al.  Content-based hierarchical classification of vacation images , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[9]  Anil K. Jain,et al.  On image classification: city images vs. landscapes , 1998, Pattern Recognit..

[10]  Tomás Lozano-Pérez,et al.  A Framework for Multiple-Instance Learning , 1997, NIPS.

[11]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[12]  Amitabha Mukerjee,et al.  A Qualitative Model for Space , 1990, AAAI.

[13]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[14]  Oded Maron,et al.  Multiple-Instance Learning for Natural Scene Classification , 1998, ICML.

[15]  Rosalind W. Picard,et al.  Texture orientation for sorting photos "at a glance" , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[16]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[17]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[18]  Ian H. Witten,et al.  Generating Accurate Rule Sets Without Global Optimization , 1998, ICML.

[19]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[20]  Tom Minka,et al.  Vision texture for annotation , 1995, Multimedia Systems.

[21]  Bernt Schiele,et al.  A Semantic Typicality Measure for Natural Scene Categorization , 2004, DAGM-Symposium.

[22]  Anil K. Jain,et al.  Texture classification and segmentation using multiresolution simultaneous autoregressive models , 1992, Pattern Recognit..