Describing images via linguistic features and hierarchical segmentation

In this paper we introduce a preliminary proposal for linguistic description of images. The approach is based on i) a hierarchical fuzzy segmentation of the image, ii) a collection of linguistic features describing each region, and iii) fuzzy spatial relations and locations. The procedure is independent from the way these elements have been obtained, and provides a description with the characteristics of a summary, i.e., a brief and accurate description of the whole image. As another characteristic of summaries, the method can be guided in the description by the user's preferences and interest. Remarkably, we are able to provide a description containing sentences about disjoint regions appearing in different levels of detail.

[1]  Junji Maeda,et al.  Rough and accurate segmentation of natural color images using fuzzy region-growing algorithm , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[2]  Marek R. Ogiela,et al.  Graph image language techniques supporting radiological, hand image interpretations , 2006, Comput. Vis. Image Underst..

[3]  Soumitra Dutta,et al.  Approximate spatial reasoning: Integrating qualitative and quantitative constraints , 1991, Int. J. Approx. Reason..

[4]  Terry Regier,et al.  The Human Semantic Potential: Spatial Language and Constrained Connectionism , 1996 .

[5]  Franck Luthon,et al.  Nonlinear color space and spatiotemporal MRF for hierarchical segmentation of face features in video , 2004, IEEE Transactions on Image Processing.

[6]  Luc Florack,et al.  The hierarchical structure of images , 2003, IEEE Trans. Image Process..

[7]  Y. Haxhimusa,et al.  Hierarchical Image Partitioning with Dual Graph Contraction 1 , 2003 .

[8]  Dietmar Saupe,et al.  Optimal hierarchical partitions for fractal image compression , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[9]  Sokratis Makrogiannis,et al.  A region dissimilarity relation that combines feature-space and spatial information for color image segmentation , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Pilar Sobrevilla Frisón,et al.  Fuzzy sets in computer vision: an overview , 2003 .

[11]  Martine De Cock,et al.  Spatial reasoning in a fuzzy region connection calculus , 2009, Artif. Intell..

[12]  Zhuowen Tu,et al.  Parsing Images into Regions, Curves, and Curve Groups , 2006, International Journal of Computer Vision.

[13]  Witold Pedrycz,et al.  Unsupervised hierarchical image segmentation with level set and additive operator splitting , 2005, Pattern Recognit. Lett..

[14]  Yorick Wilks,et al.  Image-Language Multimodal Corpora: Needs, Lacunae and an AI Synergy for Annotation , 2004, LREC.

[15]  Alfred Mertins,et al.  Scalable multiresolution color image segmentation with smoothness constraint , 2005, 2005 IEEE International Conference on Electro Information Technology.

[16]  Alireza Bab-Hadiashar,et al.  Range segmentation of large building exteriors: A hierarchical robust approach , 2010, Comput. Vis. Image Underst..

[17]  Daniel Sánchez,et al.  Fuzzy cardinality based evaluation of quantified sentences , 2000, Int. J. Approx. Reason..

[18]  Arnaldo de Albuquerque Araújo,et al.  Segmentation into fuzzy regions using topographic distance , 2001, Proceedings XIV Brazilian Symposium on Computer Graphics and Image Processing.

[19]  James M. Keller,et al.  Mapping natural language to imagery: Placing objects intelligently , 2009, 2009 IEEE International Conference on Fuzzy Systems.

[20]  Jan Verbesselt,et al.  Hierarchical image segmentation based on similarity of NDVI time series , 2008 .

[21]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  A. OsbornR.  The ISCC-NBS Method of Designating Colors and A Dictionary of Color Names , 1956 .

[23]  David Pérez,et al.  Aprendizaje de la sintaxis para la descripción de escenas compuestas por figuras geométricas , 2010 .

[24]  P. Kay,et al.  Basic Color Terms: Their Universality and Evolution , 1973 .

[25]  Anca L. Ralescu,et al.  Spatial organization in 2D segmented images: representation and recognition of primitive spatial relations , 1994, CVPR 1994.

[26]  Monique Thonnat,et al.  Ontology based complex object recognition , 2008, Image Vis. Comput..

[27]  Raymond J. Mooney,et al.  Learning to sportscast: a test of grounded language acquisition , 2008, ICML '08.

[28]  James C. Tilton,et al.  Image segmentation by region growing and spectral clustering with a natural convergence criterion , 1998, IGARSS '98. Sensing and Managing the Environment. 1998 IEEE International Geoscience and Remote Sensing. Symposium Proceedings. (Cat. No.98CH36174).

[29]  Max J. Egenhofer,et al.  Similarity of Cardinal Directions , 2001, SSTD.

[30]  María Amparo Vila Miranda,et al.  Segmenting Colour Images on the basis of a Fuzzy Hierarchical Approach , 2003 .

[31]  Bernd Neumann,et al.  On scene interpretation with description logics , 2006, Image Vis. Comput..

[32]  Nikolaos G. Bourbakis,et al.  A fuzzy region growing approach for segmentation of color images , 1997, Pattern Recognit..

[33]  Isabelle Bloch,et al.  Fuzzy spatial relation ontology for image interpretation , 2008, Fuzzy Sets Syst..

[34]  Daniel Sánchez,et al.  Linguistic Summary-Based Query Answering on Data Cubes with Time Dimension , 2009, FQAS.

[35]  Daniel Sánchez,et al.  Region-based fit of color homogeneity measures for fuzzy image segmentation , 2007, Fuzzy Sets Syst..

[36]  Daniel Sánchez,et al.  A new approach for defining a fuzzy color space , 2010, International Conference on Fuzzy Systems.

[37]  Anthony G. Cohn,et al.  Qualitative Spatial Representation and Reasoning: An Overview , 2001, Fundam. Informaticae.

[38]  Benjamin Kuipers,et al.  Navigation and Mapping in Large Scale Space , 1988, AI Mag..

[39]  Daniel Sánchez,et al.  Using FORDBMS for the linguistic description of images , 2010, International Conference on Fuzzy Systems.

[40]  K. L. Kelly.  The ISCC-NBS method of designating colors and a dictionary of color names , 1955 .

[41]  Deb K. Roy,et al.  Learning visually grounded words and syntax for a scene description task , 2002, Comput. Speech Lang..

[42]  Olga Pons,et al.  THE GENERALIZED SELECTION: AN ALTERNATIVE WAY FOR THE QUOTIENT OPERATIONS IN FUZZY RELATIONAL DATABASES , 1995 .

[43]  Daniel Sánchez,et al.  A similarity measure between fuzzy regions to obtain a hierarchy of fuzzy image segmentations , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[44]  Raymond J. Mooney,et al.  Learning to Connect Language and Perception , 2008, AAAI.

[45]  Stephan Winter.  Topological Relations in Hierarchical Partitions , 1999, COSIT.

[46]  H. D. Cheng,et al.  Fuzzy homogeneity and scale-space approach to color image segmentation , 2003, Pattern Recognit..

[47]  Laurent Wendling,et al.  A New Way to Represent the Relative Position between Areal Objects , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  P. Matsakis,et al.  The use of force histograms for affine-invariant relative position description , 2004 .

[49]  John Freeman,et al.  The modelling of spatial relations , 1975 .

[50]  Umberto Straccia,et al.  Towards spatial reasoning in fuzzy description logics , 2009, 2009 IEEE International Conference on Fuzzy Systems.