Image segmentation based on the integration of colour-texture descriptors - A review

The adaptive integration of the colour and texture attributes in the development of complex image descriptors is one of the most investigated topics of research in computer vision. The substantial interest shown by the research community in colour-texture-based segmentation is mainly motivated by two factors. The first is related to the observation that the imaged objects are often described at perceptual level by distinctive colour and texture characteristics, while the second is motivated by the large spectrum of possible applications that can be addressed by the colour-texture integration in the segmentation process. Over the past three decades a substantial number of techniques in the field of colour-texture segmentation have been reported and it is the aim of this article to thoroughly evaluate and categorise the most relevant algorithms with respect to the modality behind the integration of these two fundamental image attributes. In this paper we also provide a detailed discussion about data collections, evaluation metrics and we review the performance attained by state of the art implementations. We conclude with a discussion that samples our views on the field of colour-texture image segmentation and this is complemented with an examination of the potential future directions of research.

[1]  Mark Q. Shaw,et al.  Automatic Image Segmentation by Dynamic Region Growth and Multiresolution Merging , 2009, IEEE Transactions on Image Processing.

[2]  Jim Kalett.  People and crowds, a photographic album for artists and designers , 1978 .

[3]  Aurélio J. C. Campilho,et al.  Performance Evaluation of Image Segmentation , 2006, ICIAR.

[4]  Nikos Komodakis,et al.  Bayesian Region Growing and MRF-based Minimization for Texture and Colour Segmentation , 2007, Eighth International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS '07).

[5]  Spyros Liapis,et al.  Color and texture image retrieval using chromaticity histograms and wavelet frames , 2004, IEEE Transactions on Multimedia.

[6]  Fumio Kishino,et al.  Color/texture analysis and synthesis for model-based human image coding , 1991, Other Conferences.

[7]  Rachid Deriche,et al.  Texture and color segmentation based on the combined use of the structure tensor and the image components , 2008, Signal Process..

[8]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[9]  Michal Strzelecki,et al.  Texture Analysis Methods - A Review , 1998 .

[10]  Roberto Manduchi,et al.  Bayesian fusion of color and texture segmentations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  C. H. Chen,et al.  Handbook of Pattern Recognition and Computer Vision , 1993 .

[12]  Glenn Healey,et al.  Segmenting images using normalized color , 1992, IEEE Trans. Syst. Man Cybern..

[13]  Y. J. Zhang,et al.  A survey on evaluation methods for image segmentation , 1996, Pattern Recognit..

[14]  Jesmin F. Khan,et al.  A customized Gabor filter for unsupervised color image segmentation , 2009, Image Vis. Comput..

[15]  Michael H. Goldbaum,et al.  An Image Processing System For Automatic Retina Diagnosis , 1988, Photonics West - Lasers and Applications in Science and Engineering.

[16]  Zoltan Kato,et al.  A Markov random field image segmentation model for color textured images , 2006, Image Vis. Comput..

[17]  B. S. Manjunath,et al.  Unsupervised Segmentation of Color-Texture Regions in Images and Video , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[19]  Zhou Yue,et al.  Color-texture segmentation using JSEG based on Gaussian mixture modeling , 2006 .

[20]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Paul F. Whelan,et al.  Unsupervised Image Segmentation based on the Multi-resolution Integration of Adaptive Local Texture Descriptors , 2010, VISAPP.

[22]  David B. Cooper,et al.  Bayesian Clustering for Unsupervised Estimation of Surface and Texture Models , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Giuseppe Scarpa,et al.  Unsupervised Texture Segmentation by Spectral-Spatial-Independent Clustering , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[24]  Matti Pietikäinen,et al.  Outex - new framework for empirical evaluation of texture analysis algorithms , 2002, Object recognition supported by user interaction for service robots.

[25]  J. Kittler,et al.  Colour texture analysis using colour histogram , 1994 .

[26]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Alok Gupta,et al.  Color and texture fusion: application to aerial image segmentation and GIS updating , 2000, Image Vis. Comput..

[28]  Alok Gupta,et al.  Color and texture fusion: application to aerial image segmentation and GIS updating , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[29]  Jie Yang,et al.  Unsupervised color-texture segmentation based on soft criterion with adaptive mean-shift clustering , 2006, Pattern Recognit. Lett..

[30]  Jing Li Wang,et al.  Color image segmentation: advances and prospects , 2001, Pattern Recognit..

[31]  Matti Pietikäinen,et al.  Unsupervised Texture Segmentation Using Feature Distributions , 1997, ICIAP.

[32]  Dirk R. Padfield,et al.  Color and texture based segmentation of molecular pathology images usING HSOMS , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[33]  Charles A. Bouman,et al.  A multiscale random field model for Bayesian image segmentation , 1994, IEEE Trans. Image Process..

[34]  Martial Hebert,et al.  A Measure for Objective Evaluation of Image Segmentation Algorithms , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[35]  C. P. Lim,et al.  Fuzzy clustering of color and texture features for image segmentation: A study on satellite image retrieval , 2006, J. Intell. Fuzzy Syst..

[36]  Max Mignotte,et al.  A hierarchical graph-based markovian clustering approach for the unsupervised segmentation of textured color images , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[37]  Constantino Carlos Reyes-Aldasoro,et al.  The Bhattacharyya space for feature selection and its application to texture segmentation , 2006, Pattern Recognit..

[38]  Zoltan Kato,et al.  Multicue MRF image segmentation: combining texture and color features , 2002, Object recognition supported by user interaction for service robots.

[39]  Max Mignotte,et al.  A Label Field Fusion Bayesian Model and Its Penalized Maximum Rand Estimator for Image Segmentation , 2010, IEEE Transactions on Image Processing.

[40]  Tetsuya Takiguchi,et al.  Graph Cuts Segmentation by Using Local Texture Features of Multiresolution Analysis , 2009, IEICE Trans. Inf. Syst..

[41]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Martial Hebert,et al.  Toward Objective Evaluation of Image Segmentation Algorithms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Xianghua Xie,et al.  MAC: Magnetostatic Active Contour Model , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  David A. Clausi,et al.  Unsupervised image segmentation using a simple MRF model with a new implementation scheme , 2004, ICPR 2004.

[45]  Hannu Kauppinen,et al.  COLOR AND TEXTURE BASED WOOD INSPECTION WITH NON-SUPERVISED CLUSTERING , 2001 .

[46]  Anil K. Jain,et al.  Unsupervised texture segmentation using Gabor filters , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[47]  Anil K. Jain,et al.  Texture Analysis , 2018, Handbook of Image Processing and Computer Vision.

[48]  Tieniu Tan,et al.  Brief review of invariant texture analysis methods , 2002, Pattern Recognit..

[49]  Laura Caponetti,et al.  Color Texture Segmentation with Local Fuzzy Patterns and Spatially Constrained Fuzzy C-Means , 2007, WILF.

[50]  A P Dhawan,et al.  Segmentation of images of skin lesions using color and texture information of surface pigmentation. , 1992, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[51]  Chaur-Chin Chen,et al.  Color images' segmentation using scale space filter and markov random field , 1992, Pattern Recognit..

[52]  Djemel Ziou,et al.  Globally adaptive region information for automatic color-texture image segmentation , 2007, Pattern Recognit. Lett..

[53]  Sameer Singh,et al.  Minerva scene analysis benchmark , 2001, The Seventh Australian and New Zealand Intelligent Information Systems Conference, 2001.

[54]  Theo Gevers.  Image segmentation and similarity of color-texture objects , 2002, IEEE Trans. Multim..

[55]  Shu-Yuan Chen,et al.  Color texture segmentation using feature distributions , 2002, Pattern Recognit. Lett..

[56]  Phil Brodatz,et al.  Textures: A Photographic Album for Artists and Designers , 1966 .

[57]  Yuchou Chang,et al.  Color-texture image segmentation by integrating directional operators into JSEG method , 2006, Pattern Recognit. Lett..

[58]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.

[59]  DericheRachid,et al.  A Review of Statistical Approaches to Level Set Segmentation , 2007 .

[60]  Amit Jain,et al.  A multiscale representation including opponent color features for texture recognition , 1998, IEEE Trans. Image Process..

[61]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[62]  Xavier Lladó,et al.  Colour Texture Segmentation by Region-Boundary Cooperation , 2004, ECCV.

[63]  Hui Zhang,et al.  Image segmentation evaluation: A survey of unsupervised methods , 2008, Comput. Vis. Image Underst..

[64]  Allen Y. Yang,et al.  Unsupervised segmentation of natural images via lossy data compression , 2008, Comput. Vis. Image Underst..

[65]  Rachid Deriche,et al.  A Review of Statistical Approaches to Level Set Segmentation: Integrating Color, Texture, Motion and Shape , 2007, International Journal of Computer Vision.

[66]  Yao Chen,et al.  Address block location using color and texture analysis , 1994 .

[67]  P. Whelan,et al.  Color image segmentation using a self-initializing EM algorithm , 2006 .

[68]  Michal Haindl,et al.  Texture segmentation benchmark , 2008, 2008 19th International Conference on Pattern Recognition.

[69]  Ki-Sang Hong,et al.  Color-texture segmentation using unsupervised graph cuts , 2009, Pattern Recognit..

[70]  Fernando López García.  Real-time surface grading of ceramic tiles , 2011 .

[71]  Catherine Garbay,et al.  Image Structure Representation and Processing: A Discussion of Some Segmentation Methods in Cytology , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[72]  Jie Yang,et al.  Unsupervised color-texture image segmentation , 2008 .

[73]  Thrasyvoulos N. Pappas,et al.  An Adaptive Clustering Algorithm For Image Segmentation , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[74]  Begoña Acha,et al.  Color-texture image segmentation based on multistep region growing , 2006 .

[75]  Martial Hebert,et al.  Measures of Similarity , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[76]  Trygve Randen,et al.  Filtering for Texture Classification: A Comparative Study , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[77]  Max Mignotte,et al.  Segmentation by Fusion of Histogram-Based $K$-Means Clusters in Different Color Spaces , 2008, IEEE Transactions on Image Processing.

[78]  Anil K. Jain,et al.  Unsupervised texture segmentation using Gabor filters , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[79]  Hans M. Aus,et al.  Combined local color and texture analysis of stained cells , 1986, Comput. Vis. Graph. Image Process..

[80]  Ioannis Pitas,et al.  Color Texture Segmentation Based on the Modal Energy of Deformable Surfaces , 2009, IEEE Transactions on Image Processing.

[81]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[82]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[83]  Wenbing Tao,et al.  Image Segmentation Based on GrabCut Framework Integrating Multiscale Nonlinear Structure Tensor , 2009, IEEE Transactions on Image Processing.

[84]  Majid Mirmehdi,et al.  Segmentation of Colour Textures , 2000 .

[85]  Paul F. Whelan,et al.  CTex-An Adaptive Unsupervised Segmentation Algorithm based on Colour-Texture Coherence , 2022 .

[86]  Anil K. Jain,et al.  Markov Random Field Texture Models , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[87]  Majid Mirmehdi,et al.  Segmentation of Color Textures , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[88]  Wang Hui,et al.  Color Texture Segmentation Using Quaternion-Gabor Filters , 2006 .

[89]  A.H. Kam,et al.  General unsupervised multiscale segmentation of images , 1999, Conference Record of the Thirty-Third Asilomar Conference on Signals, Systems, and Computers (Cat. No.CH37020).

[90]  Bt Thomas,et al.  Segmentation of natural images using self-organising feature maps , 1996 .

[91]  Kimon P. Valavanis,et al.  Chromatic measures for color texture description and analysis , 1995, Proceedings of Tenth International Symposium on Intelligent Control.

[92]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[93]  Matti Pietikäinen,et al.  Unsupervised texture segmentation using feature distributions , 1997, Pattern Recognit..

[94]  Lilong Shi,et al.  Quaternion Color Texture Segmentation , 2006 .

[95]  Josef Kittler,et al.  Automatic watershed segmentation of randomly textured color images , 1997, IEEE Trans. Image Process..

[96]  Rajeev Sharma,et al.  Adaptive texture and color segmentation for tracking moving objects , 2002, Pattern Recognit..

[97]  W. S. Ooi,et al.  Fusion of colour and texture features in image segmentation: an empirical study , 2009 .

[98]  M.,et al.  Statistical and Structural Approaches to Texture , 2022 .

[99]  Wang Yonggang,et al.  Combined color and texture segmentation based on fibonacci lattice sampling and mean shift , 2005 .

[100]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[101]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[102]  Qian Huang,et al.  Quantitative methods of evaluating image segmentation , 1995, Proceedings., International Conference on Image Processing.

[103]  Minoru Asada,et al.  Initial segmentation for knowledge indexing , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[104]  Junqing Chen.  Adaptive perceptual color-texture image segmentation , 2006 .

[105]  Paul F. Whelan,et al.  A generic framework for colour texture segmentation , 2010 .

[106]  Sean Dougherty,et al.  Edge detector evaluation using empirical ROC curves , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[107]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[108]  Carlos Vázquez,et al.  Multiregion competition: A level set extension of region competition to multiple region image partitioning , 2006, Comput. Vis. Image Underst..

[109]  Jitendra Malik,et al.  An empirical approach to grouping and segmentation , 2002 .

[110]  Jian Cheng,et al.  Color- and texture-based image segmentation using local feature analysis approach , 2003, International Symposium on Multispectral Image Processing and Pattern Recognition.

[111]  A. Murat Tekalp,et al.  Fusion of color and edge information for improved segmentation and edge linking , 1997, Image Vis. Comput..

[112]  Rongchun Zhao,et al.  Adaptive Segmentation of Textured Images by Using the Coupled Markov Random Field Model , 2006, IEEE Transactions on Image Processing.

[113]  B. S. Manjunath,et al.  Edge flow: A framework of boundary detection and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[114]  Tomio Echigo,et al.  Unsupervised segmentation of colored texture images by using multiple GMRF models and a hypothesis of merging primitives , 2000, Systems and Computers in Japan.

[115]  Glenn Healey,et al.  Unsupervised segmentation of textured color images using Markov random field models , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[116]  Allan Hanbury,et al.  Morphological segmentation on learned boundaries , 2009, Image Vis. Comput..

[117]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[118]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1999, TOGS.

[119]  Zhou Yue,et al.  Color-texture based unsupervised segmentation using JSEG with fuzzy connectedness , 2006 .

[120]  Rachid Deriche,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Colour, Texture, and Motion in Level Set Based Segmentation and Tracking Colour, Texture, and Motion in Level Set Based Segmentation and Tracking , 2022 .

[121]  Marina Meila,et al.  Comparing Clusterings by the Variation of Information , 2003, COLT.

[122]  Max Mignotte,et al.  Fusion of Hidden Markov Random Field Models and Its Bayesian Estimation , 2006, IEEE Transactions on Image Processing.

[123]  Paul F. Whelan,et al.  Integration of feature distributions for colour texture segmentation , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[124]  Arnold W. M. Smeulders,et al.  Color texture measurement and segmentation , 2005, Signal Process..

[125]  Kimon P. Valavanis,et al.  A color texture based visual monitoring system for automated surveillance , 1999, IEEE Trans. Syst. Man Cybern. Part C.

[126]  Andreas Koschan,et al.  Colour Image Segmentation: A Survey , 1994 .

[127]  Alireza Khotanzad,et al.  Color image retrieval using multispectral random field texture model and color content features , 2003, Pattern Recognit..

[128]  Xianghua Xie,et al.  Active Contouring Based on Gradient Vector Interaction and Constrained Level Set Diffusion , 2010, IEEE Transactions on Image Processing.

[129]  Xavier Cufí,et al.  Yet Another Survey on Image Segmentation: Region and Boundary Information Integration , 2002, ECCV.

[130]  P. Whelan,et al.  Adaptive Pre-Filtering Techniques for Colour Image Analysis , 2007, International Machine Vision and Image Processing Conference (IMVIP 2007).

[131]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[132]  Jie Yang,et al.  Color Texture Segmentation Using Quaternion-Gabor Filters , 2006, 2006 International Conference on Image Processing.

[133]  J. M. Hans du Buf,et al.  A review of recent texture segmentation and feature extraction techniques , 1993 .

[134]  Joachim M. Buhmann,et al.  Combined color and texture segmentation by parametric distributional clustering , 2002, Object recognition supported by user interaction for service robots.

[135]  Rachid Deriche,et al.  Tensor Processing for Texture and Colour Segmentation , 2005, SCIA.

[136]  Begoña Acha,et al.  Pattern analysis of dermoscopic images based on Markov random fields , 2009, Pattern Recognit..

[137]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[138]  Tomio Echigo,et al.  Unsupervised segmentation of colored texture images by using multiple GMRF models and a hypothesis of merging primitives , 2000, Systems and Computers in Japan.

[139]  Raimund Lakmann.  Statistische Modellierung von Farbtexturen , 1998 .

[140]  Taghi M. Khoshgoftaar,et al.  Unsupervised multiscale color image segmentation based on MDL principle , 2006, IEEE Transactions on Image Processing.

[141]  David A. Clausi,et al.  Unsupervised image segmentation using a simple MRF model with a new implementation scheme , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[142]  Rama Chellappa,et al.  Unsupervised Texture Segmentation Using Markov Random Field Models , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[143]  Mustafa Ozden,et al.  A color image segmentation approach for content-based image retrieval , 2007, Pattern Recognit..

[144]  Paul F. Whelan,et al.  Colour saliency-based parameter optimisation for adaptive colour segmentation , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).