Objective comparison of edge detection assessment methods based on genetic optimization

For many image processing applications, edge detection is a very important task that needs to be assessed, since the success or failure of these applications depends on the performance of this task. Assessment of edge detection is largely subjective; however, current trends in the image processing community are moving toward objective assessment. In recent years, many different methods have been proposed to assess edge detection, although no agreement has been reached as the proper method, since previous comparisons have produced contrasting results. A comparison of assessment methods using an objective approach is presented. Methods are compared by analyzing the results of an optimization procedure using genetic algorithms and the methods as fitness. The comparison is based on the premise that better assessment methods will lead the optimization procedure to produce better results. A cross-validation is carried out to compare the results obtained using one assessment method with others. Conclusions provide recommendations for authors interested in assessing edge detection algorithms.

[1]  Paola Campadelli,et al.  Quantitative evaluation of color image segmentation results , 1998, Pattern Recognit. Lett..

[2]  Hicham Laanaya,et al.  Evaluation for uncertain image classification and segmentation , 2006, Pattern Recognit..

[3]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[4]  Peter H. A. Sneath,et al.  Numerical Taxonomy: The Principles and Practice of Numerical Classification , 1973 .

[5]  Martial Hebert,et al.  Toward Objective Evaluation of Image Segmentation Algorithms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  José Martínez-Aroza,et al.  A measure of quality for evaluating methods of segmentation and edge detection , 2001, Pattern Recognit..

[7]  Sudeep Sarkar,et al.  Comparison of Edge Detectors: A Methodology and Initial Study , 1998, Comput. Vis. Image Underst..

[8]  Jaime S. Cardoso,et al.  Toward a generic evaluation of image segmentation , 2005, IEEE Transactions on Image Processing.

[9]  Bir Bhanu,et al.  Adaptive image segmentation using a genetic algorithm , 1989, IEEE Transactions on Systems, Man, and Cybernetics.

[10]  William A. Yasnoff,et al.  Error measures for scene segmentation , 1977, Pattern Recognit..

[11]  Riccardo Poli,et al.  Genetic algorithm-based interactive segmentation of 3D medical images , 1999, Image Vis. Comput..

[12]  Daniel F. García,et al.  A Method for Assessment of Segmentation Success Considering Uncertainty in the Edge Positions , 2006, EURASIP J. Adv. Signal Process..

[13]  Sean Dougherty,et al.  Edge Detector Evaluation Using Empirical ROC Curves , 2001, Comput. Vis. Image Underst..

[14]  Linda G. Shapiro,et al.  Image Segmentation Techniques , 1984, Other Conferences.

[15]  Fritz Albregtsen,et al.  A Supervised Approach to the Evaluation of Image Segmentation Methods , 1995, CAIP.

[16]  Yee-Hong Yang,et al.  Multiresolution Color Image Segmentation , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Yitzhak Yitzhaky,et al.  A Method for Objective Edge Detection Evaluation and Detector Parameter Selection , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Hélène Laurent,et al.  Unsupervised and Supervised Image Segmentation Evaluation , 2006 .

[19]  Yu-Jin Zhang,et al.  A Summary of Racent Progresses for Segmentation Evaluation , 2006 .

[20]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Sang Uk Lee,et al.  A comparative performance study of several global thresholding techniques for segmentation , 1990, Comput. Vis. Graph. Image Process..

[22]  I.E. Abdou,et al.  Quantitative design and evaluation of enhancement/thresholding edge detectors , 1979, Proceedings of the IEEE.

[23]  Franco Oberti,et al.  ROC curves for performance evaluation of video sequences processing systems for surveillance applications , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[24]  Fernando Pereira,et al.  Objective evaluation of video segmentation quality , 2003, IEEE Trans. Image Process..

[25]  Jan J. Gerbrands,et al.  Segmentation evaluation using ultimate measurement accuracy , 1992, Electronic Imaging.

[26]  Chun Chen,et al.  Multimetric evaluation protocol for user-assisted video object extraction systems , 2003, Visual Communications and Image Processing.

[27]  Martin D. Levine,et al.  Dynamic Measurement of Computer Generated Image Segmentations , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Y. J. Zhang,et al.  A survey on evaluation methods for image segmentation , 1996, Pattern Recognit..

[29]  Rama Chellappa,et al.  Edge Detection and Linear Feature Extraction Using a 2-D Random Field Model , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Paul L. Rosin,et al.  Evaluation of global image thresholding for change detection , 2003, Pattern Recognit. Lett..

[31]  Ronen Basri,et al.  Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Hyun Seung Yang,et al.  Robust image segmentation using genetic algorithm with a fuzzy measure , 1996, Pattern Recognit..

[33]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[34]  Hui Zhang,et al.  An entropy-based objective evaluation method for image segmentation , 2003, IS&T/SPIE Electronic Imaging.

[35]  Gabriel Oliver,et al.  On the use of the overlapping area matrix for image segmentation evaluation: A survey and new performance measures , 2006, Pattern Recognit. Lett..

[36]  Henk L. Muller,et al.  Evaluating Image Segmentation Algorithms Using the Pareto Front , 2002, ECCV.

[37]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[38]  Francisco José Madrid-Cuevas,et al.  Automatic generation of consensus ground truth for the comparison of edge detection techniques , 2008, Image Vis. Comput..

[39]  Francisco José Madrid-Cuevas,et al.  Characterization of empirical discrepancy evaluation measures , 2004, Pattern Recognit. Lett..