On the Performance Characterisation of Image Segmenation Algorithms: A Case Study

An experimental vehicle is being developed for the purposes of precise crop treatment, with the aim of reducing chemical use and thereby improving quality and reducing both costs and environmental contamination. For differential treatment of crop and weed, the vehicle must discriminate between crop, weed and soil. We present a two stage algorithm designed for this purpose, and use this algorithm to illustrate how empirical discrepancy methods, notably the analysis of type I and type II statistical errors and receiver operating characteristic curves, may be used to compare algorithm performance over a set of test images which represent typical working conditions for the vehicle. Analysis of performance is presented for the two stages of the algorithm separately, and also for the combined algorithm. This analysis allows us to understand the effects of various types of misclassification error on the overall algorithm performance, and as such is a valuable methodology for computer vision engineers.

[1]  John A. Marchant,et al.  Model Based Tracking for Navigation and Segmentation , 1998, ECCV.

[2]  M. Degroot,et al.  Probability and Statistics , 2021, Examining an Operational Approach to Teaching Probability.

[3]  Bernard F. Buxton,et al.  Modelling of single mode distributions of colour data using directional statistics , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Bobby Rao,et al.  Data association methods for tracking systems , 1993 .

[5]  Timothy F. Cootes,et al.  Improving identification performance by integrating evidence from sequences , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[6]  Sean Dougherty,et al.  Edge detector evaluation using empirical ROC curves , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[7]  J B Southall,et al.  The development and evaluation of computer vision algorithms for the control of an autonomous horticultural vehicle , 1999 .

[8]  John A. Marchant,et al.  Controllability and Observability: Tools for Kalman Filter Design , 1998, BMVC.

[9]  J. Koenderink,et al.  Diffuse and specular reflectance from rough surfaces. , 1998, Applied optics.

[10]  Harry L. Van Trees,et al.  Detection, Estimation, and Modulation Theory, Part I , 1968 .

[11]  Y. J. Zhang,et al.  A survey on evaluation methods for image segmentation , 1996, Pattern Recognit..

[12]  Mahesan Niranjan,et al.  Realisable Classifiers: Improving Operating Performance on Variable Cost Problems , 1998, BMVC.

[13]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[14]  Herbert Freeman,et al.  On the Encoding of Arbitrary Geometric Configurations , 1961, IRE Trans. Electron. Comput..

[15]  H. V. Trees.  Detection, Estimation, And Modulation Theory , 2001 .

[16]  Rainer H. Biller,et al.  Reduced input of herbicides by use of optoelectronic sensors , 1998 .

[17]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[18]  M. Hunink,et al.  ROC analysis of the clinical, CT and MRI diagnosis of orbital space-occupying lesions , 1989 .

[19]  John A. Marchant,et al.  Vision-Aided Outdoor Navigation of an Autonomous Horticultural Vehicle , 1999, ICVS.

[20]  Ikram E Abdou.  Quantitative Methods of Edge Detection , 1978 .

[21]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[22]  Robert M. Haralick,et al.  Performance Characterization in Computer Vision , 1993, BMVC.