Quantifying performance of mine detectors with fewer than 10,000 targets

The two primary measures of detection performance are probability of detection (PD) and false alarm rate (FAR). Since these are statistical measures, the large statistical uncertainties associated with measurements on small data sets impose a severe limitation on the interpretation and extrapolation of the test results. The main difficulty with the data collected from many of the landmine detection experiments is that the number of mine targets is small, and the clutter data is measured on small areas. As a result of analyzing data from several detection tests for both UXO and mine detection technologies, the Institute for Defense Analyses (IDA) has generated a number of suggestions to improve the results obtained from future demonstrations and tests. In the absence of a rigorous error analysis, we can estimate the uncertainty in a probability of detection measurement with a simple model. If detections can be described as a binomial process weighted by the true probability of detection, then uncertainty equals (root)N, where N is the number of mines detected, and uncertainty in PD (percent) equals (root)N/N X 100 for large N. A similar statistical uncertainty is encountered in the measurement of probability of false alarm. An opportunity for a false alarm can be defined by the amount of ground covered by the projected area of the target plus an allowable miss distance, say 1 m2. Thus, for each 1 m2 of ground that it passes over, the system has one opportunity to declare a false alarm. A test field that is 1 m by 20 m offers only 20 m2 of area and thus only 20 samples for probability of false alarm measurement. A site that is 4 m wide and 4 km long covers 16,000 m2. Thus, there will be a substantial increase in the clutter environment samples by the sensor and a concomitant improvement in false alarm measurements. Of course, false alarm density will be highly site dependent, so extrapolation of the result is still uncertain. Unfortunately, this limitation is insurmountable in single test and at a single site.