The conspicuity of different targets in image sequences taken by approaching sensors is addressed in applications such as the assessment of camouflage effectiveness or the performance evaluation of autonomous systems. In such evaluation processes the consideration of background characteristics is essential due to the propensity to confuse target and background signatures. Several discriminating features of target and background signature can be derived. Furthermore, the changing aspect and spatial resolution during an approach on a target have to be taken into account. Considering salient points in image sequences, we perform a nominal/actual value comparison by evaluating the receiver operating characteristic (ROC) curve for the detections in each image. Hence, reference regions for targets and backgrounds are provided for the entire image sequence by means of robust image registration. The consideration of the uncertainty for the temporal progression of the ROC curve enables hypothesis testing for well-founded statements about the significance of the determined distinctiveness of targets with respect to their background. The approach is neither restricted to images taken by IR sensors nor applicable to low level image analysis steps only, but can be considered as a general method for the assessment of feature evaluation and target distinctiveness. The analysis method proposed facilitates an objective comparison of object appearance with both, its relevant background and other targets, using different image analysis features. The feasibility and the usefulness of the approach are demonstrated with real data recorded with a FLIR sensor during a field trial on a bare and mock-up target.
[1]
Lothar Hofmann,et al.
Evaluation of essential design criteria for IRST systems
,
1993,
Optics & Photonics.
[2]
H. B. Mann,et al.
On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other
,
1947
.
[3]
Robert C. Bolles,et al.
Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography
,
1981,
CACM.
[4]
D. Bamber.
The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
,
1975
.
[5]
J. Hanley,et al.
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
,
1982,
Radiology.
[6]
Bernhard P. Wrobel,et al.
Multiple View Geometry in Computer Vision
,
2001
.
[7]
Andrew P. Bradley,et al.
The use of the area under the ROC curve in the evaluation of machine learning algorithms
,
1997,
Pattern Recognit..