Method of evaluating image-recovery algorithms based on task performance

A method of evaluating image-recovery algorithms is presented that is based on the numerical computation of how well a specified visual task can be performed on the basis of the reconstructed images. A Monte Carlo technique is used to simulate the complete imaging process including generation of scenes appropriate to the desired application, subsequent data taking, image recovery, and performance of the stated task based on the final image. The pseudorandom-simulation process permits one to assess the response of an image-recovery algorithm to many different realizations of the same type of scene. The usefulness of this method is demonstrated through a study of the algebraic reconstruction technique (ART), a tomographic reconstruction algorithm that reconstructs images from their projections. The task chosen for this study is the detection of disks of known size and position. Task performance is rated on the basis of the detectability index derived from the area under the receiver operating characteristic curve. In the imaging situations explored, the use of the nonnegativity constraint in the ART dramatically increases the detectability of objects in some instances, particularly when the data consist of a limited number of noiseless projections. Conversely, the nonnegativity constraint does not improve detectability when the data are complete but noisy.