Effect of observer inattention in a detection task on ROC analysis

ROC experiments generally assume observers to be fully attentive to the task of the experiment and, therefore, that the experiment measures observers' innate ability in performing the task. In a detection task, inattention can cause an observer to overlook signals or signal-like potential false positives in an image. In an ROC experiment that involves a detection task, inattention can either cause an observer to report a strong confidence for signal absent, or cause the observer to report on some other finding less salient than the finding that the observer fails to notice. The purpose of this study was to determine the effect of observer inattention on empirical ROC estimates and the appropriateness of the conventional binormal model for fitting such ROC curves. An experiment was designed in which observers were asked to detect a signal of a simple geometric shape and varying contrast in a background of Gaussian noise. The images sometimes also contained other objects of different geometric shapes in locations that did not overlap the signal. Observer inattention was simulated by blocking a quarter of each image from observer view. Results showed that observer inattention caused the empirical ROC curve to decrease, but curve fitting with the binormal model appeared to be equally accurate with and without observer inattention. However, a "proper" ROC model might be more appropriate than the binormal model because the binormal model invariably produced a "hook" near the upper-right corner of the ROC unit square that was not indicated by the empirical operating points. We conclude that observer inattention in a detection task can be evaluated as part of a conventional ROC experiment.