A fuzzy expert system for evaluating human observers in a visual detection task

The Receiver Operating Characteristic (ROC) method has been successfully used to evaluate the performance of human observers in visual detection and discrimination tasks, to quantitatively assess image quality and to evaluate imaging systems. In this work, we present a Fuzzy Expert System to facilitate the discrimination of the performance of human observers in a visual detection task. To reduce the complexity of our system, we implemented it using two Fuzzy Subsystems in cascade in order to combine the effect of our input variables. We used the performance indexes dk and p(a) of ROC theory to define the input and output variables of 30 empirical cases and 23 synthetic ones.We compared the dkd index (the output of our expert system) with dk and found that both were similar for the cases of average performance. However, we found significant advantages when our system was applied to analyze the poor and good performance cases, which is important for several applications. Our system was developed to analyze data of ROC rating experiments using four categories, but can easily be adapted to analyze data of rating experiments using more categories or data of Multiple Alternative Forced Choice (MAFC) experiments.