Machine learning of human responses to images

The human user is an often ignored component of the imaging chain. In medical diagnostic tasks, the human observer plays the role of the decision-maker, forming opinions based on visual assessment of images. In content-based image retrieval, the human user is the ultimate judge of the relevance of images recalled from a database. We argue that data collected from human observers should be used in conjunction with machine-learning algorithms to model and optimize performance in tasks that involve humans. In essence, we treat the human observer as a nonlinear system to be identified. In this paper, we review our work in two applications of this general idea. In the first, a learning machine is trained to predict the accuracy of human observers in a lesion detection task for purposes of assessing image quality. In the second, a learning machine is trained to predict human users' perception of the similarity of two images for purposes of content-based image retrieval from a database. In both examples, it is shown that a nonlinear learning machine can accurately identify the nonlinear human system that maps images into numerical values, such as detection performance or image similarity.