An improved pairwise comparison scaling method for subjective image quality assessment

As human is the final receptor of images, user-perceptive quality of images need to be scaled by subjective quality assessment method, which is also important for learning a high performance of objective image quality assessment model. There are characteristics of randomness and individual difference among subjective testers. Those scores made by unreliable subjects that may affect the scaling of image quality scores, are defined as outliers. Recent studies have shown that pairwise comparison is more reliable and accurate than single stimulus method. However, the traditional outlier detection method for pairwise comparison, i.e, transitivity satisfaction, only compared results for individual participant, ignoring the statistical information among subjects for each pair of images. In this paper, an improved scaling method for pair-wise comparison was proposed, which combines transitivity satisfaction and statistical information among all participants for each pair of images to evaluate the reliability of participant and reject the outlier. Pairwise comparison experimental result on a geometric distorted stereoscopic image database demonstrates the improvement of the proposed method.