No-reference screen content image quality assessment based on multi-region features

Abstract Unlike natural image captured with cameras, screen content image (SCI) is a composite image including textual and pictorial regions. The different characteristics lead to many difficulties in image quality assessment (IQA). Most existing models based on the convolutional neural network (CNN) divide large SCIs into image patches to increase training samples for CNN training. This brings two problems: (1) a single image patch can not represent the quality of the entire image, especially in IQA of SCI; (2) SCI patches of an entire image degraded by the same distortion type and strength may have drastically different quality. In addition, these models adopt the mean square error (MSE) between the predicted quality and the subjective differential mean opinion score (DMOS) to train the CNN, without considering quality ranking between different SCIs. In this paper, we propose a novel no-reference (NR) IQA model based on the convolutional neural network (CNN). The contributions of our algorithm can be concluded as follows: (1) considering a large difference exists in different regions in a SCI, the pseudo global features generated with multi-region local features are utilized for quality evaluation, which better reflect image quality than local features of each image patch; (2) the noise classification task is used as an auxiliary task which aids the quality score prediction task to improve the representation ability; (3) the Siamese networks are used to predict the quality scores of two different SCIs, and a new ranking loss is proposed to rank the predicted scores, aiming to enhance the ability of the model to rank image in terms of quality. Experimental results verify that our model outperforms all test NR IQA methods and full-reference (FR) IQA methods on the screen content image quality assessment database (SIQAD).