Residual Frame for Noisy Video Classification According to Perceptual Quality in Convolutional Neural Networks

Perceptual quality of a video describes the quality consistent with human perception. The growing popularity of short video sharing on mobile platforms such as Tik Tok and WeSee makes the video assessment system based on perceptual quality a necessity. In practice, short videos captured by mobile devices often contain different types of distortions incurred by sensor noise or compression noise, which potentially makes the videos visually unpleasing to users and may degrade the performance of deep neural networks when applied to these noisy videos. Thus, it is necessary to identify noisy videos based on video perceptual quality. However, traditional video/image noise estimation methods are designed to estimate the variance of homogeneously distributed synthetic noise, not real noise. In this paper, we propose a simple yet effective method to recognize the noisy videos using their residual frames. Since the original video frame contains rich content information, which may result in under-or over-estimation of the noise, we construct residual frames to reduce the influence of the content information while maintaining the main noise information in the video. We also create a new data set with more than 30 thousand images captured from videos with real noise. Experimental results demonstrate the effectiveness of our proposed method.