Non-reference Quality Assessment Model using Deep learning for Omnidirectional Images

Image quality assessment (IQA) has been a popular research topic in image processing. However, most studies until now have been focusing on traditional images and only a few focused on omnidirectional images. Unlike in the case of traditional images, the users can only view a part of 360-degree images at a time, and thus tend to focus more on specific regions of the image. This makes predicting quality scores for omnidirectional images a challenging task since most existing models for traditional images usually treat all regions of the image equally. In this paper, we propose an omnidirectional image quality assessment model based on deep learning. This model focuses on learning the features of the middle region of input images. The model first automatically predicts the quality scores for patches sampled from the input image. The quality score of the image will then be calculated by weighted averaging of the patch quality scores based on their positions. Experimental results show that the proposed model provides very promising accuracy for predicting quality scores of omnidirectional images.

[1]  Shenghua Gao,et al.  Saliency Detection in 360 ◦ Videos , 2022 .

[2]  Krista A. Ehinger,et al.  Recognizing scene viewpoint using panoramic place representation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Bernd Girod,et al.  A Framework to Evaluate Omnidirectional Video Coding Schemes , 2015, 2015 IEEE International Symposium on Mixed and Augmented Reality.

[4]  Zhengfang Duanmu,et al.  End-to-End Blind Image Quality Assessment Using Deep Neural Networks , 2018, IEEE Transactions on Image Processing.

[5]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Shenghua Gao,et al.  Saliency Detection in 360 ^\circ ∘ Videos , 2018, ECCV.

[7]  Paolo Napoletano,et al.  On the use of deep learning for blind image quality assessment , 2016, Signal Image Video Process..

[8]  Yong Man Ro,et al.  Deep Virtual Reality Image Quality Assessment With Human Perception Guider for Omnidirectional Image , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[10]  Mingkai Huang,et al.  Modeling the Perceptual Quality of Immersive Images Rendered on Head Mounted Displays: Resolution and Compression , 2018, IEEE Transactions on Image Processing.

[11]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[12]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[13]  Sebastian Bosse,et al.  Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment , 2016, IEEE Transactions on Image Processing.

[14]  Thomas Maugey,et al.  Saliency-based navigation in omnidirectional image , 2017, 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP).

[15]  Yong Man Ro,et al.  VR IQA NET: Deep Virtual Reality Image Quality Assessment Using Adversarial Learning , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Truong Cong Thang,et al.  An evaluation of quality metrics for 360 videos , 2017, 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN).