Stereo Image Quality Assessment Based on Sparse Binocular Fusion Convolution Neural Network

In this paper, a sparse binocular fusion convolution neural network is proposed to evaluate the quality of stereo image. In order to simulate the long-term fusion and processing of the left and right views in the brain visual pathway, the network combines the two views of the stereo image four times, and the information processing is carried out through convolution together with the fusion operation. In addition, in order to overcome the computational-intensive and memory-intensive problems of convolution neural networks, a structural sparsity learning (SSL) method is used to regularize the proposed convolution neural network. The experimental results demonstrate that our proposed method performs effectively and efficiently. And the proposed method can achieve 2.0× speedups on LIVE I database and 2.3× speedup on LIVE II database on the basis of improved performance.

[1]  Hassan Foroosh,et al.  Sparse Convolutional Neural Networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Yong Ding,et al.  No-reference Stereoscopic Image Quality Assessment Based on Saliency-guided Binocular Feature Consolidation , 2017 .

[3]  Tim S. Meese,et al.  Basic vision: an introduction to visual perception , 2007 .

[4]  Qionghai Dai,et al.  Learning Blind Quality Evaluator for Stereoscopic Images Using Joint Sparse Representation , 2016, IEEE Transactions on Multimedia.

[5]  Touradj Ebrahimi,et al.  Assessment of Stereoscopic Crosstalk Perception , 2012, IEEE Transactions on Multimedia.

[6]  Mei Yu,et al.  Joint structure–texture sparse coding for quality prediction of stereoscopic images , 2015 .

[7]  Yiran Chen,et al.  Learning Structured Sparsity in Deep Neural Networks , 2016, NIPS.

[8]  Andrey S. Krylov,et al.  No-Reference Stereoscopic Image Quality Assessment Using Convolutional Neural Network for Adaptive Feature Extraction , 2018, IEEE Access.

[9]  Zhihan Lv,et al.  Quality Index for Stereoscopic Images by Jointly Evaluating Cyclopean Amplitude and Cyclopean Phase , 2017, IEEE Journal of Selected Topics in Signal Processing.

[10]  R. Blake,et al.  Neural bases of binocular rivalry , 2006, Trends in Cognitive Sciences.

[11]  Bin Jiang,et al.  No-Reference Stereoimage Quality Assessment for Multimedia Analysis Towards Internet-of-Things , 2018, IEEE Access.

[12]  Kai Li,et al.  No-reference stereo image quality assessment by learning gradient dictionary-based color visual characteristics , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[13]  Kai Li,et al.  Reduced-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics and Structural Degradation , 2018, IEEE Access.

[14]  Nader Karimi,et al.  Blind Stereo Quality Assessment Based on Learned Features From Binocular Combined Images , 2017, IEEE Transactions on Multimedia.

[15]  Mei Yu,et al.  No-reference Stereoscopic Image Quality Assessment Using Binocular Self-similarity and Deep Neural Network , 2016, Signal Process. Image Commun..

[16]  Chaofeng Li,et al.  Stereoscopic image quality assessment via convolutional neural networks , 2017, 2017 International Smart Cities Conference (ISC2).