An EEG-Based Study on Perception of Video Distortion Under Various Content Motion Conditions

Human perception sensitivity to video distortion is vital for visual quality assessment (VQA). Different from the perception mechanism of image distortion that has been thoroughly studied, the perception of video distortion is inevitably influenced by motion of dynamic content due to the characteristics of the human visual system (HVS). In this paper, electroencephalography (EEG) is used as a novel psychophysiological method to study the human perception sensitivity to quantification-aroused video distortion under various content motion conditions. For this purpose, we conduct experiments to record the EEG signals of the subjects when they are watching distorted videos. According to the feature analysis of EEG data, the P300 component aroused by human perception of video quality change is selected as the indicator of human perception of distortion. By the means of classification based on linear discriminant analysis (LDA), it is found that the separability of the P300 component, which is measured by the area under curve (AUC) of the receiver operating characteristic (ROC), is positively correlated with the perceptibility of distortion. The correlation provides a valid psychophysiological method, which is exempt from being influenced by subjective bias due to human high-level cognitive activities, for evaluating distortion perceptibility. In addition, the regression analysis results demonstrate a sigmoid-typed quantitative relation between the perceptibility of distortion and separability of the P300 component. Based on such relation, the perceptibility thresholds of distortion corresponding to various content motion speeds are calibrated by EEG signals and it is found that the content motion speed has a significant impact on distortion perceptibility.

[1]  Sebastian Möller,et al.  A Physiological Approach to Determine Video Quality , 2011, 2011 IEEE International Symposium on Multimedia.

[2]  J. Cohen,et al.  P300, stimulus intensity, modality, and probability. , 1996, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[3]  Sebastian Möller,et al.  Using eye-tracking and correlates of brain activity to predict quality scores , 2014, 2014 Sixth International Workshop on Quality of Multimedia Experience (QoMEX).

[4]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[5]  Kun Li,et al.  Assessment visual fatigue of watching 3DTV using EEG power spectral parameters , 2014, Displays.

[6]  Sebastian Bosse,et al.  Objective quality assessment of stereoscopic images with vertical disparity using EEG , 2017, Journal of neural engineering.

[7]  K. Müller,et al.  Single-trial analysis of the neural correlates of speech quality perception , 2013, Journal of neural engineering.

[8]  Sebastian Bosse,et al.  Psychophysiology-Based QoE Assessment: A Survey , 2017, IEEE Journal of Selected Topics in Signal Processing.

[9]  Tapani Ristaniemi,et al.  Linking Brain Responses to Naturalistic Music Through Analysis of Ongoing EEG and Stimulus Features , 2013, IEEE Transactions on Multimedia.

[10]  Marcus A. Magnor,et al.  Assessing the quality of compressed images using EEG , 2011, 2011 18th IEEE International Conference on Image Processing.

[11]  Sebastian Möller,et al.  Changes of vigilance caused by varying bit rate conditions , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[12]  J. Friedman Regularized Discriminant Analysis , 1989 .

[13]  Thomas Wiegand,et al.  Brain-Computer Interfacing for multimedia quality assessment , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[14]  Gabriel Curio,et al.  Subjective Listening Tests and Neural Correlates of Speech Degradation in Case of Signal-Correlated Noise , 2010 .

[15]  J. Fermaglich Electric Fields of the Brain: The Neurophysics of EEG , 1982 .

[16]  Manish Narwaria,et al.  Data Analysis in Multimedia Quality Assessment: Revisiting the Statistical Tests , 2017, IEEE Transactions on Multimedia.

[17]  Sebastian Möller,et al.  Perception of low-quality videos analyzed by means of electroencephalography , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[18]  Curtis F. Gerald,et al.  APPLIED NUMERICAL ANALYSIS , 1972, The Mathematical Gazette.

[19]  Cristina Hava Muntean,et al.  User-centered EEG-based multimedia quality assessment , 2013, 2013 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB).

[20]  Chun-Hsien Chou,et al.  A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile , 1995, IEEE Trans. Circuits Syst. Video Technol..

[21]  Sebastian Bosse,et al.  EEG-based classification of video quality perception using steady state visual evoked potentials (SSVEPs) , 2015, Journal of neural engineering.

[22]  Robert Schleicher,et al.  Does Low Quality Audiovisual Content Increase Fatigue of Viewers? , 2017 .

[23]  Sebastian Möller,et al.  Using electroencephalography to analyze sleepiness due to low-quality audiovisual stimuli , 2016, Signal Process. Image Commun..

[24]  Sebastian Möller,et al.  Using electroencephalography and subjective self-assessment to measure the influence of quality variations in cloud gaming , 2015, 2015 Seventh International Workshop on Quality of Multimedia Experience (QoMEX).

[25]  Marcus A. Magnor,et al.  Single-trial EEG classification of artifacts in videos , 2012, TAP.

[26]  Lina J. Karam,et al.  Adaptive image coding with perceptual distortion control , 2002, IEEE Trans. Image Process..

[27]  Nicole Krämer,et al.  Time Domain Parameters as a feature for EEG-based Brain-Computer Interfaces , 2009, Neural Networks.

[28]  Kun Li,et al.  EEG-based detection and evaluation of fatigue caused by watching 3DTV , 2013, Displays.

[29]  Klaus-Robert Müller,et al.  Integration of Multivariate Data Streams With Bandpower Signals , 2013, IEEE Transactions on Multimedia.

[30]  J. Polich Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[31]  Is-Haka Mkwawa,et al.  Investigation of relationships between changes in EEG features and subjective quality of HDR images , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[32]  Sebastian Bosse,et al.  Neurally informed assessment of perceived natural texture image quality , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[33]  Marcus A. Magnor,et al.  ElectroEncephaloGraphics: Making Waves in Computer Graphics Research , 2014, IEEE Computer Graphics and Applications.

[34]  Sebastian Möller,et al.  Using Electroencephalography to Measure Perceived Video Quality , 2014, IEEE Journal of Selected Topics in Signal Processing.

[35]  Marcus A. Magnor,et al.  Evaluation of video artifact perception using event-related potentials , 2011, APGV '11.

[36]  Sebastian Möller,et al.  Analyzing Speech Quality Perception Using Electroencephalography , 2012, IEEE Journal of Selected Topics in Signal Processing.

[37]  Sebastian Bosse,et al.  Assessing Perceived Image Quality Using Steady-State Visual Evoked Potentials and Spatio-Spectral Decomposition , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[38]  Gabriel Curio,et al.  Using ERPs for assessing the (sub) conscious perception of noise , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[39]  Xiaoming Tao,et al.  Calibrating Human Perception Threshold of Video Distortion Using EEG , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[40]  ITU-T Rec. P.910 (04/2008) Subjective video quality assessment methods for multimedia applications , 2009 .

[41]  D. H. Kelly Motion and vision. II. Stabilized spatio-temporal threshold surface. , 1979, Journal of the Optical Society of America.

[42]  Xiang Chen,et al.  A Joint Multimodal Group Analysis Framework for Modeling Corticomuscular Activity , 2013, IEEE Transactions on Multimedia.

[43]  Touradj Ebrahimi,et al.  EEG correlates during video quality perception , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[44]  José Higino Correia,et al.  Wireless instrumentation system based on dry electrodes for acquiring EEG signals. , 2012, Medical engineering & physics.

[45]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[46]  Sebastian Bosse,et al.  Toward a Direct Measure of Video Quality Perception Using EEG , 2012, IEEE Transactions on Image Processing.

[47]  Seong-Eun Moon,et al.  Perceptual Experience Analysis for Tone-mapped HDR Videos Based on EEG and Peripheral Physiological Signals , 2015, IEEE Transactions on Autonomous Mental Development.

[48]  Sebastian Möller,et al.  Too tired for calling? A physiological measure of fatigue caused by bandwidth limitations , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[49]  Mahmoud Reza Hashemi,et al.  A View-Level Rate Distortion Model for Multi-View/3D Video , 2016, IEEE Transactions on Multimedia.

[50]  Munchurl Kim,et al.  Learning-Based Just-Noticeable-Quantization- Distortion Modeling for Perceptual Video Coding , 2018, IEEE Transactions on Image Processing.