Predicting subjective sensation of reality during multimedia consumption based on EEG and peripheral physiological signals

Sensation of reality refers to the ability of users to feel present in a multimedia experience. As 3D technologies target to provide more immersive and higher quality multimedia experiences, it is important to understand Quality of Experience (QoE) and sensation of reality. Recently, there have been efforts to measure brain activity in order to understand implicitly QoE for various multimedia contents. However, brain activity accounting for sensation of reality has not been adequately investigated. The goal of this paper is twofold. First, we investigate how various aspects, such as perceived quality, perceived depth, and content preference affect subjective sensation of reality through explicit subjective ratings. Second, we construct subjective classification systems to predict sensation of reality from multimedia experiences based on electroencephalography (EEG) and peripheral physiological signals such as heart rate and respiration.

[1]  Pascal Bianchi,et al.  Classification of Periodic Activities Using the Wasserstein Distance , 2012, IEEE Transactions on Biomedical Engineering.

[2]  ITU-T Rec. P.910 (04/2008) Subjective video quality assessment methods for multimedia applications , 2009 .

[3]  Willis J. Tompkins,et al.  A Real-Time QRS Detection Algorithm , 1985, IEEE Transactions on Biomedical Engineering.

[4]  Sebastian Bosse,et al.  Toward a Direct Measure of Video Quality Perception Using EEG , 2012, IEEE Transactions on Image Processing.

[5]  Aleksandar Kalauzi,et al.  Extracting complexity waveforms from one-dimensional signals , 2009, Nonlinear biomedical physics.

[6]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[7]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[8]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[9]  Ronald D Berger,et al.  Heart Rate Variability , 2006, Journal of cardiovascular electrophysiology.

[10]  Subjective methods for the assessment of stereoscopic 3DTV systems , 2015 .

[11]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[13]  Yong Man Ro,et al.  Human brain response to visual fatigue caused by stereoscopic depth perception , 2011, 2011 17th International Conference on Digital Signal Processing (DSP).

[14]  Keetaek Kham,et al.  Measurement of 3D Visual Fatigue Using Event-Related Potential (ERP): 3D Oddball Paradigm , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.