Gaze-tracking-based audio-visual correlation analysis employing quality of experience methodology

This paper investigates a new approach to audio-visual correlation assessment based on the gaze-tracking system developed at the Multimedia Systems Department (MSD) of Gdansk University of Technology (GUT). The gaze-tracking methodology, having roots in Human-Computer Interaction borrows the relevance feedback through gaze-tracking and applies it to the new area of interests, which is Quality of Experience (QoE). Results of subjective tests carried out at the MSD showed a strong dependency between video presented in the screen and the perceived audio. It has also been shown that the application of gaze-tracking to the audio-visual correlation analysis allows for the objectivization of results obtained in subjective tests. Therefore this research study concentrates on the possibility to apply this methodology to the area of Quality of Experience.

[1]  Andrzej Czyzewski,et al.  Determination of Influence of Visual Cues on Perception of Spatial Sound , 2001 .

[2]  G. J. Thomas.  Experimental study of the influence of vision on sound localization. , 1941 .

[3]  Andrzej Czyzewski,et al.  Determining Influence of Visual Cues on the Perception of Surround Sound Using Soft Computing , 2000, Rough Sets and Current Trends in Computing.

[4]  Bozena Kostek,et al.  An new method of audio-visual correlation analysis , 2009, 2009 International Multiconference on Computer Science and Information Technology.

[5]  Andrzej Czyzewski,et al.  Influence of visual cues on the perception of surround sound , 2000 .

[6]  David J. Meares.  Perceptual Attributes of Multichannel Sound , 1993 .

[7]  Søren Bech,et al.  Interaction Between Audio-Visual Factors in a Home Theater System: Definition of Subjective Attributes , 1995 .

[8]  Andrzej Czyzewski,et al.  Concentration Tests - An Application of Gaze Tracker to Concentration Exercises , 2009, CSEDU.

[9]  Francis Rumsey,et al.  Computer Games and Multichannel Audio Quality - The Effect of Division of Attention between Auditory and Visual Modalities , 2003 .

[10]  Daniel Thalmann,et al.  Evaluation of Gaze Tracking Technology for Social Interaction in Virtual Environments , 2004 .

[11]  Carlos Hitoshi Morimoto,et al.  A computer vision framework for eye gaze tracking , 2003, 16th Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2003).

[12]  Yoshinobu Ebisawa,et al.  Improved video-based eye-gaze detection method , 1994, Conference Proceedings. 10th Anniversary. IMTC/94. Advanced Technologies in I & M. 1994 IEEE Instrumentation and Measurement Technolgy Conference (Cat. No.94CH3424-9).

[13]  Myung Jin Chung,et al.  Non-contact eye gaze tracking system by mapping of corneal reflections , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[14]  Andrzej Czyzewski,et al.  Making Surround Audio Considering Image Proximity Effect , 2002 .

[15]  Bozena Kostek.  Perception-Based Data Processing in Acoustics: Applications to Music Information Retrieval and Psychophysiology , 2005, Studies in Computational Intelligence.

[16]  H. A. Witkin,et al.  Sound localization with conflicting visual and auditory cues. , 1952, Journal of experimental psychology.

[17]  Toshiyuki Gotoh,et al.  Controlling Sound-Image Localization in Stereophonic Reproduction: II , 1981 .

[18]  Andrzej Czyzewski,et al.  Determining the influence of visual stimuli on the peception of surround sound using data mining algorithms , 2001 .

[19]  Francis Rumsey,et al.  Can Playing a Computer Game Affect Perception of Audio-Visual Synchrony? , 2004 .

[20]  Andrzej Czyzewski,et al.  Discovering the Influence of Visual Stimuli on the Perception of Surround Sound Using Genetic Algorithms , 2001 .

[21]  Y Dong Hyun,et al.  NON-INTRUSIVE EYE GAZE ESTIMATION WITHOUT KNOWLEDGE OF EYE POSE , 2004 .

[22]  Zhiwei Zhu,et al.  Eye gaze tracking under natural head movements , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Rainer Stiefelhagen,et al.  Gaze tracking for multimodal human-computer interaction , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[24]  Kalevi Kilkki,et al.  Quality of Experience in Communications Ecosystem , 2008, J. Univers. Comput. Sci..