Development of an Estimation Model for Instantaneous Presence in Audio-Visual Content

The sense of presence is often used to evaluate the performances of audio-visual (AV) content and systems. However, a presence meter has yet to be realized. We consider that the sense of presence can be divided into two aspects: system presence and content presence. In this study we focused on content presence. To estimate the overall presence of a content item, we have developed estimation models for the sense of presence in audio-only and audio-visual content. In this study, the audio-visual model is expanded to estimate the instantaneous presence in an AV content item. Initially, we conducted an evaluation experiment of the presence with 40 content items to investigate the relationship between the features of the AV content and the instantaneous presence. Based on the experimental data, a neural-network-based model was developed by expanding the previous model. To express the variation in instantaneous presence, 6 audiorelated features and 14 visual-related features, which are extracted from the content items in 500-ms intervals, are used as inputs for the model. The audio-related features are loudness, sharpness, roughness, dynamic range and standard deviation in sound pressure levels, and movement of sound images. The visual-related features involve hue, lightness, saturation, and movement of visual images. After constructing the model, a generalization test confirmed that the model is sufficiently accurate to estimate the instantaneous presence. Hence, the model should contribute to the development of a presence meter. key words: sense of presence, instantaneous presence, audio-visual content, neural networks, time-series data

[1]  Arie E. Kaufman,et al.  A human factors study of color notation systems for computer graphics , 1982, CACM.

[2]  Holger Regenbrecht,et al.  The Experience of Presence: Factor Analytic Insights , 2001, Presence: Teleoperators & Virtual Environments.

[3]  Hugo Fastl,et al.  Psychoacoustics: Facts and Models , 1990 .

[4]  Masanori Morise,et al.  Development of an estimation model for instantaneous presence in audio content , 2014, 2014 IEEE Fourth International Conference on Consumer Electronics Berlin (ICCE-Berlin).

[5]  S. Kuwano,et al.  Continuous judgment of level-fluctuating sounds and the relationship between overall loudness and instantaneous loudness , 1985, Psychological research.

[6]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[7]  Yôiti Suzuki,et al.  Effects of visual information on auditory presence , 2003 .

[8]  Pantelis N. Vassilakis,et al.  SRA: A Web-based Research Tool for Spectral and Roughness Analysis of Sound Signals , 2007 .

[9]  Morise Masanori,et al.  On Development of an Estimation Model for Instantaneous Presence in Audio-visual Content , 2015 .

[10]  Mariano Alcañiz Raya,et al.  Assessment of the influence of navigation control and screen size on the sense of presence in virtual reality using EEG , 2014, Expert Syst. Appl..

[11]  Umpei Nagashima,et al.  Development of NEural network simulator for structure-activity COrrelation of molecules: Neco(2). The structure-activity relationship between 13C-NMR chemical shifts and the carcinogenicity of polycyclic aromatic hydrocarbons(PAH). , 1994 .

[12]  Umpei Nagashima,et al.  Development of NEural network simulator for structure-activity COrrelation of molecules:Neco. (3). Performance Evaluation of Self-organized Network and Perceptron. , 1998 .

[13]  Masanori Morise,et al.  Instantaneous Evaluation of the Sense of Presence in Audio-Visual Content , 2015, IEICE Trans. Inf. Syst..

[14]  長嶋 雲兵,et al.  分子の構造活性相関解析のためのニューラルネットワークシミュレータ : Neco (NEural network simulator for structure-activity COrrelation of molecules) の開発 , 1994 .

[15]  Kenji Ozawa,et al.  Kansei Estimation Models for the Sense of Presence in Audio-Visual Content with Different Audio Reproduction Methods , 2012, 2012 13th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing.

[16]  Thomas B. Sheridan.  Further Musings on the Psychophysics of Presence , 1996, Presence: Teleoperators & Virtual Environments.

[17]  Kenji Ozawa,et al.  Development of Kansei estimation models for the sense of presence in audio-visual content , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[18]  Masaki Emoto,et al.  Viewing angle effects from wide field video projection images on the human equilibrium , 2005, Displays.

[19]  Kenji Ozawa,et al.  CONTENTS WHICH YIELD HIGH AUDITORY-PRESENCE IN SOUND REPRODUCTION , 2002 .

[20]  Akio Ando,et al.  A 22.2 Multichannel Sound System for Ultrahigh-Definition TV (UHDTV) , 2008 .

[21]  Manfred R. Schroeder,et al.  Computer simulation of sound transmission in rooms , 1963 .

[22]  Kenji Ozawa,et al.  Content Presence vs. System Presence in Audio Reproduction Systems , 2008, 2008 Second International Symposium on Universal Communication.

[23]  Thomas B. Sheridan,et al.  Musings on Telepresence and Virtual Presence , 1992, Presence: Teleoperators & Virtual Environments.

[24]  Michael Meehan,et al.  Physiological measures of presence in stressful virtual environments , 2002, SIGGRAPH.