A rapid prototyping tool to produce 360º video-based immersive experiences enhanced with virtual/multimedia elements

Abstract While the popularity of virtual reality (VR) grows in a wide range of application contexts – e.g. entertainment, training, cultural heritage and medicine –, its economic impact is expected to reach around 15bn USD, by the year of 2020. Within VR field, 360° video has been sparking the interest of development and research communities. However, editing tools supporting 360° panoramas are usually expensive and/or demand programming skills and/or advanced user knowledge. Besides, application approaches to quickly and intuitively set up such 360° video-based VR environments complemented with diverse types of parameterizable virtual assets and multimedia elements are still hard to find. Thereby, this paper aims to propose a system specification to simply and rapidly configure immersive VR environments composed of surrounding 360° video spheres that can be complemented with parameterizable multimedia contents – namely 3D models, text and spatial sound –, whose behavior can be either time-range or user-interaction dependent. Moreover, a preliminary prototype that follows a substantial part of the previously mentioned specification and implements the enhancement of 360° videos with time-range dependent virtual assets is presented. Preliminary tests evaluating usability and user satisfaction were also carried out with 30 participants, from which encouraging results were achieved.

[1]  Taehyun Rhee,et al.  MR360: Mixed Reality Rendering for 360° Panoramic Videos , 2017, IEEE Transactions on Visualization and Computer Graphics.

[2]  Lizzy Bleumers,et al.  Seeing the bigger picture: a user perspective on 360° TV , 2012, EuroITV.

[3]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[4]  Shana Smith,et al.  A virtual reality keyboard with realistic haptic feedback in a fully immersive virtual environment , 2017, Virtual Reality.

[5]  Akio Kameda,et al.  Dive into Remote Events: Omnidirectional Video Streaming with Acoustic Immersion , 2015, ACM Multimedia.

[6]  Teresa Chambel,et al.  Towards immersive interactive video through 360° hypervideo , 2011, Advances in Computer Entertainment Technology.

[7]  Ismo Rakkolainen,et al.  Interaction with WebVR 360° video player: Comparing three interaction paradigms , 2017, 2017 IEEE Virtual Reality (VR).

[8]  Michael Beigl,et al.  pARnorama: 360 degree interactive video for augmented reality prototyping , 2013, UbiComp.

[9]  Maximino Bessa,et al.  Information system for the management and visualization of multisensorial contents , 2015, 2015 10th Iberian Conference on Information Systems and Technologies (CISTI).

[10]  Miguel Melo,et al.  Immersive 360∘\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$360^{\circ }$$\end{document} video user experience: impa , 2017, Universal Access in the Information Society.

[11]  E. Peres,et al.  Bringing together UAS-based land surveying and procedural modelling of buildings to set up enhanced VR environments for cultural heritage , 2017, 2017 24º Encontro Português de Computação Gráfica e Interação (EPCGI).

[12]  Miguel Melo,et al.  Does 3D 360 video enhance user's VR experience?: An Evaluation Study , 2016, Interacción.

[13]  Bum-Jae You,et al.  Be closer as you being there: HMD-based social interaction system , 2016, SIGGRAPH ASIA VR Showcase.

[14]  Yahong Rosa Zheng,et al.  Immersive audio for human-machine interface of unmanned ground vehicles , 2016, 2016 IEEE International Workshop on Acoustic Signal Enhancement (IWAENC).

[15]  Madhukar Budagavi,et al.  360 degrees video coding using region adaptive smoothing , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[16]  David White,et al.  Getting up your nose: a virtual reality education tool for nasal cavity anatomy , 2017, SIGGRAPH Asia 2017 Symposium on Education.

[17]  Miguel Melo,et al.  The impact of different exposure times to 360° video experience on the sense of presence , 2016, 2016 23° Encontro Português de Computação Gráfica e Interação (EPCGI).

[18]  Javier Taibo,et al.  Immersive video for virtual tourism , 2001, SPIE ITCom.

[19]  J E Deutsch,et al.  Virtual reality for stroke rehabilitation: an abridged version of a Cochrane review. , 2015, European journal of physical and rehabilitation medicine.

[20]  Judy M. Vance,et al.  Industry use of virtual reality in product design and manufacturing: a survey , 2017, Virtual Reality.

[21]  Truong Cong Thang,et al.  A subjective study on QoE of 360 video for VR communication , 2017, 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP).

[22]  Min Sun,et al.  Tell Me Where to Look: Investigating Ways for Assisting Focus in 360° Video , 2017, CHI.

[23]  Iis P. Tussyadiah,et al.  Virtual reality, presence, and attitude change: Empirical evidence from tourism , 2018, Tourism Management.

[24]  José Martins,et al.  Immersive Edition of Multisensory 360 Videos , 2018, WorldCIST.