Real-Virtual World Device Synchronization in a Cloud-Enabled Social Virtual Reality IoT Network

Virtual reality (VR) is currently being used in many different areas such as car prototyping, gaming, medical training, teaching, and so on. The Internet of Things (IoT) devices such as systems-on-a-chip (e.g. Raspberry Pi), smart appliances, and sensors support a wide range of services, including machine automation, and remote monitoring and control. This paper introduces a novel social VR-IoT environment, which allows users to share and control local or remote IoT devices in a virtual platform. Two approaches using the VR-IoT solution are presented: one local network-based and one cloud-based. The proposed VR-IoT environment contains VRITESS, the novel VR-IoT Environment Synchronization Scheme, which facilitates a consistent and integrated experience for users by enabling control of real IoT objects with VR headsets. The control of some IoT objects in extreme environments or devices which are complex to operate can be simplified in a virtual environment. The VRITESS synchronization scheme maintains the real objects updated following instructions given in the virtual world and vice-versa. Testing involved local network-based and cloud-based testbeds created with a VR headset and IoT devices with the Performance Engineering Laboratory, Dublin City University’s, Ireland. Test results demonstrated that lower latency is experienced in the local-network testbed in comparison with the cloud testbed. Furthermore, tests regarding the communications protocols implemented in the cloud testbed indicated that MQTT generates less delay and data traffic than REST.

[1]  G Santhosh Kumar,et al.  A framework for predicting the performance of IoT protocols, a use case based approach , 2017, 2017 International Conference On Smart Technologies For Smart Nation (SmartTechCon).

[2]  Michele Zorzi,et al.  WebIoT: A web application framework for the internet of things , 2012, 2012 IEEE Wireless Communications and Networking Conference Workshops (WCNCW).

[3]  Ying-Tsung Lee,et al.  An integrated cloud-based smart home management system with community hierarchy , 2016, IEEE Transactions on Consumer Electronics.

[4]  Bin He,et al.  Brain-Computer Interface Control in a Virtual Reality Environment and Applications for the Internet of Things , 2018, IEEE Access.

[5]  L. Mainetti,et al.  A web based virtual environment as a connection platform between people and IoT , 2016, 2016 International Multidisciplinary Conference on Computer and Energy Science (SpliTech).

[6]  Claire Rowland,et al.  User experience design for the Internet of Things , 2015 .

[7]  Guiou Kobayashi,et al.  The internet of things and its impact on social relationships involving mutual trust , 2015, 2015 IEEE International Symposium on Technology and Society (ISTAS).

[8]  Sarvpal Singh,et al.  Virtual reality: A brief survey , 2017, 2017 International Conference on Information Communication and Embedded Systems (ICICES).

[9]  Yuna Jeong,et al.  AVIoT: web-based interactive authoring and visualization of indoor internet of things , 2015, IEEE Transactions on Consumer Electronics.

[10]  Bong-Seok Seo,et al.  Internet of Things (IoT) for Seamless Virtual Reality Space: Challenges and Perspectives , 2018, IEEE Access.

[11]  Andrés Marín López,et al.  Seamless human-device interaction in the internet of things , 2017, IEEE Transactions on Consumer Electronics.

[12]  Yong Man Ro,et al.  Deep Virtual Reality Image Quality Assessment With Human Perception Guider for Omnidirectional Image , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Yunhee Kang,et al.  Docker based computation off-loading for video game based mobile VR framework , 2017, 2017 8th IEEE International Conference on Software Engineering and Service Science (ICSESS).

[14]  Jerker Delsing,et al.  Enabling IoT automation using local clouds , 2016, 2016 IEEE 3rd World Forum on Internet of Things (WF-IoT).

[15]  Luca Chittaro,et al.  Effects of Different Types of Virtual Reality Display on Presence and Learning in a Safety Training Scenario , 2018, IEEE Transactions on Visualization and Computer Graphics.

[16]  Peter Corcoran A Matter of Timing: Consumer Electronics and Network Time , 2013, IEEE Consumer Electronics Magazine.

[17]  Kwang-Roh Park,et al.  User-friendly home automation based on 3D virtual world , 2010, IEEE Transactions on Consumer Electronics.

[18]  Gabriel-Miro Muntean,et al.  Synchronisation Between Real and Virtual-World Devices in a VR-IoT Environment , 2018, 2018 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB).

[19]  Gabriel-Miro Muntean,et al.  Age of Information as a QoS Metric in a Relay-Based IoT Mobility Solution , 2018, 2018 14th International Wireless Communications & Mobile Computing Conference (IWCMC).

[20]  Liam Murphy,et al.  Objective and subjective evaluation of QOAS video streaming over broadband networks , 2005, IEEE Transactions on Network and Service Management.

[21]  Liam Murphy,et al.  Subjective assessment of the quality-oriented adaptive scheme , 2005, IEEE Transactions on Broadcasting.

[22]  Gabriel-Miro Muntean,et al.  Performance analysis of the Quality of Service-aware NETworking Scheme for sMart Internet of Things gatewayS , 2017, 2017 13th International Wireless Communications and Mobile Computing Conference (IWCMC).

[23]  Tetsuya Yokotani,et al.  Transfer protocols of tiny data blocks in IoT and their performance evaluation , 2016, 2016 IEEE 3rd World Forum on Internet of Things (WF-IoT).

[24]  Andrew Hines,et al.  Streaming VR for immersion: Quality aspects of compressed spatial audio , 2017, 2017 23rd International Conference on Virtual System & Multimedia (VSMM).

[25]  Sujit Dey,et al.  Wireless VR/AR with Edge/Cloud Computing , 2017, 2017 26th International Conference on Computer Communication and Networks (ICCCN).

[26]  David Swapp,et al.  An ‘In the Wild’ Experiment on Presence and Embodiment using Consumer Virtual Reality Equipment , 2016, IEEE Transactions on Visualization and Computer Graphics.

[27]  Mohsen Hallaj Asghar,et al.  Design and simulation of energy efficiency in node based on MQTT protocol in Internet of Things , 2015, 2015 International Conference on Green Computing and Internet of Things (ICGCIoT).

[28]  Wilhelm Hasselbring,et al.  Exploring software cities in virtual reality , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[29]  Philip Lelyveld,et al.  Virtual Reality Primer with an Emphasis on Camera-Captured VR , 2015 .

[30]  Xicheng Tan,et al.  Virtual Reality in Smart City , 2013, GRMSE.

[31]  Luigi Atzori,et al.  Quality of Experience in the Multimedia Internet of Things: Definition and practical use-cases , 2015, 2015 IEEE International Conference on Communication Workshop (ICCW).

[32]  Mingshao Zhang,et al.  Collaborative Virtual Laboratory Environments with Hardware in the Loop , 2018 .

[33]  Raffaele Giaffreda,et al.  IoT and cloud convergence: Opportunities and challenges , 2014, 2014 IEEE World Forum on Internet of Things (WF-IoT).

[34]  Veton Kepuska,et al.  Next-generation of virtual personal assistants (Microsoft Cortana, Apple Siri, Amazon Alexa and Google Home) , 2018, 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC).

[35]  Gabriel-Miro Muntean,et al.  A NETworking scheme for an Internet of Things Integration Platform , 2017, 2017 IEEE International Conference on Communications Workshops (ICC Workshops).

[36]  Levent Gürgen,et al.  Sharing user IoT devices in the cloud , 2014, 2014 IEEE World Forum on Internet of Things (WF-IoT).

[37]  Gabriel-Miro Muntean,et al.  A Relay and Mobility Scheme for QoS Improvement in IoT Communications , 2018, 2018 IEEE International Conference on Communications Workshops (ICC Workshops).

[38]  Sehyun Park,et al.  Design and implementation of Hyper-connected IoT-VR Platform for customizable and intuitive remote services , 2017, 2017 IEEE International Conference on Consumer Electronics (ICCE).

[39]  Eun-Seok Ryu,et al.  Computing Offloading Over mmWave for Mobile VR: Make 360 Video Streaming Alive , 2018, IEEE Access.

[40]  Cristina Hava Muntean,et al.  EcoLearn : Battery Power Friendly e-Learning Environment for Mobile Device Users , 2011 .

[41]  Zhihan Lv,et al.  Virtual Reality Smart City Based on WebVRGIS , 2016, IEEE Internet of Things Journal.