Monitoring Maritime Industry 4.0 Systems through VR Environments

In this paper, we present a novel system aimed at replacing large-display information dashboards in industrial control rooms, with a flexible, dynamic and reconfigurable immersive virtual reality environment, which can afford high mobility without constraints, to remote engineers. In this context, we investigate the role of semantic and spatial cues for delivering event notifications within the control environment, and present empirical evidence from a controlled laboratory study, simulating a marine industrial environment. We find that spatial and semantic cues can both offer significant benefits to operator awareness and their combination can significantly improve the findability and response time to particular information in the extended information space surrounding the user.

[1]  Nikos Deligiannis,et al.  Visualization of Real-Time Heterogeneous Smart City Data Using Virtual Reality , 2019, 2019 IEEE International Smart Cities Conference (ISC2).

[2]  Andreas Komninos,et al.  Natural interaction with large map interfaces in VR , 2017, PCI.

[3]  Eric Horvitz,et al.  Disruption and recovery of computing tasks: field study, analysis, and directions , 2007, CHI.

[4]  Marianne Sargent.  Gorillas in our midst , 2010 .

[5]  Elaine C. Silva,et al.  The Immersion in Virtual Reality of Control Room Activity , 2021 .

[6]  Niels Henze,et al.  Notification in VR: The Effect of Notification Placement, Task and Environment , 2019, CHI PLAY.

[7]  Henrique M. Gaspar,et al.  Challenges when creating a cohesive digital twin ship: a data modelling perspective , 2021 .

[8]  Chen Shuai,et al.  The Technological Development and Prospect of Unmanned Surface Vessel , 2019, 2019 IEEE 3rd Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC).

[9]  Lee Lisle.  [DC] The Immersive Space to Think: Immersive Analytics for Multimedia Data , 2020, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW).

[10]  Chris North,et al.  Do we still need physical monitors? An evaluation of the usability of AR virtual monitors for productivity work , 2021, 2021 IEEE Virtual Reality and 3D User Interfaces (VR).

[11]  Antonio Barrientos,et al.  A training system for Industry 4.0 operators in complex assemblies based on virtual reality and process mining , 2019, Robotics Comput. Integr. Manuf..

[12]  Vasja Roblek,et al.  A Complex View of Industry 4.0 , 2016 .

[13]  Chrysostomos D. Stylios,et al.  Realizing Virtual Reality Learning Environment for Industry 4.0 , 2019, Procedia CIRP.

[14]  Joseph S. Valacich,et al.  The Effects of Interruptions, Task Complexity, and Information Presentation on Computer-Supported Decision-Making Performance , 2003, Decis. Sci..

[15]  Mohsen Zare,et al.  Application of Virtual Reality to Improve Physical Ergonomics in a Control Room of a Chemical Industry , 2018 .

[16]  Alain Pinsonneault,et al.  The many faces of information technology interruptions: a taxonomy and preliminary investigation of their performance effects , 2015, Inf. Syst. J..

[17]  Eunhee Chang,et al.  Virtual Reality Sickness: A Review of Causes and Measurements , 2020, Int. J. Hum. Comput. Interact..

[18]  Christopher Andrews,et al.  Space to think: large high-resolution displays for sensemaking , 2010, CHI.

[19]  Stephen Brown,et al.  The Hiperwall tiled-display wall system for Big-Data research , 2018, Journal of Big Data.

[20]  Xianzhuo Luo,et al.  Virtual reality: A survey of enabling technologies and its applications in IoT , 2021, J. Netw. Comput. Appl..

[21]  R. Cox,et al.  Dimensions of Attentional Processing , 2007 .

[22]  Roberto Revetria,et al.  Augmented and virtual reality applications in industrial systems: A qualitative review towards the industry 4.0 era , 2018 .

[23]  Norberto Martínez García,et al.  Creation of Virtual Environment for Teaching Mechanical Maintenance on a Tug Ship , 2020, 2020 IEEE International Conference on Engineering Veracruz (ICEV).

[24]  Constantine Stephanidis,et al.  Virtual Reality for Smart City Visualization and Monitoring , 2018, Progress in IS.

[25]  Danielle Albers Szafir,et al.  Graphical Perception for Immersive Analytics , 2020, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[26]  Roberto Pierdicca,et al.  Cyber Physical Systems for Industry 4.0: Towards Real Time Virtual Reality in Smart Manufacturing , 2018, AVR.

[27]  Mishal Dholakia,et al.  Dataspace: A Reconfigurable Hybrid Reality Environment for Collaborative Information Analysis , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[28]  Ahm Shamsuzzoha,et al.  Digital factory – virtual reality environments for industrial training and maintenance , 2019, Interact. Learn. Environ..

[29]  Barrett Ens,et al.  Maps Around Me , 2020, Proc. ACM Hum. Comput. Interact..

[30]  Yung-Ju Chang,et al.  Bridging the Virtual and Real Worlds: A Preliminary Study of Messaging Notifications in Virtual Reality , 2020, CHI.

[31]  Leandro Barbosa S. Gatto,et al.  Virtual simulation of a nuclear power plant's control room as a tool for ergonomic evaluation , 2013 .

[32]  Per Ola Kristensson,et al.  Digital Twin Assessments in Virtual Reality: An Explorational Study with Aeroengines , 2020, 2020 IEEE Aerospace Conference.

[33]  Jason Yon,et al.  Characterising the Digital Twin: A systematic literature review , 2020, CIRP Journal of Manufacturing Science and Technology.

[34]  Dirk Werth,et al.  Virtual remote inspection — A new concept for virtual reality enhanced real-time maintenance , 2017, 2017 23rd International Conference on Virtual System & Multimedia (VSMM).

[35]  Melanie Tory,et al.  What Do We Talk About When We Talk About Dashboards? , 2019, IEEE Transactions on Visualization and Computer Graphics.

[36]  S. Brewster,et al.  Expanding the Bounds of Seated Virtual Workspaces , 2020, ACM Trans. Comput. Hum. Interact..

[37]  Raimund Dachselt,et al.  Personal Augmented Reality for Information Visualization on Large Interactive Displays , 2020, IEEE Transactions on Visualization and Computer Graphics.

[38]  Gregory D. Abowd,et al.  NotifiVR: Exploring Interruptions and Notifications in Virtual Reality , 2018, IEEE Transactions on Visualization and Computer Graphics.

[39]  Antonio Krüger,et al.  Immersive Notification Framework: Adaptive & Plausible Notifications in Virtual Reality , 2018, CHI Extended Abstracts.

[40]  C. Chabris,et al.  Gorillas in Our Midst: Sustained Inattentional Blindness for Dynamic Events , 1999, Perception.

[41]  Hanna Koskinen,et al.  Use of Immersive 3-D Virtual Reality Environments in Control Room Validations , 2018 .