A Fog Computing and Cloudlet Based Augmented Reality System for the Industry 4.0 Shipyard

Augmented Reality (AR) is one of the key technologies pointed out by Industry 4.0 as a tool for enhancing the next generation of automated and computerized factories. AR can also help shipbuilding operators, since they usually need to interact with information (e.g., product datasheets, instructions, maintenance procedures, quality control forms) that could be handled easily and more efficiently through AR devices. This is the reason why Navantia, one of the 10 largest shipbuilders in the world, is studying the application of AR (among other technologies) in different shipyard environments in a project called “Shipyard 4.0”. This article presents Navantia’s industrial AR (IAR) architecture, which is based on cloudlets and on the fog computing paradigm. Both technologies are ideal for supporting physically-distributed, low-latency and QoS-aware applications that decrease the network traffic and the computational load of traditional cloud computing systems. The proposed IAR communications architecture is evaluated in real-world scenarios with payload sizes according to demanding Microsoft HoloLens applications and when using a cloud, a cloudlet and a fog computing system. The results show that, in terms of response delay, the fog computing system is the fastest when transferring small payloads (less than 128 KB), while for larger file sizes, the cloudlet solution is faster than the others. Moreover, under high loads (with many concurrent IAR clients), the cloudlet in some cases is more than four times faster than the fog computing system in terms of response delay.

[1]  Dixys L. Hernández-Rojas,et al.  Design and Practical Evaluation of a Family of Lightweight Protocols for Heterogeneous Sensing through BLE Beacons in IoT Telemetry Applications , 2017, Sensors.

[2]  D.W. Engels,et al.  Characterization of RF Propagation in Helical and Toroidal Metal Pipes for Passive RFID Systems , 2008, 2008 IEEE International Conference on RFID.

[3]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[4]  Alexandru Dancu,et al.  The Ultimate Display , 2014 .

[5]  Carlos Eduardo Pereira,et al.  Visualising the digital twin using web services and augmented reality , 2016, 2016 IEEE 14th International Conference on Industrial Informatics (INDIN).

[6]  Harald Wuest,et al.  Augmented reality pipe layout planning in the shipbuilding industry , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[7]  Nassir Navab Developing killer apps for industrial augmented reality , 2004, IEEE Computer Graphics and Applications.

[8]  Tian Ma,et al.  A 3D model simplification algorithm based on edge-collapse , 2012, IEEE 10th International Conference on Industrial Informatics.

[9]  Samarjit Chakraborty,et al.  Design methods for augmented reality in-vehicle infotainment systems , 2014, 2014 51st ACM/EDAC/IEEE Design Automation Conference (DAC).

[10]  Timothy Gifford,et al.  Virtual training for welding , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[11]  Raja Lavanya,et al.  Fog Computing and Its Role in the Internet of Things , 2019, Advances in Computer and Electrical Engineering.

[12]  Soumya Kanti Datta,et al.  Comparison of edge computing implementations: Fog computing, cloudlet and mobile edge computing , 2017, 2017 Global Internet of Things Summit (GIoTS).

[13]  R. Radkowski,et al.  AR-based modular construction system for automobile advance development , 2003, 2003 IEEE International Augmented Reality Toolkit Workshop.

[14]  Dieter Schmalstieg,et al.  Adaptive ghosted views for Augmented Reality , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[15]  Dimitris Mavrikios,et al.  A virtual and augmented reality approach to collaborative product design and demonstration , 2008, 2008 IEEE International Technology Management Conference (ICE).

[16]  Hong-Seok Park,et al.  Augmented Reality based Cockpit Module Assembly System , 2008, 2008 International Conference on Smart Manufacturing Application.

[17]  Michael Beigl,et al.  Augmented service in the factory of the future , 2012, 2012 Ninth International Conference on Networked Sensing (INSS).

[18]  Tiago M. Fernández-Caramés,et al.  A Practical Evaluation of a High-Security Energy-Efficient Gateway for IoT Fog Computing Applications , 2017, Sensors.

[19]  Tiago M. Fernández-Caramés,et al.  An Electricity Price-Aware Open-Source Smart Socket for the Internet of Energy , 2017, Sensors.

[20]  Luis Castedo,et al.  A Methodology for Evaluating Security in Commercial RFID Systems , 2017 .

[21]  Fei Tao,et al.  Digital Twin and Big Data Towards Smart Manufacturing and Industry 4.0: 360 Degree Comparison , 2018, IEEE Access.

[22]  Tiago M. Fernández-Caramés,et al.  Enabling automatic event detection for the pipe workshop of the shipyard 4.0 , 2017, 2017 56th FITCE Congress.

[23]  J. Novak-Marcincin,et al.  Visualization of intelligent assembling process by augmented reality tools application , 2012, 2012 4th IEEE International Symposium on Logistics and Industrial Informatics.

[24]  Alois Ferscha,et al.  Augmented reality navigation systems , 2006, Universal Access in the Information Society.

[25]  J. R. Llata,et al.  Working Together: A Review on Safe Human-Robot Collaboration in Industrial Environments , 2017, IEEE Access.

[26]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[27]  D. W. Mizell Virtual reality and augmented reality in aircraft design and manufacturing , 1994, Proceedings of WESCON '94.

[28]  Paula Fraga Lamas Enabling technologies and cyber-physical systems for mission-critical scenarios , 2017 .

[29]  Fabian Quint,et al.  Comparing Video and Augmented Reality Assistance in Manual Assembly , 2016, 2016 12th International Conference on Intelligent Environments (IE).

[30]  Michel Cordonnier,et al.  Contribution of augmented reality to the maintenance of network equipment , 2017 .

[31]  O. R. Goode,et al.  QUALITY CONTROL IN SHIPBUILDING , 1964 .

[32]  David Baudry,et al.  Augmented reality maintenance demonstrator and associated modelling , 2015, 2015 IEEE Virtual Reality (VR).

[33]  Tiago M. Fernández-Caramés,et al.  A Review on Human-Centered IoT-Connected Smart Labels for the Industry 4.0 , 2018, IEEE Access.

[34]  R. Radkowski,et al.  AR-based product design in automobile industry , 2002, The First IEEE International Workshop Agumented Reality Toolkit,.

[35]  Tiago M. Fernández-Caramés,et al.  A Review on Industrial Augmented Reality Systems for the Industry 4.0 Shipyard , 2018, IEEE Access.

[36]  Jürgen Jasperneite,et al.  A context-aware assistance system for maintenance applications in smart factories based on augmented reality and indoor localization , 2015, 2015 IEEE 20th Conference on Emerging Technologies & Factory Automation (ETFA).

[37]  Tiago M. Fernández-Caramés,et al.  RSS stabilization techniques for a real-time passive UHF RFID pipe monitoring system for smart shipyards , 2017, 2017 IEEE International Conference on RFID (RFID).

[38]  S. F. Wong,et al.  The effect of metal noise factor to RFID location system , 2013, 2013 IEEE International Conference on Industrial Engineering and Engineering Management.

[39]  Kok-Why Ng,et al.  Simplification of 3D Triangular Mesh for Level of Detail Computation , 2014, 2014 11th International Conference on Computer Graphics, Imaging and Visualization.

[40]  Teodor Petrescu,et al.  An experimental evaluation of 3D mesh decimation techniques , 2011, ISSCS 2011 - International Symposium on Signals, Circuits and Systems.

[41]  Tiago M. Fernández-Caramés,et al.  A Practical Evaluation of Commercial Industrial Augmented Reality Systems in an Industry 4.0 Shipyard , 2018, IEEE Access.

[42]  Thomas B. Moeslund,et al.  Task space HRI for cooperative mobile robots in fit-out operations inside ship superstructures , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[43]  Tiago M. Fernández-Caramés,et al.  Smart Pipe System for a Shipyard 4.0 , 2016, Sensors.

[44]  Wolfgang Friedrich,et al.  ARVIKA-augmented reality for development, production and service , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[45]  Heping Chen,et al.  Augmented reality enhanced "top-down" nano-manufacturing , 2004, 4th IEEE Conference on Nanotechnology, 2004..

[46]  Axel Gräser,et al.  A step forward in manual welding: demonstration of augmented reality helmet , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[47]  Tiago M. Fernández-Caramés,et al.  Towards the Internet of Smart Trains: A Review on Industrial IoT-Connected Railways , 2017, Sensors.

[48]  Paula Fraga-Lamas,et al.  Enabling technologies and cyber-physical systems for mission-critical scenarios , 2017 .

[49]  Tiago M. Fernández-Caramés,et al.  A Review on Internet of Things for Defense and Public Safety , 2016, Sensors.

[50]  Didier Stricker,et al.  Augmented reality based on edge computing using the example of remote live support , 2017, 2017 IEEE International Conference on Industrial Technology (ICIT).