Augmented Reality-Based Tele-Robotic System Architecture for On-Site Construction

The planning of construction process is crucial for successful project management and based on which all other sub-tasks and activities follow. Labourintensive construction process which relies on field workers’ own intuition and interpretation, has been too complicated to control, which can lead to mistakes, wasted time, and fatal accidents in worst but commonly happening cases. In this sense, applying innovative techniques to on-site construction process for construction planning, scheduling, operation, and monitoring has the potential to assist stakeholders in making better decisions. A head-mounted display (HMD) such as Oculus Rift can visualize construction processes information anytime whether during on-site work or beforehand by providing realistic augmented superimposed environment which is captured via the proposed 2-DOF stereo head. Moreover, by calculating the physical movement (translation, rotation) of building components, which can be accomplished by reading physical, material property data from Building Information Modeling (BIM), and combining it with computational kinematics model of on-site robotic equipment, the field workers can have the digital information for off-site component assembly in real-time manner. This research develops a new methodology for integration of tele-operation with 4D modelling in order to improve building component positioning and erection in terms of efficiency and quality by operating onsite robotic system intelligently. Based on the proposed methodology, an intelligent remotely operated, considering the control and video streaming processes, virtual tele-presence system is prototyped. The proposed system is designed for tele-operated construction robots and comprises a 2DOF stereo head, a pair of high spatial resolution cameras, and a head mounted display with integrated head tracking mechanism. To validate the proposed methodology, controlled laboratory experiments were designed and implemented.

[1]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[2]  Philip H. S. Torr,et al.  The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix , 1997, International Journal of Computer Vision.

[3]  Peter Willemsen,et al.  The effects of head-mounted display mechanics on distance judgments in virtual environments , 2004, APGV '04.

[4]  Meng-Han Tsai,et al.  On-Site Building Information Retrieval by Using Projection-Based Augmented Reality , 2012, J. Comput. Civ. Eng..

[5]  Iain E. G. Richardson,et al.  H.264 and MPEG-4 Video Compression: Video Coding for Next-Generation Multimedia , 2003 .

[6]  Raimar J. Scherer,et al.  Construction processes configuration using process patterns , 2012, Adv. Eng. Informatics.

[7]  Xiangyu Wang,et al.  Compatibility issues in Augmented Reality systems for AEC: An experimental prototype study , 2006 .

[8]  Parth Rajesh Desai,et al.  A Review Paper on Oculus Rift-A Virtual Reality Headset , 2014, ArXiv.

[9]  Hironao Yamada,et al.  Haptic interaction in tele-operation control system of construction robot based on virtual reality , 2009, 2009 International Conference on Mechatronics and Automation.

[10]  Phillip S. Dunston,et al.  Identification of application areas for Augmented Reality in industrial construction based on technology suitability , 2008 .

[11]  Jonathan Kofman,et al.  Teleoperation of a robot manipulator using a vision-based human-robot interface , 2005, IEEE Transactions on Industrial Electronics.

[12]  William B. Thompson,et al.  Disparity Analysis of Images , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Fredrik Rehnmark,et al.  Robonaut: A Robot Designed to Work with Humans in Space , 2003, Auton. Robots.

[14]  Hironao Yamada,et al.  Construction Tele-robot System With Virtual Reality , 2008, 2008 IEEE Conference on Robotics, Automation and Mechatronics.

[15]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[16]  Richard I. Hartley,et al.  In defence of the 8-point algorithm , 1995, Proceedings of IEEE International Conference on Computer Vision.

[17]  Olivier D. Faugeras,et al.  The fundamental matrix: Theory, algorithms, and stability analysis , 2004, International Journal of Computer Vision.

[18]  Masami Kobayashi,et al.  Telexistence cockpit for humanoid robot control , 2003, Adv. Robotics.

[19]  Hironao Yamada,et al.  Development of a Hydraulic Tele-Operated Construction Robot using Virtual Reality , 2003 .

[20]  Georgios Ch. Sirakoulis,et al.  Real-Time Stereo Vision Applications , 2010 .

[21]  Martin Fischer,et al.  Areas of Application for 3D and 4D Models on Construction Projects , 2008 .

[22]  Andrew W. Fitzgibbon,et al.  Learning epipolar geometry from image sequences , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[23]  Javier Irizarry,et al.  InfoSPOT: A mobile Augmented Reality method for accessing building information through a situation awareness approach , 2013 .

[24]  Raul Wirz,et al.  A multimodal interface to control a robot arm via the web: a case study on remote programming , 2005, IEEE Transactions on Industrial Electronics.

[25]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.