Construction performance monitoring via still images, time-lapse photos, and video streams: Now, tomorrow, and the future

Timely and accurate monitoring of onsite construction operations can bring an immediate awareness on project specific issues. It provides practitioners with the information they need to easily and quickly make project control decisions. Despite their importance, the current practices are still time-consuming, costly, and prone to errors. To facilitate the process of collecting and analyzing performance data, researchers have focused on devising methods that can semi-automatically or automatically assess ongoing operations both at project level and operation level. A major line of work has particularly focused on developing computer vision techniques that can leverage still images, time-lapse photos and video streams for documenting the work in progress. To this end, this paper extensively reviews these state-of-the-art vision-based construction performance monitoring methods. Based on the level of information perceived and the types of output, these methods are mainly divided into two categories (namely project level: visual monitoring of civil infrastructure or building elements vs. operation level: visual monitoring of construction equipment and workers). The underlying formulations and assumptions used in these methods are discussed in detail. Finally the gaps in knowledge that need to be addressed in future research are identified.

[1]  Paul M. Goodrum,et al.  Activity Analysis for Direct-Work Rate Improvement in Construction , 2011 .

[2]  Mani Golparvar-Fard,et al.  Automated Worker Activity Analysis in Indoor Environments for Direct-Work Rate Improvement from long sequences of RGB-D Images , 2014 .

[3]  Juan Carlos Niebles,et al.  Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers , 2013, Adv. Eng. Informatics.

[4]  Liang Y Liu,et al.  D4ar- 4 dimensional augmented reality - models or automation and interactive visualization of construction progress monitoring , 2010 .

[5]  J. Borenstein,et al.  Integration of Global Positioning System and Inertial Navigation for Ubiquitous Context-Aware Engineering Applications , 2010 .

[6]  Zhongke Shi,et al.  Vision-Based Tower Crane Tracking for Understanding Construction Activity , 2014, J. Comput. Civ. Eng..

[7]  Changmin Kim,et al.  Automated construction progress measurement using a 4D building information model and 3D data , 2013 .

[8]  Burcu Akinci,et al.  Automated Recognition of 3D CAD Objects in Site Laser Scans for Project 3D Status Visualization and Performance Control , 2009 .

[9]  Ioannis K. Brilakis,et al.  Automated vision tracking of project related entities , 2011, Adv. Eng. Informatics.

[10]  Jie Gong,et al.  An intelligent video computing method for automated productivity analysis of cyclic construction operations , 2009 .

[11]  Seung Heon Han,et al.  Object recognition in construction-site images using 3D CAD-based filtering , 2010 .

[12]  SangUk Han,et al.  A vision-based motion capture and recognition framework for behavior-based safety management , 2013 .

[13]  Zhongke Shi,et al.  Tracking multiple workers on construction sites using video cameras , 2010, Adv. Eng. Informatics.

[14]  John M. Kamara,et al.  A framework for using mobile computing for information management on construction sites , 2011 .

[15]  Erez N. Allouche,et al.  PHOTO-NET II: a computer-based monitoring system applied to project management , 2003 .

[16]  Abhinav Peddi Development of human pose analyzing algorithms for the determination of construction productivity in real-time , 2009 .

[17]  Brenda McCabe,et al.  Automated Visual Recognition of Dump Trucks in Construction Videos , 2012, J. Comput. Civ. Eng..

[18]  Ioannis Brilakis,et al.  Construction worker detection in video frames for initializing vision trackers , 2012 .

[19]  Jiri Matas,et al.  P-N learning: Bootstrapping binary classifiers by structural constraints , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Sven J. Dickinson,et al.  Server-Customer Interaction Tracker: Computer Vision-Based System to Estimate Dirt-Loading Cycles , 2013 .

[21]  Mani Golparvar-Fard,et al.  Automated Visual Recognition of Construction Equipment Actions Using Spatio-Temporal Features and Mu , 2012 .

[22]  Mani Golparvar-Fard,et al.  Image-Based Localization and Content Authoring in Structure-from-Motion Point Cloud Models for Real-Time Field Reporting Applications , 2015 .

[23]  Lucio Soibelman,et al.  Multimodal Image Retrieval from Construction Databases and Model-Based Systems , 2006 .

[24]  Changyoon Kim,et al.  4D CAD model updating using image processing-based construction progress monitoring , 2013 .

[25]  SangHyun Lee,et al.  Computer Vision Techniques for Worker Motion Analysis to Reduce Musculoskeletal Disorders in Construction , 2011 .

[26]  Mani Golparvar-Fard,et al.  Automated Monitoring of Operation-Level Construction Progress Using 4D BIM and Daily Site Photologs , 2014 .

[27]  Feniosky Peña-Mora,et al.  Visual representation of construction progress monitoring metrics on time-lapse photographs , 2007 .

[28]  Mani Golparvar-Fard,et al.  Vision-based workface assessment using depth images for activity analysis of interior construction operations , 2014 .

[29]  Naruo Kano,et al.  Comparison of construction photograph and VR image in construction progress , 2008 .

[30]  X. Zhang,et al.  Towards automated progress assessment of workpackage components in construction projects using computer vision , 2009, Adv. Eng. Informatics.

[31]  Frédéric Bosché,et al.  Tracking the Built Status of MEP Works: Assessing the Value of a Scan-vs-BIM System , 2014, J. Comput. Civ. Eng..

[32]  Fei Dai,et al.  Remote Sensing Enabling Technologies for Assessment of Construction Worker’s Musculoskeletal Disorder Risks: A Review and Future Extension , 2013 .

[33]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[34]  Seokho Chi,et al.  Image-Based Safety Assessment: Automated Spatial Safety Risk Identification of Earthmoving and Surface Mining Activities , 2012 .

[35]  Ioannis Brilakis,et al.  Comparative study of vision tracking methods for tracking of construction site resources , 2011 .

[36]  Frédéric Bosché,et al.  Automated retrieval of 3D CAD model objects in construction range images , 2008 .

[37]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[38]  Markus König,et al.  Drywall State Detection in Image Data for Automatic Indoor Progress Monitoring , 2014 .

[39]  Lucio Soibelman,et al.  Material-Based Construction Site Image Retrieval , 2005 .

[40]  Danijel Rebolj,et al.  Automated construction activity monitoring system , 2008, Adv. Eng. Informatics.

[41]  Silvio Savarese,et al.  Application of D4AR - A 4-Dimensional augmented reality model for automating construction progress monitoring data collection, processing and communication , 2009, J. Inf. Technol. Constr..

[42]  Mani Golparvar-Fard,et al.  Automated Vision-based Recognition of Construction Worker Actions for Building Interior Construction , 2012 .

[43]  Seokho Chi,et al.  Automated Object Identification Using Optical Video Cameras on Construction Sites , 2011, Comput. Aided Civ. Infrastructure Eng..

[44]  Janaka Y. Ruwanpura,et al.  Automated Multiple Objects Tracking System (AMOTS) , 2010 .

[45]  Jochen Teizer,et al.  An information fusion approach for filtering GNSS data sets collected during construction operations , 2014, Adv. Eng. Informatics.

[46]  E. Rezazadeh Azar,et al.  A Visual Sensing Approach to Estimate Material Hauling Cycles in Heavy Construction and Surface Mining Jobsites , 2013 .

[47]  Mani Golparvar-Fard,et al.  Visualization of construction progress monitoring with 4D simulation model overlaid on time-lapsed photographs , 2009 .

[48]  Mani Golparvar-Fard,et al.  Automated Benchmarking and Monitoring of an Earthmoving Operation's Carbon Footprint Using Video Cameras and a Greenhouse Gas Estimation Model , 2012 .

[49]  Jie Gong,et al.  An object recognition, tracking, and contextual reasoning-based video interpretation method for rapid productivity analysis of construction operations , 2011 .

[50]  Emanuele Trucco,et al.  Towards Automated Visual Assessment of Progress in Construction Projects , 2007, BMVC.

[51]  Mani Golparvar-Fard,et al.  Interactive Visual Construction Progress Monitoring with D 4 AR — 4D Augmented Reality — Models , 2009 .

[52]  Andrey Dimitrov,et al.  Vision-based material recognition for automated monitoring of construction progress and generating building information modeling from unordered site image collections , 2014, Adv. Eng. Informatics.

[53]  Mani Golparvar-Fard,et al.  Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors , 2013 .

[54]  Jochen Teizer,et al.  Benefits and Barriers of Monitoring Construction Activities Using Hi-Resolution Automated Cameras , 2009 .

[55]  E. Ergen,et al.  An Overview of Approaches for Utilizing RFID in Construction Industry , 2007, 2007 1st Annual RFID Eurasia.

[56]  Carlos H. Caldas,et al.  Vision-based action recognition in the internal construction site using interactions between worker actions and construction objects , 2013 .

[57]  Feniosky Peña-Mora,et al.  Application of Visualization Techniques for Construction Progress Monitoring , 2007 .

[58]  Mani Golparvar-Fard,et al.  Real-Time and Automated Recognition and 2D Tracking of Construction Workers and Equipment from Site Video Streams , 2012 .

[59]  SangUk Han,et al.  A Machine-Learning Classification Approach to Automatic Detection of Workers' Actions for Behavior-Based Safety Analysis , 2012 .

[60]  JoonOh Seo,et al.  A Stereo Vision-based Approach to Marker-less Motion Capture for On-Site Kinematic Modeling of Construction Worker Tasks , 2014 .

[61]  Frédéric Bosché,et al.  Tracking Secondary and Temporary Concrete Construction Objects Using 3D Imaging Technologies , 2013 .

[62]  Mani Golparvar-Fard,et al.  High-precision vision-based mobile augmented reality system for context-aware architectural, engineering, construction and facility management (AEC/FM) applications , 2013 .

[63]  Mani Golparvar-Fard,et al.  Multi-Sample Image-Based Material Recognition and Formalized Sequencing Knowledge for Operation-Level Construction Progress Monitoring , 2014 .

[64]  Mani Golparvar-Fard,et al.  Appearance-based material classification for monitoring of operation-level construction progress using 4D BIM and site photologs , 2015 .

[65]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[66]  Ioannis Brilakis,et al.  Progressive 3D reconstruction of infrastructure with videogrammetry , 2011 .

[67]  Jochen Teizer,et al.  Real-time construction worker posture analysis for ergonomics training , 2012, Adv. Eng. Informatics.

[68]  Ioannis Brilakis,et al.  Automated Detection of Concrete Columns from Visual Data , 2009 .

[69]  Horst Bischof,et al.  Augmented Reality for Construction Site Monitoring and Documentation , 2014, Proceedings of the IEEE.

[70]  T. Tsubouchi,et al.  Stereo Vision Based Worker Detection System for Excavator , 2013 .

[71]  David A. Forsyth,et al.  ConstructAide: analyzing and visualizing construction sites through photographs and building models , 2014, ACM Trans. Graph..

[72]  Mani Golparvar-Fard,et al.  Automated Progress Monitoring Using Unordered Daily Construction Photographs and IFC-Based Building Information Models , 2015, J. Comput. Civ. Eng..

[73]  Hyoungkwan Kim,et al.  Using Hue, Saturation, and Value Color Space for Hydraulic Excavator Idle Time Analysis , 2007 .

[74]  Jimmie Hinze,et al.  Autonomous pro-active real-time construction worker and equipment operator proximity safety alert system , 2010 .

[75]  H. Son,et al.  Automated Color Model-Based Concrete Detection in Construction-Site Images by Using Machine Learning Algorithms , 2012, J. Comput. Civ. Eng..

[76]  Ayman Habib,et al.  Application of Microsoft Kinect Sensor for Tracking Construction Workers , 2012 .

[77]  Rafael Sacks,et al.  Assessing research issues in Automated Project Performance Control (APPC) , 2007 .

[78]  Patricio A. Vela,et al.  Automated Trajectory and Path Planning Analysis Based on Ultra Wideband Data , 2012, J. Comput. Civ. Eng..

[79]  Brenda McCabe,et al.  Part based model and spatial–temporal reasoning to recognize hydraulic excavators in construction images and videos , 2012 .

[80]  Dong Zhai,et al.  Model to Predict the Impact of a Technology on Construction Productivity , 2011 .

[81]  Manolis I. A. Lourakis,et al.  Toward automated generation of parametric BIMs based on hybrid video and laser scanning data , 2010, Adv. Eng. Informatics.

[82]  John E. Hummel Object Recognition , 2014, Computer Vision, A Reference Guide.

[83]  Jochen Teizer,et al.  Mobile 3D mapping for surveying earthwork projects using an Unmanned Aerial Vehicle (UAV) system , 2014 .

[84]  Christian Koch,et al.  Three-Dimensional Tracking of Construction Resources Using an On-Site Camera System , 2012, J. Comput. Civ. Eng..

[85]  Feniosky Peña-Mora,et al.  Integrated Sequential As-Built and As-Planned Representation with D4AR Tools in Support of Decision-Making Tasks in the AEC/FM Industry , 2011 .

[86]  Ioannis Brilakis,et al.  Enhancement of Construction Equipment Detection in Video Frames by Combining with Tracking , 2012 .

[87]  Patricio A. Vela,et al.  Personnel tracking on construction sites using video cameras , 2009, Adv. Eng. Informatics.

[88]  Patricio A. Vela,et al.  A Comprehensive Methodology for Vision-Based Progress and Activity Estimation of Excavation Processes for Productivity Assessment , 2014 .

[89]  Chao Wang,et al.  Smart scanning and near real-time 3D surface modeling of dynamic construction equipment from a point cloud , 2015 .

[90]  Zhongke Shi,et al.  A performance evaluation of vision and radio frequency tracking methods for interacting workforce , 2011, Adv. Eng. Informatics.

[91]  Fei Dai,et al.  Identification of Construction Cumulative Trauma Disorders: A Machine Vision Approach , 2013 .

[92]  Carlos H. Caldas,et al.  Learning and classifying actions of construction workers and equipment using Bag-of-Video-Feature-Words and Bayesian network models , 2011, Adv. Eng. Informatics.

[93]  Carlos H. Caldas,et al.  Application of Sensing Technology in the Prevention of Backing Accidents in Construction Work Zones , 2013 .

[94]  Ioannis Brilakis,et al.  Testing in harsh conditions: Tracking resources on construction sites with machine vision , 2011 .

[95]  Mike Kagioglou,et al.  Automating progress measurement of construction projects , 2009 .

[96]  Arash Shahi,et al.  Onsite 3D marking for construction activity tracking , 2013 .

[97]  Yong K. Cho,et al.  Projection-Recognition-Projection Method for Automatic Object Recognition and Registration for Dynamic Heavy Equipment Operations , 2014 .