Fusion of Photogrammetry and Video Analysis for Productivity Assessment of Earthwork Processes

The high complexity of modern large-scale construction projects leads their schedules to be sensitive to delays. At underground construction sites, the earthwork processes are vital, as most of the following tasks depend on them. This article presents a method for estimating the productivity of soil removal by combining two technologies based on computer vision: photogrammetry and video analysis. Photogrammetry is applied to create a time series of point clouds throughout excavation, which are used to measure the volume of the excavated soil for daily estimates of productivity. Video analysis is used to generate statistics regarding the construction activities for estimating productivity at finer time scales, when combined with the output from the photogrammetry pipeline. As there may be multiple causes for specific productivity levels, the automated generation of progress and activity statistics from both measurement methods supports interpretation of the productivity estimates. Comparison to annotated ground truth for the tracking and activity monitoring method highlights the reliability of the extracted information. The suitability of the approach is demonstrated by two case studies of real-world urban excavation projects.

[1]  Frédéric Bosché,et al.  Automated recognition of 3D CAD model objects in laser scans and calculation of as-built dimensions for dimensional compliance control in construction , 2010, Adv. Eng. Informatics.

[2]  Janaka Y. Ruwanpura,et al.  Automated Multiple Objects Tracking System (AMOTS) , 2010 .

[3]  Jochen Teizer,et al.  An information fusion approach for filtering GNSS data sets collected during construction operations , 2014, Adv. Eng. Informatics.

[4]  Uwe Stilla,et al.  A concept for automated construction progress monitoring using BIM-based geometric constraints and photogrammetric point clouds , 2015, J. Inf. Technol. Constr..

[5]  Amin Hammad,et al.  Framework for near real-time simulation of earthmoving projects using location tracking technologies , 2014 .

[6]  Mani Golparvar-Fard,et al.  Real-Time and Automated Recognition and 2D Tracking of Construction Workers and Equipment from Site Video Streams , 2012 .

[7]  Jochen Teizer,et al.  Mobile 3D mapping for surveying earthwork projects using an Unmanned Aerial Vehicle (UAV) system , 2014 .

[8]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Ioannis Brilakis,et al.  Enhancement of Construction Equipment Detection in Video Frames by Combining with Tracking , 2012 .

[10]  Jerome F. Hajjar,et al.  Data Processing of Point Clouds for Object Detection for Structural Engineering Applications , 2013, Comput. Aided Civ. Infrastructure Eng..

[11]  Paul M. Goodrum,et al.  Assessing the impact of materials tracking technologies on construction craft productivity , 2009 .

[12]  Jason J. Ford,et al.  Vision Based Control for Fixed Wing UAVs Inspecting Locally Linear Infrastructure Using Skid-to-Turn Maneuvers , 2011, J. Intell. Robotic Syst..

[13]  Ioannis Brilakis,et al.  Initializing Vision Based Trackers Using Semantic Texton Forests , 2011 .

[14]  David A. Forsyth,et al.  ConstructAide: analyzing and visualizing construction sites through photographs and building models , 2014, ACM Trans. Graph..

[15]  Zhongke Shi,et al.  Automatic Recognition of Construction Worker Activities Using Dense Trajectories , 2015 .

[16]  Patricio A. Vela,et al.  Personnel tracking on construction sites using video cameras , 2009, Adv. Eng. Informatics.

[17]  Sadi Assaf,et al.  Causes of delay in large construction projects , 2006 .

[18]  Mani Golparvar-Fard,et al.  Evaluation of image-based modeling and laser scanning accuracy for emerging automated performance monitoring techniques , 2011 .

[19]  Carlos H. Caldas,et al.  Framework for Real-Time Three-Dimensional Modeling of Infrastructure , 2005 .

[20]  Dong Zhai,et al.  Relationship between Automation and Integration of Construction Information Systems and Labor Productivity , 2009 .

[21]  Paul M. Goodrum,et al.  Activity Analysis for Direct-Work Rate Improvement in Construction , 2011 .

[22]  Jeffrey S. Bohn,et al.  Construction Project Monitoring Using High-Resolution Automated Cameras , 2010 .

[23]  Ioannis K. Brilakis,et al.  A videogrammetric as-built data collection method for digital fabrication of sheet metal roof panels , 2013, Adv. Eng. Informatics.

[24]  Abhinav Peddi.  Development of human pose analyzing algorithms for the determination of construction productivity in real-time , 2009 .

[25]  Christoph van Treeck,et al.  Integrating Feedback from Image based 3D Reconstruction into Models of Excavation Processes for efficient Scheduling , 2013 .

[26]  Brenda McCabe,et al.  Automated Visual Recognition of Dump Trucks in Construction Videos , 2012, J. Comput. Civ. Eng..

[27]  Jie Gong,et al.  An intelligent video computing method for automated productivity analysis of cyclic construction operations , 2009 .

[28]  Juan Carlos Niebles,et al.  Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers , 2013, Adv. Eng. Informatics.

[29]  Mani Golparvar-Fard,et al.  Automated Worker Activity Analysis in Indoor Environments for Direct-Work Rate Improvement from long sequences of RGB-D Images , 2014 .

[30]  Ioannis K. Brilakis,et al.  Automated vision tracking of project related entities , 2011, Adv. Eng. Informatics.

[31]  Seokho Chi,et al.  Automated Object Identification Using Optical Video Cameras on Construction Sites , 2011, Comput. Aided Civ. Infrastructure Eng..

[32]  Kamel S. Saidi,et al.  Static and dynamic performance evaluation of a commercially-available ultra wideband tracking system , 2011 .

[33]  Alexander G. Gray,et al.  Reduced Set KPCA for Improving the Training and Execution Speed of Kernel Machines , 2013, SDM.

[34]  Yinhai Wang,et al.  Model‐Free Video Detection and Tracking of Pedestrians and Bicyclists , 2009, Comput. Aided Civ. Infrastructure Eng..

[35]  M. Paul Lewis,et al.  A Comprehensive Methodology , 1995 .

[36]  Takeo Kanade,et al.  Introduction to the Special Section on Video Surveillance , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Mubarak Shah,et al.  KNIGHT/spl trade/: a real time surveillance system for multiple and non-overlapping cameras , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[38]  Jochen Teizer,et al.  Automatic spatio-temporal analysis of construction site equipment operations using GPS data , 2013 .

[39]  Jochen Teizer,et al.  Mobile 3D Mapping for Surveying Earthwork Using an Unmanned Aerial Vehicle (UAV) , 2013 .

[40]  Zhongke Shi,et al.  Tracking multiple workers on construction sites using video cameras , 2010, Adv. Eng. Informatics.

[41]  Enno Koehn,et al.  Cost of Delays in Construction , 1978 .

[42]  Ioannis Brilakis,et al.  Comparative study of vision tracking methods for tracking of construction site resources , 2011 .

[43]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[44]  Mohamed Marzouk,et al.  Tracking construction projects progress using mobile hand-held devices , 2015 .

[45]  Lisa M. Brown,et al.  IBM smart surveillance system (S3): a open and extensible framework for event based surveillance , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[46]  Cevdet Coskun Aydin,et al.  Designing building façades for the urban rebuilt environment with integration of digital close-range photogrammetry and geographical information systems , 2014 .

[47]  Dong Zhai,et al.  Relationship between Changes in Material Technology and Construction Productivity , 2009 .

[48]  Chunsun Zhang,et al.  An Unmanned Aerial Vehicle‐Based Imaging System for 3D Measurement of Unpaved Road Surface Distresses 1 , 2012, Comput. Aided Civ. Infrastructure Eng..

[49]  Tao Cheng,et al.  Automated task-level activity analysis through fusion of real time location sensors and worker's tho , 2013 .

[50]  Kuldeep Kumar,et al.  Robust Statistics, 2nd edn , 2011 .

[51]  Brian C. Lovell,et al.  An efficient and robust sequential algorithm for background estimation in video surveillance , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[52]  Jochen Teizer,et al.  Leveraging passive RFID technology for construction resource field mobility and status monitoring in a high-rise renovation project , 2012 .

[53]  M. Shah,et al.  KNIGHT M : A REAL TIME SURVEILLANCE SYSTEM FOR MULTIPLE OVERLAPPING AND NON-OVERLAPPING CAMERAS , 2003 .

[54]  Brenda McCabe,et al.  Part based model and spatial–temporal reasoning to recognize hydraulic excavators in construction images and videos , 2012 .

[55]  Zhongke Shi,et al.  Vision-Based Crane Tracking for Understanding Construction Activity , 2011 .

[56]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[57]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Jie Gong,et al.  An object recognition, tracking, and contextual reasoning-based video interpretation method for rapid productivity analysis of construction operations , 2011 .

[59]  Lisa M. Brown,et al.  IBM smart surveillance system (S3): event based video surveillance system with an open and extensible framework , 2008, Machine Vision and Applications.

[60]  Andrey Dimitrov,et al.  Vision-based material recognition for automated monitoring of construction progress and generating building information modeling from unordered site image collections , 2014, Adv. Eng. Informatics.

[61]  Ronie Navon,et al.  Automated project performance control of construction projects , 2005 .