Status quo and open challenges in vision-based sensing and tracking of temporary resources on infrastructure construction sites

Display Omitted Potential impact of data collection and processing in construction is highlighted.Benefits and current limitations of vision-based sensors is explained.Vision-based sensing framework from raw data to information to knowledge presented.Various applications of vision-based resource detection and tracking are shown.Challenges in academic research and development environment are identified. Modern construction projects require sufficient planning and management of resources to become successful. Core issues are tasks that deal with maintaining the schedule, such as procuring materials, guaranteeing the supply chain, controlling the work status, and monitoring safety and quality. Timely feedback of project status aids project management by providing accurate percentages of task completions and appropriately allocating resources (workforce, equipment, material) to coordinate the next work packages. However, current methods for measuring project status or progress, especially on large infrastructure projects, are mostly based on manual assessments. Recent academic research and commercial development has focused on semi- or fully-automated approaches to collect and process images of evolving worksites. Preliminary results are promising and show capturing, analyzing, and documenting construction progress and linking to information models is possible. This article presents first an overview to vision-based sensing technology available for temporary resource tracking at infrastructure construction sites. Second, it provides the status quo of research applications by highlighting exemplary case. Third, a discussion follows on existing advantages and current limitations of vision based sensing and tracking. Open challenges that need to be addressed in future research efforts conclude this paper.

[1]  Ioannis Brilakis,et al.  Comparative study of vision tracking methods for tracking of construction site resources , 2011 .

[2]  David Gerónimo Gómez,et al.  Computer Vision Approaches to Pedestrian Detection: Visible Spectrum Survey , 2007, IbPRIA.

[3]  Fei Dai,et al.  Comparison of Image-Based and Time-of-Flight-Based Technologies for Three-Dimensional Reconstruction of Infrastructure , 2013 .

[4]  Julio C. Martinez,et al.  Visualizing Simulated Construction Operations in 3D , 2001 .

[5]  Jochen Teizer,et al.  Coarse head pose estimation of construction equipment operators to formulate dynamic blind spots , 2012, Adv. Eng. Informatics.

[6]  Burcu Akinci,et al.  Tracking and locating components in a precast storage yard utilizing radio frequency identification technology and GPS , 2007 .

[7]  Rachid Deriche,et al.  A Review of Statistical Approaches to Level Set Segmentation: Integrating Color, Texture, Motion and Shape , 2007, International Journal of Computer Vision.

[8]  Zhongke Shi,et al.  Vision-Based Crane Tracking for Understanding Construction Activity , 2011 .

[9]  Jochen Teizer,et al.  Range Imaging as Emerging Optical Three-Dimension Measurement Technology , 2007 .

[10]  Jochen Teizer,et al.  Automatic design and planning of scaffolding systems using building information modeling , 2014, Adv. Eng. Informatics.

[11]  Jochen Teizer,et al.  Global Positioning System Data to Model and Visualize Workspace Density in Construction Safety Planning , 2015 .

[12]  SangHyun Lee and Feniosky Peña-Mora Visualization Of Construction Progress Monitoring , 2006 .

[13]  Stefano Soatto,et al.  Relevant Feature Selection for Human Pose Estimation and Localization in Cluttered Images , 2008, ECCV.

[14]  Shai Avidan,et al.  Ensemble Tracking , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Luc Van Gool,et al.  From images to 3D models , 2002, CACM.

[16]  Juan Carlos Niebles,et al.  Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers , 2013, Adv. Eng. Informatics.

[17]  Zhongke Shi,et al.  Vision-Based Tower Crane Tracking for Understanding Construction Activity , 2014, J. Comput. Civ. Eng..

[18]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[19]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[20]  Jochen Teizer,et al.  RFID and BIM-enabled worker location tracking to support real-time building protocol and data visualization , 2015, J. Inf. Technol. Constr..

[21]  Frédéric Bosché,et al.  Automated recognition of 3D CAD model objects in laser scans and calculation of as-built dimensions for dimensional compliance control in construction , 2010, Adv. Eng. Informatics.

[22]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[23]  Zhongke Shi,et al.  Tracking multiple workers on construction sites using video cameras , 2010, Adv. Eng. Informatics.

[24]  Tarek Zayed,et al.  Cell-based representation and analysis of spatial resources in construction simulation , 2007 .

[25]  Patricio A. Vela,et al.  A Comprehensive Methodology for Vision-Based Progress and Activity Estimation of Excavation Processes for Productivity Assessment , 2014 .

[26]  Wen Gao,et al.  Group-sensitive multiple kernel learning for object categorization , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[27]  Deva Ramanan,et al.  Local Distance Functions: A Taxonomy, New Algorithms, and an Evaluation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  L. Gool,et al.  From Images to 3 D Models How computers can automatically build realistic 3 D models from images acquired with a hand-held camera , 2002 .

[29]  Jochen Teizer,et al.  Automatic spatio-temporal analysis of construction site equipment operations using GPS data , 2013 .

[30]  Per-Erik Forssén,et al.  Maximally Stable Colour Regions for Recognition and Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Jochen Teizer,et al.  Real-time construction worker posture analysis for ergonomics training , 2012, Adv. Eng. Informatics.

[32]  Ioannis Brilakis,et al.  Automated Detection of Concrete Columns from Visual Data , 2009 .

[33]  Rita Cucchiara,et al.  Covariance descriptors on moving regions for human detection in very complex outdoor scenes , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[34]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[35]  Zhongke Shi,et al.  A performance evaluation of vision and radio frequency tracking methods for interacting workforce , 2011, Adv. Eng. Informatics.

[36]  Patricio A. Vela,et al.  Non-rigid object localization and segmentation using eigenspace representation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[37]  Silvio Savarese,et al.  Application of D4AR - A 4-Dimensional augmented reality model for automating construction progress monitoring data collection, processing and communication , 2009, J. Inf. Technol. Constr..

[38]  Kamel S. Saidi,et al.  Static and dynamic performance evaluation of a commercially-available ultra wideband tracking system , 2011 .

[39]  Janaka Y. Ruwanpura,et al.  Automated Multiple Objects Tracking System (AMOTS) , 2010 .

[40]  Jochen Teizer,et al.  An information fusion approach for filtering GNSS data sets collected during construction operations , 2014, Adv. Eng. Informatics.

[41]  Frédéric Bosché,et al.  Automated progress tracking using 4D schedule and 3D sensing technologies , 2012 .

[42]  Tobias Bjerregaard,et al.  A survey of research and practices of Network-on-chip , 2006, CSUR.

[43]  Ioannis Brilakis,et al.  Detection of large-scale concrete columns for automated bridge inspection , 2010 .

[44]  Mani Golparvar-Fard,et al.  Evaluation of image-based modeling and laser scanning accuracy for emerging automated performance monitoring techniques , 2011 .

[45]  Carlos H. Caldas,et al.  Framework for Real-Time Three-Dimensional Modeling of Infrastructure , 2005 .

[46]  O. Salem,et al.  Application of lean manufacturing principles to construction , 2005 .

[47]  Jochen Teizer,et al.  Mobile 3D mapping for surveying earthwork projects using an Unmanned Aerial Vehicle (UAV) system , 2014 .

[48]  Robert T. Ratay Temporary Structures in Construction Operations , 1987 .

[49]  Markus König,et al.  Drywall State Detection in Image Data for Automatic Indoor Progress Monitoring , 2014 .

[50]  Frédéric Bosché,et al.  Rapid human-assisted, obstacle avoidance system using sparse range point clouds , 2004 .

[51]  Dong Zhai,et al.  Relationship between Automation and Integration of Construction Information Systems and Labor Productivity , 2009 .

[52]  Patricio A. Vela,et al.  A Sparsity‐Inducing Optimization‐Based Algorithm for Planar Patches Extraction from Noisy Point‐Cloud Data , 2015, Comput. Aided Civ. Infrastructure Eng..

[53]  Mani Golparvar-Fard,et al.  Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors , 2013 .

[54]  Lucio Soibelman,et al.  Shape-Based Retrieval of Construction Site Photographs , 2008 .

[55]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[56]  Yogesh Rathi,et al.  Multi-Object Tracking Through Clutter Using Graph Cuts , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[57]  Patricio A. Vela,et al.  Personnel tracking on construction sites using video cameras , 2009, Adv. Eng. Informatics.

[58]  Patricio A. Vela,et al.  Performance evaluation of ultra wideband technology for construction resource location tracking in harsh environments , 2011 .

[59]  Gregory Hager,et al.  Multiple kernel tracking with SSD , 2004, CVPR 2004.

[60]  Burcu Akinci,et al.  Combining Reality Capture Technologies for Construction Defect Detection: A Case Study , 2003 .

[61]  Tao Cheng,et al.  Automated task-level activity analysis through fusion of real time location sensors and worker's tho , 2013 .

[62]  Jeffrey S. Bohn,et al.  Construction Project Monitoring Using High-Resolution Automated Cameras , 2010 .

[63]  Patricio A. Vela,et al.  A Probabilistic Contour Observer for Online Visual Tracking , 2010, SIAM J. Imaging Sci..

[64]  Zixiao Zhang,et al.  Four-dimensional and vision-based framework for infrastructure maintenance management , 2009 .

[65]  Ming Yang,et al.  Multiple Collaborative Kernel Tracking , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Steve Branson,et al.  Similarity metrics for categorization: From monolithic to category specific , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[67]  Jun Wang,et al.  Geotechnical and safety protective equipment planning using range point cloud data and rule checking in building information modeling , 2015 .

[68]  Tao Cheng,et al.  Modeling Tower Crane Operator Visibility to Minimize the Risk of Limited Situational Awareness , 2014 .

[69]  Soon-Wook Kwon,et al.  Fitting range data to primitives for rapid local 3D modeling using sparse range point clouds , 2004 .

[70]  Shai Avidan Ensemble Tracking , 2007, IEEE Trans. Pattern Anal. Mach. Intell..

[71]  Jie Gong,et al.  An object recognition, tracking, and contextual reasoning-based video interpretation method for rapid productivity analysis of construction operations , 2011 .

[72]  Andrey Dimitrov,et al.  Vision-based material recognition for automated monitoring of construction progress and generating building information modeling from unordered site image collections , 2014, Adv. Eng. Informatics.

[73]  Jason J. Ford,et al.  Vision Based Control for Fixed Wing UAVs Inspecting Locally Linear Infrastructure Using Skid-to-Turn Maneuvers , 2011, J. Intell. Robotic Syst..

[74]  Daniel Cremers Nonlinear Dynamical Shape Priors for Level Set Segmentation , 2007, CVPR.

[75]  Donald Washburn,et al.  Knowledge-Based Segmentation for Tracking Through Deep Turbulence , 2008, IEEE Transactions on Control Systems Technology.

[76]  R. Navon,et al.  Research in automated measurement of project performance indicators , 2007 .

[77]  Tao Cheng,et al.  Location tracking and data visualization technology to advance construction ironworkers' education and training in safety and productivity , 2013 .

[78]  Tao Cheng,et al.  Real-time resource location data collection and visualization technology for construction safety and activity monitoring applications , 2013 .

[79]  Jochen Teizer,et al.  Computing 3D blind spots of construction equipment: Implementation and evaluation of an automated measurement and visualization method utilizing range point cloud data , 2013 .

[80]  Manu Venugopal,et al.  Ultrawideband for Automated Real-Time Three-Dimensional Location Sensing for Workforce, Equipment, and Material Positioning and Tracking , 2008 .

[81]  John E. Hummel Object Recognition , 2014, Computer Vision, A Reference Guide.

[82]  Burcu Akinci,et al.  Tracking Components and Maintenance History within a Facility Utilizing Radio Frequency Identification Technology , 2007 .

[83]  Jochen Teizer,et al.  Leveraging passive RFID technology for construction resource field mobility and status monitoring in a high-rise renovation project , 2012 .

[84]  Janaka Y. Ruwanpura,et al.  AUTOMATED DATA ACQUISITION SYSTEM TO ASSESS CONSTRUCTION WORKER PERFORMANCE , 2009 .

[85]  Luc Van Gool,et al.  Robust tracking-by-detection using a detector confidence particle filter , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[86]  Dong Zhai,et al.  Relationship between Changes in Material Technology and Construction Productivity , 2009 .

[87]  Frédéric Bosché,et al.  Automated retrieval of 3D CAD model objects in construction range images , 2008 .

[88]  Rustam Stolkin,et al.  Continuous Machine Learning in Computer Vision - Tracking with Adaptive Class Models , 2007 .

[89]  Burcu Akinci,et al.  Data Fusion Approaches and Applications for Construction Engineering , 2011 .

[90]  J. Teizer,et al.  Madras RAPID AUTOMATED MONITORING OF CONSTRUCTION SITE ACTIVITIES USING ULTRA-WIDEBAND , 2007 .

[91]  Burcu Akinci,et al.  Formalization and Automation of Time-Space Conflict Analysis , 2002 .

[92]  Paul M. Goodrum,et al.  Assessing the impact of materials tracking technologies on construction craft productivity , 2009 .

[93]  Carlos H. Caldas,et al.  Real-Time Three-Dimensional Occupancy Grid Modeling for the Detection and Tracking of Construction Resources , 2007 .

[94]  Vineet R. Kamat,et al.  Large-Scale Dynamic Terrain in Three-Dimensional Construction Process Visualizations , 2005 .