Application of dynamic time warping to the recognition of mixed equipment activities in cycle time measurement

Abstract Analyzing and measuring construction equipment operation are key tasks for managing construction projects. In monitoring construction equipment operation, the cycle-time provides fundamental information. Traditional cycle-time measurement methods have been limited by requiring significant efforts such as additional observers, time, and cost. Thus, this study investigates the feasibility of measuring cycle times by using inertial measurement units (IMUs) embedded in a smartphone. Because the mixed activities of construction equipment involve simultaneous actions of multiple parts, they cause low accuracy in equipment activity classification and cycle-time measurement. To enhance the recognition of these mixed activities and translate the results into reliable cycle time measurements, a dynamic time warping (DTW) algorithm was applied and the DTW distances of IMU signals were used as additional features in activity classification. To test its feasibility, data was collected on-site and the excavator's operation was recorded via IMUs embedded in a smartphone attached to a cabin. Using DTW, the suggested method achieved 91.83% accuracy for cycle-time measurement. This result demonstrates an opportunity to use operators' prevalent mobile devices to measure and report their equipment's cycle times in a cost-effective and continuous manner.

[1]  Bob McCulloch Automating Field Data Collection in Construction Organizations , 1997 .

[2]  Lingguang Song,et al.  Adaptive real-time tracking and simulation of heavy construction operations for look-ahead scheduling , 2012 .

[3]  Fengshou Gu,et al.  Phase-compensation-based dynamic time warping for fault diagnosis using the motor current signal , 2012 .

[4]  Mehmet C. Vuran,et al.  Semi-supervised near-miss fall detection for ironworkers with a wearable inertial measurement unit , 2016 .

[5]  Gabi Nakibly,et al.  Gyrophone: Recognizing Speech from Gyroscope Signals , 2014, USENIX Security Symposium.

[6]  Amir H. Behzadan,et al.  Construction equipment activity recognition for simulation input modeling using mobile sensors and machine learning classifiers , 2015, Adv. Eng. Informatics.

[7]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[8]  Aaron E. Rosenberg,et al.  Performance tradeoffs in dynamic time warping algorithms for isolated word recognition , 1980 .

[9]  Hyoungkwan Kim,et al.  Using Hue, Saturation, and Value Color Space for Hydraulic Excavator Idle Time Analysis , 2007 .

[10]  Amr A. Oloufa,et al.  Situational awareness of construction equipment using GPS, wireless and web technologies , 2003 .

[11]  SangHyun Lee,et al.  Automated cycle time measurement and analysis of excavator's loading operation using smart phone-embedded IMU sensors , 2015 .

[12]  Peter E.D. Love,et al.  A computational intelligent fuzzy model approach for excavator cycle time simulation , 2003 .

[13]  Carl T. Haas,et al.  Action Recognition Using a Wristband-Type Activity Tracker: Case Study of Masonry Work , 2016 .

[14]  Fengshou Gu,et al.  Fault diagnosis of motor drives using stator current signal analysis based on dynamic time warping , 2013 .

[15]  Arman Melkumyan,et al.  A Dynamic Time Warping based covariance function for Gaussian Processes signature identification , 2016, Comput. Geosci..

[16]  Paul Lukowicz,et al.  Recognizing Workshop Activity Using Body Worn Microphones and Accelerometers , 2004, Pervasive.

[17]  Koshy Varghese,et al.  Accelerometer-Based Activity Recognition in Construction , 2011, J. Comput. Civ. Eng..

[18]  Feniosky Peña-Mora,et al.  Acceleromter-Based Measurement of Construction Equipment Operating Efficiency for Monitoring Environmental Performance , 2013 .

[19]  Jie Gong,et al.  Computer Vision-Based Video Interpretation Model for Automated Productivity Analysis of Construction Operations , 2010 .

[20]  Kang G. Shin,et al.  Invisible Sensing of Vehicle Steering with Smartphones , 2015, MobiSys.

[21]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[22]  Jimmie Hinze,et al.  Autonomous pro-active real-time construction worker and equipment operator proximity safety alert system , 2010 .

[23]  Feniosky Peña-Mora,et al.  Application of Low-Cost Accelerometers for Measuring the Operational Efficiency of a Construction Equipment Fleet , 2015, J. Comput. Civ. Eng..

[24]  Tao Cheng,et al.  Automated task-level activity analysis through fusion of real time location sensors and worker's tho , 2013 .

[25]  Amin Hammad,et al.  Optimization-based excavator pose estimation using real-time location systems , 2015 .

[26]  H. W. Parker,et al.  Productivity improvement in construction , 1988 .

[27]  Jie Gong,et al.  An object recognition, tracking, and contextual reasoning-based video interpretation method for rapid productivity analysis of construction operations , 2011 .

[28]  Mehmet C. Vuran,et al.  Collective sensing of workers’ gait patterns to identify fall hazards in construction , 2017 .

[29]  Amir H. Behzadan,et al.  Knowledge-Based Simulation Modeling of Construction Fleet Operations Using Multimodal-Process Data Mining , 2013 .

[30]  Ronie Navon,et al.  Automated project performance control of construction projects , 2005 .

[31]  Osama Moselhi,et al.  RFID+ for Tracking Earthmoving Operations , 2012 .

[32]  Philip Chan,et al.  Toward accurate dynamic time warping in linear time and space , 2007, Intell. Data Anal..

[33]  Robert C. Williams,et al.  Construction Equipment Telematics , 2011 .

[34]  Jochen Teizer,et al.  Automatic spatio-temporal analysis of construction site equipment operations using GPS data , 2013 .

[35]  Jochen Teizer,et al.  Status quo and open challenges in vision-based sensing and tracking of temporary resources on infrastructure construction sites , 2015, Adv. Eng. Informatics.

[36]  Mani Golparvar-Fard,et al.  Automated Vision-based Recognition of Construction Worker Actions for Building Interior Construction , 2012 .

[37]  Geraldine S. Cheok,et al.  Ladars for construction assessment and update , 2000 .

[38]  Mohan Kumar,et al.  Using dynamic time warping for online temporal fusion in multisensor systems , 2008, Inf. Fusion.

[39]  Roozbeh Jafari,et al.  A Wearable System for Recognizing American Sign Language in Real-Time Using IMU and Surface EMG Sensors , 2016, IEEE Journal of Biomedical and Health Informatics.

[40]  Mark A. Hall,et al.  Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning , 1999, ICML.