Computer vision-based structural assessment exploiting large volumes of images

Author: Yeum, Chul Min. Ph.D. Institution: Purdue University Degree Received: December 2016 Title: Computer Vision-Based Structural Assessment Exploiting Large Volumes of Images. Major Professor: Shirley J. Dyke. Visual assessment is a process to understand the state of a structure based on evaluations originating from visual information. Recent advances in computer vision to explore new sensors, sensing platforms and high-performance computing have shed light on the potential for vision-based visual assessment in civil engineering structures. The use of lowcost, high-resolution visual sensors in conjunction with mobile and aerial platforms can overcome spatial and temporal limitations typically associated with other forms of sensing in civil structures. Also, GPU-accelerated parallel computing offers unprecedented speed and performance, accelerating processing the collected visual data. However, despite the enormous endeavor in past research to implement such technologies, there are still many practical challenges to overcome to successfully apply these techniques in real world situations. A major challenge lies in dealing with a large volume of unordered and complex visual data, collected under uncontrolled circumstance (e.g. lighting, cluttered region, and variations in environmental conditions), while just a tiny fraction of them are useful for conducting actual assessment. Such difficulty induces an undesirable high rate of falsepositive and false-negative errors, reducing the trustworthiness and efficiency of their implementation. To overcome the inherent challenges in using such images for visual assessment, high-level computer vision algorithms must be integrated with relevant prior knowledge and guidance, thus aiming to have similar performance with those of humans conducting visual assessment. Moreover, the techniques must be developed and validated in the realistic context of a large volume of real-world images, which is likely contain numerous practical challenges. In this dissertation, the use of computer vision algorithms is explored to address two promising applications of vision-based visual assessment in civil engineering: visual inspection, and visual data analysis for post-disaster evaluation. For both applications, techniques are developed here to enable reliable and efficient visual

[1]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Kenichi Soga,et al.  Distortion-Free Image Mosaicing for Tunnel Inspection Based on Robust Cylindrical Surface Estimation through Structure from Motion , 2016 .

[4]  Bob J. Wielinga,et al.  Ontology-Based Photo Annotation , 2001, IEEE Intell. Syst..

[5]  James Pustejovsky,et al.  Natural Language Annotation for Machine Learning , 2012 .

[6]  Jonathan Miller,et al.  ROBOTIC SYSTEMS FOR INSPECTION AND SURVEILLANCE OF CIVIL STRUCTURES , 2004 .

[7]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[8]  Yun Liu,et al.  Automatic Crack Detection and Classification Method for Subway Tunnel Safety Monitoring , 2014, Sensors.

[9]  Mete A. Sozen,et al.  Performance of School Buildings in Turkey During the 1999 Düzce and the 2003 Bingöl Earthquakes , 2009 .

[10]  Gaurav S. Sukhatme,et al.  Multi-image stitching and scene reconstruction for evaluating defect evolution in structures , 2011 .

[11]  M. Newman,et al.  Finding community structure in very large networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[12]  Andrea Prota,et al.  The SERIES Virtual Database: Exchange Data Format and Local/Central Databases , 2015 .

[13]  Jitendra Malik,et al.  Region-Based Convolutional Networks for Accurate Object Detection and Segmentation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Luc Van Gool,et al.  Multibody Structure-from-Motion in Practice , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  M. Westoby,et al.  ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications , 2012 .

[16]  Ioannis Brilakis,et al.  Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation , 2011 .

[17]  J. Schanda,et al.  Colorimetry : understanding the CIE system , 2007 .

[18]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[19]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[20]  Steffen Staab,et al.  International Handbooks on Information Systems , 2013 .

[21]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Luh-Maan Chang,et al.  Support-vector-machine-based method for automated steel bridge rust assessment , 2012 .

[23]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[24]  Andrew Curtis,et al.  The clearinghouse concept: a model for geospatial data centralization and dissemination in a disaster. , 2008, Disasters.

[25]  Qingquan Li,et al.  CrackTree: Automatic crack detection from pavement images , 2012, Pattern Recognit. Lett..

[26]  Mohammad Farukh Hashmi,et al.  VISUAL INSPECTION AND CRACK DETECTION OF RAILROAD TRACKS , 2014 .

[27]  Shuji Hashimoto,et al.  Fast crack detection method for large-size concrete surface images using percolation-based image processing , 2010, Machine Vision and Applications.

[28]  Ikhlas Abdel-Qader,et al.  ANALYSIS OF EDGE-DETECTION TECHNIQUES FOR CRACK IDENTIFICATION IN BRIDGES , 2003 .

[29]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[30]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[31]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[32]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[33]  Paul Fieguth,et al.  Automated detection of cracks in buried concrete pipe images , 2006 .

[34]  Mani Golparvar-Fard,et al.  Image-Based Automated 3D Crack Detection for Post-disaster Building Assessment , 2014, J. Comput. Civ. Eng..

[35]  Francisco Bonnin-Pascual,et al.  Corrosion Detection for Automated Visual Inspection , 2014 .

[36]  Annika Hinze,et al.  Storing RDF as a graph , 2003, Proceedings of the IEEE/LEOS 3rd International Conference on Numerical Simulation of Semiconductor Optoelectronic Devices (IEEE Cat. No.03EX726).

[37]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[38]  Chul Min Yeum,et al.  Vision‐Based Automated Crack Detection for Bridge Inspection , 2015, Comput. Aided Civ. Infrastructure Eng..

[39]  K. Kraus.  Photogrammetry: Geometry from Images and Laser Scans , 2007 .

[40]  K. Telleen,et al.  Practical Lessons for Concrete Wall Design , Based on Studies of the 2010 Chile Earthquake , 2012 .

[41]  Antonio Torralba,et al.  Sharing features: efficient boosting procedures for multiclass object detection , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[42]  Kpalma Kidiyo,et al.  A Survey of Shape Feature Extraction Techniques , 2008 .

[43]  C. Lawrence Zitnick,et al.  Edge Boxes: Locating Object Proposals from Edges , 2014, ECCV.

[44]  N. F. Noy,et al.  Ontology Development 101: A Guide to Creating Your First Ontology , 2001 .

[45]  Wilfried Philips,et al.  Extrinsic Calibration of Camera Networks Using a Sphere , 2015, Sensors.

[46]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[47]  Christian Koch,et al.  Pothole detection in asphalt pavement images , 2011, Adv. Eng. Informatics.

[48]  Sangwook Lee,et al.  Automated recognition of surface defects using digital color image processing , 2006 .

[49]  A. T. Schreiber,et al.  Semantic Annotation of Image Collections , 2003 .

[50]  Pietro Perona,et al.  Integral Channel Features , 2009, BMVC.

[51]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[52]  Ioannis Brilakis,et al.  Progressive 3D reconstruction of infrastructure with videogrammetry , 2011 .

[53]  Geun-Duk Park,et al.  Linked tag: image annotation using semantic relationships between image tags , 2014, Multimedia Tools and Applications.

[54]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[55]  Gaurav S. Sukhatme,et al.  A survey and evaluation of promising approaches for automatic image-based defect detection of bridge structures , 2009 .

[56]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[57]  James Caverlee,et al.  Dublin Core , 2009, Encyclopedia of Database Systems.

[58]  S. Deans.  The Radon Transform and Some of Its Applications , 1983 .

[59]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[60]  Santiago Pujol,et al.  Database on Performance of High-Rise Reinforced Concrete Buildings in the 2015 Nepal Earthquake , 2015 .

[61]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[62]  Sami F. Masri,et al.  A new methodology for non-contact accurate crack width measurement through photogrammetry for automated structural safety evaluation , 2013 .

[63]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[64]  Reginald DesRoches,et al.  Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments , 2012, Adv. Eng. Informatics.

[65]  Erich Elsen,et al.  Deep Speech: Scaling up end-to-end speech recognition , 2014, ArXiv.

[66]  Ji Yeong Lee,et al.  Intelligent Bridge Inspection Using Remote Controlled Robot and Image Processing Technique , 2011 .

[67]  Benjamin A. Graybeal,et al.  RELIABILITY OF VISUAL INSPECTION FOR HIGHWAY BRIDGES, VOLUME I: FINAL REPORT , 2001 .

[68]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Qixiang Ye,et al.  Pedestrian Detection with Deep Convolutional Neural Network , 2014, ACCV Workshops.

[70]  Carol J. Friedland,et al.  A SURVEY OF UNMANNED AERIAL VEHICLE ( UAV ) USAGE FOR IMAGERY , 2011 .

[71]  Michael S. Bernstein,et al.  Image retrieval using scene graphs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[72]  Krista A. Ehinger,et al.  SUN Database: Exploring a Large Collection of Scene Categories , 2014, International Journal of Computer Vision.

[73]  Dusmanta Kumar Mohanta,et al.  Review of vision-based steel surface inspection systems , 2014, EURASIP Journal on Image and Video Processing.