Segmentation and Recognition of Highway Assets Using Image-Based 3D Point Clouds and Semantic Texton Forests

AbstractEfficient data collection of high-quantity and low-cost highway assets such as road signs, traffic signals, light poles, and guardrails is a critical element to the operation, maintenance, and preservation of transportation infrastructure systems. Despite its importance, current practice of highway asset data collection is time-consuming, subjective, and potentially unsafe. The high volume of the data that needs to be collected can also negatively impact the quality of the analysis. To address these limitations, this paper proposes a new algorithm for semantic segmentation and recognition of highway assets using video frames collected from a car-mounted camera. The proposed set of algorithms (1) takes the captured frames and using a pipeline of structure from motion and multiview stereo reconstructs a three-dimensional (3D) point cloud model of the highway and surrounding assets; (2) using a Semantic Texton Forest classifier, each geo-registered two-dimensional (2D) video frame at the pixel-level ...

[1]  Saturnino Maldonado-Bascón,et al.  Shape Classification Algorithm Using Support Vector Machines for Traffic Sign Recognition , 2005, IWANN.

[2]  Mani Golparvar-Fard,et al.  Evaluation of image-based modeling and laser scanning accuracy for emerging automated performance monitoring techniques , 2011 .

[3]  Steven M. Seitz,et al.  Finding paths through the world's photos , 2008, SIGGRAPH 2008.

[4]  Zhaohua Wang,et al.  Image Processing Algorithms for an Enhanced Roadway Sign Data Collection , 2008 .

[5]  Jamie Shotton,et al.  Semantic Texton Forests , 2010, Computer Vision: Detection, Recognition and Reconstruction.

[6]  Roberto Cipolla,et al.  Segmentation and Recognition Using Structure from Motion Point Clouds , 2008, ECCV.

[7]  Andrew Owens,et al.  Discrete-continuous optimization for large-scale structure from motion , 2011, CVPR.

[8]  Luc Van Gool,et al.  Multi-view traffic sign detection, recognition, and 3D localisation , 2014, 2009 Workshop on Applications of Computer Vision (WACV).

[9]  Mani Golparvar-Fard,et al.  Image-based 3D reconstruction and Recognition for Enhanced Highway Condition Assessment , 2011 .

[10]  Reza A. Zoroofi,et al.  Gabor wavelet for road sign detection and recognition using a hybrid classifier , 2010, 2010 International Conference on Multimedia Computing and Information Technology (MCIT).

[11]  Xiaohong W. Gao,et al.  Application of Vision Models to Traffic Sign Recognition , 2003, ICANN.

[12]  David W. Smith,et al.  Interrater Reliability of Manual Pavement Distress Evaluations , 2010 .

[13]  Mohamed S. Kamel,et al.  Image Analysis and Recognition , 2014, Lecture Notes in Computer Science.

[14]  Zhaozheng Hu,et al.  Generalized Image Recognition Algorithm for Sign Inventory , 2011 .

[15]  Sudipta N. Sinha,et al.  REAL-TIME VIDEO-BASED RECONSTRUCTION OF URBAN ENVIRONMENTS , 2007 .

[16]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Joseph E. Hummer,et al.  IT Issues for the Management of High-Quantity, Low-Cost Assets , 2009 .

[18]  Marc Pollefeys,et al.  Fast robust large-scale mapping from video and internet photo collections , 2010 .

[19]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[21]  Anton Kummert,et al.  3D Traffic Sign Tracking Using a Particle Filter , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[22]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Roberto Cipolla,et al.  Semantic texton forests for image categorization and segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Yichang Tsai,et al.  Real-time Speed Limit Sign Recognition Based on Locally Adaptive Thresholding and Depth-First-Search , 2005 .

[25]  Jianping Wu,et al.  Enhanced Roadway Inventory Using a 2-D Sign Video Image Recognition Algorithm , 2006, Comput. Aided Civ. Infrastructure Eng..

[26]  Antonio Criminisi,et al.  TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context , 2007, International Journal of Computer Vision.

[27]  Vincent Lepetit,et al.  Randomized trees for real-time keypoint recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  Zhu Shuang-dong,et al.  Detection for triangle traffic sign based on neural network , 2005, IEEE International Conference on Vehicular Electronics and Safety, 2005..

[29]  Mani Golparvar-Fard,et al.  Automated Progress Monitoring Using Unordered Daily Construction Photographs and IFC-Based Building Information Models , 2015, J. Comput. Civ. Eng..

[30]  Kamel S. Saidi,et al.  Assessing Technology Gaps for the Federal Highway Administration Digital Highway Measurement Program (NISTIR 7685) , 2010 .

[31]  Marc Pollefeys,et al.  Real-time photo-realistic 3D mapping for micro aerial vehicles , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Jianping Wu,et al.  Shape- and Texture-Based 1-D Image Processing Algorithm for Real-Time Stop Sign Road Inventory Data Collection , 2002, J. Intell. Transp. Syst..

[33]  Francisco López-Ferreras,et al.  Road-Sign Detection and Recognition Based on Support Vector Machines , 2007, IEEE Transactions on Intelligent Transportation Systems.

[34]  J. M. De la Garza,et al.  A Study of Implementation of IP-S2 Mobile Mapping Technology for Highway Asset Condition Assessment , 2011 .

[35]  Ioannis Brilakis,et al.  Progressive 3D reconstruction of infrastructure with videogrammetry , 2011 .

[36]  Yun-Shin Lee,et al.  Detection and recognition of speed limit signs , 2010, 2010 International Computer Symposium (ICS2010).

[37]  Carlos H. Caldas,et al.  Methodology for Automating the Identification and Localization of Construction Components on Industrial Projects , 2009 .

[38]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[39]  Zoran Popovic,et al.  PhotoCity: training experts at large-scale image acquisition through a competitive game , 2011, CHI.

[40]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.