Three-dimensional texture measurement using deep learning and multi-view pavement images

Abstract Surface macro-texture measurement plays an essential role in assessing pavement performance and quality. This paper presents a framework for macro-texture measurement and reconstruction using any number of pavement views. In the framework, an encoder based on deep CNN is used to extract features from single-view pavement images. The features are then converted into a voxel model using a feature mapping unit and a decoder. A multi-view combination module aggregates the voxel models from any number of views to build a 3D model of pavement macro-texture. The 3D model can be used to assess pavement performance. The experiment demonstrates the effectiveness of the framework using a database with 16 asphalt pavements. The results show that the proposed framework performs 3D model reconstruction with satisfactory precision and stability. The 3D models assess mean texture depth and dynamic fraction coefficient with the average errors of 7.62% and 6.32%, respectively.

[1]  Q. Mcnemar.  Note on the sampling error of the difference between correlated proportions or percentages , 1947, Psychometrika.

[2]  Zhenjun Wang,et al.  High-throughput design of fiber reinforced cement-based composites using deep learning , 2020 .

[3]  Wendy Flores-Fuentes,et al.  Influence of data clouds fusion from 3D real-time vision system on robotic group dead reckoning in unknown terrain , 2020, IEEE/CAA Journal of Automatica Sinica.

[4]  Zheng Tong,et al.  Pavement defect detection with fully convolutional network and an uncertainty framework , 2020, Comput. Aided Civ. Infrastructure Eng..

[5]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[6]  Oleg Sergiyenko,et al.  Improve three-dimensional point localization accuracy in stereo vision systems using a novel camera calibration method , 2020 .

[7]  Jia Liang,et al.  A novel pavement mean texture depth evaluation strategy based on three-dimensional pavement data filtered by a new filtering approach , 2020 .

[8]  Yu Liu,et al.  3D object understanding with 3D Convolutional Neural Networks , 2016, Inf. Sci..

[9]  Jie Gao,et al.  Convolutional Neural Network for Asphalt Pavement Surface Texture Analysis , 2018, Comput. Aided Civ. Infrastructure Eng..

[10]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network , 2017, Comput. Aided Civ. Infrastructure Eng..

[11]  Zheng Tong,et al.  Advances of deep learning applications in ground-penetrating radar: A survey , 2020 .

[12]  Jerzy Hoła,et al.  New paradigm in the metrology of concrete surface morphology: Methods, parameters and applications , 2021 .

[13]  Adam Zofka,et al.  The effect of exposed aggregate concrete gradation on the texture characteristics and durability , 2020 .

[14]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  M. Staiano.  Tire–Pavement Noise and Pavement Texture , 2018, Journal of Transportation Engineering, Part B: Pavements.

[16]  Zhang Xiong,et al.  3D object retrieval with stacked local convolutional autoencoder , 2015, Signal Process..

[17]  Akash Kumar,et al.  Fast segmentation of industrial quality pavement images using Laws texture energy measures and k-means clustering , 2016, J. Electronic Imaging.

[18]  Jiong Zhang,et al.  Measurement method of asphalt pavement mean texture depth based on multi-line laser and binocular vision , 2017 .

[19]  Georges Bou-Saab,et al.  Pavement Friction Modeling using Texture Measurements and Pendulum Skid Tester , 2018, Transportation Research Record: Journal of the Transportation Research Board.

[20]  Rob Fergus,et al.  Stochastic Pooling for Regularization of Deep Convolutional Neural Networks , 2013, ICLR.

[21]  Jiri Matas,et al.  All you need is a good init , 2015, ICLR.

[22]  Jerzy Ejsmont,et al.  Tyre rolling resistance and its influence on fuel consumption , 2017 .

[23]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces with a Recurrent Neural Network , 2018, Comput. Aided Civ. Infrastructure Eng..

[24]  Liqun Hu,et al.  Effect of three-dimensional macrotexture characteristics on dynamic frictional coefficient of asphalt pavement surface , 2016 .

[25]  Nhat-Duc Hoang,et al.  Automatic detection of asphalt pavement raveling using image texture based feature extraction and stochastic gradient descent logistic regression , 2019, Automation in Construction.

[26]  M. Nichols,et al.  Temporal and spatial evolution of soil surface roughness on stony plots , 2020, Soil and Tillage Research.

[27]  Wei Jia,et al.  3D-FHNet: Three-Dimensional Fusion Hierarchical Reconstruction Method for Any Number of Views , 2019, IEEE Access.

[28]  Filippo Giammaria Praticò,et al.  A study on the relationship between mean texture depth and mean profile depth of asphalt pavements , 2015 .

[29]  Yuanyuan Wang,et al.  The detection effect of pavement 3D texture morphology using improved binocular reconstruction algorithm with laser line constraint , 2020 .

[30]  Xiaoming Huang,et al.  Real-time identification system of asphalt pavement texture based on the close-range photogrammetry , 2019 .

[31]  Łukasz Sadowski,et al.  Multi-scale metrology of concrete surface morphology: Fundamentals and specificity , 2016 .

[32]  Liang Song,et al.  Faster region convolutional neural network for automated pavement distress detection , 2019, Road Materials and Pavement Design.

[33]  Song Bai,et al.  Deep learning representation using autoencoder for 3D shape retrieval , 2014, Proceedings 2014 IEEE International Conference on Security, Pattern Analysis, and Cybernetics (SPAC).

[34]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[35]  Yiannis Kompatsiaris,et al.  Deep Learning Advances in Computer Vision with 3D Data , 2017, ACM Comput. Surv..

[36]  Wendy Flores-Fuentes,et al.  Surface Measurement Techniques in Machine Vision , 2019, Optoelectronics in Machine Vision-Based Theories and Applications.

[37]  Thierry Denoeux,et al.  ConvNet and Dempster-Shafer Theory for Object Recognition , 2019, SUM.

[38]  Kris De Brabanter,et al.  Wavelet Filter Design for Pavement Roughness Analysis , 2016, Comput. Aided Civ. Infrastructure Eng..

[39]  Kelvin C. P. Wang,et al.  Hybrid Procedure for Automated Detection of Cracking with 3D Pavement Data , 2016, J. Comput. Civ. Eng..