Bucket Teeth Detection Based on Faster Region Convolutional Neural Network

The electric shovel is a bucket-equipped mining excavator widely used in open-pit mining today. The prolonged direct impact of the bucket teeth with hard and abrasive materials such as ore during the process of the mining excavation can cause the bucket teeth to break and fall off prematurely, resulting in unplanned downtime and productivity losses. In response to this problem, we have developed a vision-based bucket teeth fault detection algorithm with deep learning. Using a dataset based on the images of both real shovel teeth and 3D-printed models, we trained a Faster Region Convolutional Neural Network (Faster R-CNN) to obtain the number of normal bucket teeth and the positions of the bucket teeth from the images, using the additional bucket dataset from 3D-printed models to pre-train the network for improving its detection accuracy on the real bucket data. We compared the resulting Faster R-CNN model with the ZFNet, the ResNet-50, and the VGG16 and found our Faster R-CNN model to perform best in terms of accuracy and speed.

[1]  Wei Feng,et al.  A high precision image registration method for measurement based on the stereo camera system , 2020 .

[2]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[3]  K. Silpaja Chandrasekar,et al.  Multiple objects tracking by a highly decisive three-frame differencing-combined-background subtraction method with GMPFM-GMPHD filters and VGG16-LSTM classifier , 2020, J. Vis. Commun. Image Represent..

[4]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Mohammad Waqar Ali Asad,et al.  A review of cut-off grade policy models for open pit mining operations , 2016 .

[6]  Jing Xu,et al.  Application of neural network based on Caffe framework for object detection in Hilens , 2019, 2019 Chinese Automation Congress (CAC).

[7]  Zhenwei Wang,et al.  Mathematical modeling and fuzzy approach for disaster analysis on geo-spatial rock mass in open-pit mining , 2020, Comput. Commun..

[8]  Li Yang,et al.  Tunnel contour detection during construction based on digital image correlation , 2020 .

[9]  Rui Li,et al.  Kiwifruit detection in field images using Faster R-CNN with VGG16 , 2019, IFAC-PapersOnLine.

[10]  Hong Zhang,et al.  Object detection by parts using appearance, structural and shape features , 2011, 2011 IEEE International Conference on Mechatronics and Automation.

[11]  Jacqueline Glass,et al.  Improving hydraulic excavator performance through in line hydraulic oil contamination monitoring , 2017 .

[12]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Deyong You,et al.  Tight butt joint weld detection based on optical flow and particle filtering of magneto-optical imaging ☆ , 2017 .

[14]  E. Bakhtavar,et al.  Development of a scenario-based robust model for the optimal truck-shovel allocation in open-pit mining , 2020, Comput. Oper. Res..

[15]  Satish Kumar Injeti,et al.  An efficient approach for optimal multilevel thresholding selection for gray scale images based on improved differential search algorithm , 2016, Ain Shams Engineering Journal.

[16]  A. Keleş,et al.  Improvement of mechanical properties by means of titanium alloying to steel teeth used in the excavator , 2020 .

[17]  Tarig Ahmed Khalid,et al.  Benchmark analysis of popular ImageNet classification deep CNN architectures , 2017, 2017 International Conference On Smart Technologies For Smart Nation (SmartTechCon).

[18]  Luciano E. Chiang,et al.  Model-Based Condition-Based Maintenance of Mining Shovels using Electric Motor Signal Information , 2007 .

[19]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Mauro Castelli,et al.  Analysis of the proficiency of fully connected neural networks in the process of classifying digital images. Benchmark of different classification algorithms on high-level image features from convolutional layers , 2019, Expert Syst. Appl..

[21]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Jun Liu,et al.  High precision detection algorithm based on improved RetinaNet for defect recognition of transmission lines , 2020 .

[23]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Amardeep Singh Kang,et al.  Wear Behavior of Weld Overlays on Excavator Bucket Teeth , 2014 .

[25]  Hong Yang,et al.  Application of open-pit and underground mining technology for residual coal of end slopes , 2010 .

[26]  Ning Zhou,et al.  Tooth guard: A vision system for detecting missing tooth in rope mine shovel , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[27]  Yongchao Yang,et al.  CNN-LSTM deep learning architecture for computer vision-based modal frequency detection , 2020 .

[28]  Donghai Li,et al.  Toward New Retail: A Benchmark Dataset for Smart Unmanned Vending Machines , 2020, IEEE Transactions on Industrial Informatics.

[29]  Huanfeng Shen,et al.  Deeply supervised convolutional neural network for shadow detection based on a novel aerial shadow imagery dataset , 2020 .