Integrated Multiple-Defect Detection and Evaluation of Rail Wheel Tread Images using Convolutional Neural Networks

The wheel-rail interface is regarded as the most important factor for the dynamic behavior of a railway vehicle, affecting the safety of the service, the passenger comfort, and the life of the wheelset asset. The degradation of the wheels in contact with the rail is visibly manifest on their treads in the form of defects such as indentations, flats, cavities, etc. To guarantee a reliable rail service and maximize the availability of the rolling-stock assets, these defects need to be constantly and periodically monitored as their severity evolves. This inspection task is usually conducted manually at the fleet level and therefore it takes a lot of human resources. In order to add value to this maintenance activity, this article presents an automatic Deep Learning method to jointly detect and classify wheel tread defects based on smartphone pictures taken by the maintenance team. The architecture of this approach is based on a framework of Convolutional Neural Networks, which is applied to the different tasks of the diagnosis process including the location of the defect area within the image, the prediction of the defect size, and the identification of defect type. With this information determined, the maintenance-criteria rules can ultimately be applied to obtain the actionable results. The presented neural approach has been evaluated with a set of wheel defect pictures collected over the course of nearly two years, concluding that it can reliably automate the condition diagnosis of half of the current workload and thus reduce the lead time to take maintenance action, significantly reducing Alexandre Trilla et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. engineering hours for verification and validation. Overall, this creates a platform of significant progress in automated predictive maintenance of rolling stock wheelsets.

[1]  Shubin Li,et al.  Detection of rail surface defects based on CNN image recognition and classification , 2018, 2018 20th International Conference on Advanced Communication Technology (ICACT).

[2]  Wan-Jui Lee,et al.  Potential, challenges and future directions for deep learning in prognostics and health management applications , 2020, Eng. Appl. Artif. Intell..

[3]  Ali Borji,et al.  What are the Receptive, Effective Receptive, and Projective Fields of Neurons in Convolutional Neural Networks? , 2017, ArXiv.

[4]  Allan Pinkus,et al.  Approximation theory of the MLP model in neural networks , 1999, Acta Numerica.

[5]  Thurston Sexton,et al.  Agreement Behavior of Isolated Annotators for Maintenance Work-Order Data Mining , 2019, Annual Conference of the PHM Society.

[6]  Andrea Vedaldi,et al.  Understanding deep image representations by inverting them , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Mohammad Hassan Shojaeefard,et al.  Sensitivity Analysis of the Artificial Neural Network Outputs in Friction Stir Lap Joining of Aluminum to Brass , 2013 .

[9]  Pierre Dersin,et al.  Health Assessment of Railway Turnouts : A Case Study , 2016 .

[10]  Csaba Simon,et al.  Open Source FaaS Performance Aspects , 2020, 2020 43rd International Conference on Telecommunications and Signal Processing (TSP).

[11]  Jing Tian,et al.  Anomaly Detection Using Self-Organizing Maps-Based K-Nearest Neighbor Algorithm , 2014 .

[12]  Pascal Vincent,et al.  Visualizing Higher-Layer Features of a Deep Network , 2009 .

[13]  Andrea Vedaldi,et al.  Interpretable Explanations of Black Boxes by Meaningful Perturbation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[15]  Bolei Zhou,et al.  Understanding Intra-Class Knowledge Inside CNN , 2015, ArXiv.

[16]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[17]  Yoshua Bengio,et al.  Measuring the tendency of CNNs to Learn Surface Statistical Regularities , 2017, ArXiv.

[18]  Ashish Kapoor,et al.  Do Adversarially Robust ImageNet Models Transfer Better? , 2020, NeurIPS.

[19]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[20]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[21]  Lei Zhang,et al.  CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Paul Hyde,et al.  Development and testing of an automatic remote condition monitoring system for train wheels , 2016 .

[23]  Joachim M. Buhmann,et al.  Wheel Defect Detection With Machine Learning , 2018, IEEE Transactions on Intelligent Transportation Systems.

[24]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Bolei Zhou,et al.  Network Dissection: Quantifying Interpretability of Deep Visual Representations , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Tor-Morten Grønli,et al.  Progressive Web Apps: The Possible Web-native Unifier for Mobile Development , 2017, WEBIST.

[27]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[28]  Werner Dubitzky,et al.  Fundamentals of Data Mining in Genomics and Proteomics , 2009 .

[29]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[30]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Salahaldin Juba,et al.  Learning PostgreSQL 11 : a beginner's guide to building high-performance PostgreSQL database solutions , 2019 .

[32]  Kaisheng Ma,et al.  Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[33]  Aleksander Madry,et al.  Exploring the Landscape of Spatial Robustness , 2017, ICML.

[34]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[35]  Aleksander Madry,et al.  Adversarial Examples Are Not Bugs, They Are Features , 2019, NeurIPS.

[36]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[37]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[38]  Quoc V. Le,et al.  Large-Scale Evolution of Image Classifiers , 2017, ICML.

[39]  Junsheng Zhang,et al.  Defect Detection of Aluminum Alloy Wheels in Radiography Images Using Adaptive Threshold and Morphological Reconstruction , 2018, Applied Sciences.

[40]  Thomas Serre,et al.  Same-different problems strain convolutional neural networks , 2018, CogSci.

[41]  Jason Yosinski,et al.  Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks , 2016, ArXiv.

[42]  Indra Prasetya Aji.  Landmark Classification Service Using Convolutional Neural Network and Kubernetes , 2020 .

[43]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  S. Roweis,et al.  An Adversarial View of Covariate Shift and A Minimax Approach , 2009 .

[45]  Asifullah Khan,et al.  A survey of the recent architectures of deep convolutional neural networks , 2019, Artificial Intelligence Review.

[46]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[47]  Yan Zhang,et al.  Tire Defects Classification Using Convolution Architecture for Fast Feature Embedding , 2018, Int. J. Comput. Intell. Syst..

[48]  Li Wang,et al.  The defects recognition of wheel tread based on linear CCD , 2014, 2014 IEEE Far East Forum on Nondestructive Evaluation/Testing.

[49]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[50]  Simon Kornblith,et al.  The Origins and Prevalence of Texture Bias in Convolutional Neural Networks , 2019, NeurIPS.

[51]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[52]  Thomas G. Dietterich,et al.  Benchmarking Neural Network Robustness to Common Corruptions and Perturbations , 2018, ICLR.

[53]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[54]  Yuzhe Yang,et al.  Rethinking the Value of Labels for Improving Class-Imbalanced Learning , 2020, NeurIPS.

[55]  Michael Eickenberg,et al.  Seeing it all: Convolutional network layers map the function of the human visual system , 2017, NeuroImage.

[56]  Yi Shang,et al.  Deep learning for prognostics and health management: State of the art, challenges, and opportunities , 2020 .

[57]  John D. Austin,et al.  Adaptive histogram equalization and its variations , 1987 .

[58]  Meina Song,et al.  An Automatic Artificial Intelligence Training Platform Based on Kubernetes , 2020, BDET.

[59]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[60]  Geoffrey E. Hinton,et al.  Similarity of Neural Network Representations Revisited , 2019, ICML.

[61]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[62]  James Moyne,et al.  A Requirements Driven Digital Twin Framework: Specification and Opportunities , 2020, IEEE Access.

[63]  Quoc V. Le,et al.  Meta Pseudo Labels , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  Junmo Kim,et al.  Learning Not to Learn: Training Deep Neural Networks With Biased Data , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Kai Han,et al.  A new method in wheel hub surface defect detection: Object detection algorithm based on deep learning , 2017, 2017 International Conference on Advanced Mechatronic Systems (ICAMechS).

[66]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[67]  Ryong Lee,et al.  Development of an AI Analysis Service System based on OpenFaaS , 2020 .

[68]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[69]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[70]  Jianping Peng,et al.  Wheel Tread Defects Inspection Based on SVM , 2017, 2017 Far East NDT New Technology & Application Forum (FENDT).

[71]  Dimitris Samaras,et al.  Texture classification for rail surface condition evaluation , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[72]  C. Rudin,et al.  Concept whitening for interpretable image recognition , 2020, Nature Machine Intelligence.

[73]  Sami Kara,et al.  Manufacturing big data ecosystem: A systematic literature review , 2020, Robotics Comput. Integr. Manuf..

[74]  Choopan Rattanapoka,et al.  An IoT System Design with Real-Time Stream Processing and Data Flow Integration , 2019, 2019 Research, Invention, and Innovation Congress (RI2C).

[75]  Bolei Zhou,et al.  Object Detectors Emerge in Deep Scene CNNs , 2014, ICLR.

[76]  Matthew Botvinick,et al.  On the importance of single directions for generalization , 2018, ICLR.

[77]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[78]  Jianjun Zhao,et al.  An Empirical Study Towards Characterizing Deep Learning Development and Deployment Across Different Frameworks and Platforms , 2019, 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[79]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[80]  Christian Mathis,et al.  Data Lakes , 2017, Datenbank-Spektrum.

[81]  Sergey I. Nikolenko.  Synthetic Data for Deep Learning , 2019, ArXiv.

[82]  Christophe Gransart,et al.  Cyber Security for Railways - A Huge Challenge - Shift2Rail Perspective , 2017, Nets4Cars/Nets4Trains/Nets4Aircraft.

[83]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[84]  Adam Bevan,et al.  Predictive wheel–rail management in London Underground: Validation and verification , 2020, Proceedings of the Institution of Mechanical Engineers, Part F: Journal of Rail and Rapid Transit.

[85]  Jay Lee,et al.  An Integrated Framework of Drivetrain Degradation Assessment and Fault Localization for Offshore Wind Turbines , 2020 .

[86]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.