Microstructure Instance Segmentation from Aluminum Alloy Metallographic Image Using Different Loss Functions

Automatic segmentation of metallographic image is very important for the implementation of an automatic metallographic analysis system. In this paper, a novel instance segmentation framework of a metallographic image was implemented, which can assign each pixel to a physical instance of a microstructure. In this framework, we used the Mask R-CNN as the basic network to complete the learning and recognition of the latent feature of an aluminum alloy microstructure. Meanwhile, we implemented five different loss functions based on this framework and compared the influence of these loss functions on metallographic image segmentation performance. We carried out several experiments to verify the effectiveness of the proposed framework. In these experiments, we compared and analyzed six different evaluation metrics and provided constructive suggestions for the performance evaluation of metallographic image segmentation method. A large number of experimental results have shown that the proposed method can achieve the instance segmentation of an aluminum alloy metallographic image and the segmentation results are satisfactory.

[1]  Elizabeth A. Holm,et al.  A computer vision approach for automated analysis and classification of microstructural image data , 2015 .

[2]  Alán Aspuru-Guzik,et al.  Inverse molecular design using machine learning: Generative models for matter engineering , 2018, Science.

[3]  João Manuel R S Tavares,et al.  Automatic evaluation of nickel alloy secondary phases from SEM images , 2011, Microscopy research and technique.

[4]  W. S. Miller,et al.  Recent development in aluminium alloys for aerospace applications , 2000 .

[5]  Pascal Jacques,et al.  Metallographic methods for revealing the multiphase microstructure of TRIP-assisted steels , 1998 .

[6]  Marc Winter,et al.  Advanced microstructure classification by data mining methods , 2018, Computational Materials Science.

[7]  P. Murray,et al.  New methods for automatic quantification of microstructural features using digital image processing , 2018 .

[8]  Yang Wang,et al.  Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation , 2016, ISVC.

[9]  Qi Zhang,et al.  Algorithm based on regional separation for automatic grain boundary extraction using improved mean shift method , 2018 .

[10]  O. V. Zabolotnyi,et al.  Metallographic Analysis and Microstructural Image Processing of Sandblasting Nozzles Produced by Powder Metallurgy Methods , 2015, Powder Metallurgy and Metal Ceramics.

[11]  Mario Fritz,et al.  Advanced Steel Microstructural Classification by Deep Learning Methods , 2017, Scientific Reports.

[12]  Rodrigo Nakamura,et al.  Computer techniques towards the automatic characterization of graphite particles in metallographic images of industrial materials , 2013, Expert Syst. Appl..

[13]  Hao Chen,et al.  DCAN: Deep contour‐aware networks for object instance segmentation from histology images , 2017, Medical Image Anal..

[14]  F. H. Samuel,et al.  Porosity formation in AI-9 Wt Pct Si-3 Wt Pct Cu alloy systems: Metallographic observations , 1996 .

[15]  Mary L. Comer,et al.  Physics of MRF regularization for segmentation of materials microstructure images , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[16]  Bo Song,et al.  A review of selective laser melting of aluminum alloys: Processing, microstructure, property and developing trends , 2019, Journal of Materials Science & Technology.

[17]  Dimitris N. Metaxas,et al.  Attentive neural cell instance segmentation , 2019, Medical Image Anal..

[18]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[19]  João Manuel R. S. Tavares,et al.  Evaluation of multilayer perceptron and self-organizing map neural network topologies applied on microstructure segmentation from metallographic images , 2009 .

[20]  Lvping Chen,et al.  Two-Dimensional Fuzzy Clustering Algorithm (2DFCM) for Metallographic Image Segmentation Based on Spatial Information , 2015, 2015 2nd International Conference on Information Science and Control Engineering.

[21]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[22]  T. Pollock,et al.  3D printing of high-strength aluminium alloys , 2017, Nature.

[23]  J. Hirsch,et al.  Superior light metals by texture engineering: Optimized aluminum and magnesium alloys for automotive applications , 2013 .

[24]  Wanbo Liu,et al.  Deep Learning-Based Image Segmentation for Al-La Alloy Microscopic Images , 2018, Symmetry.

[25]  Wei Chen,et al.  Metallographical Image Segmentation and Compression , 2012 .

[26]  Amitabh Varshney,et al.  Interactive exploration of microstructural features in gigapixel microscopy images , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[27]  Masahiko Demura,et al.  Pattern recognition with machine learning on optical microscopy images of typical metallurgical microstructures , 2018, Scientific Reports.

[28]  Deniz Erdogmus,et al.  Tversky Loss Function for Image Segmentation Using 3D Fully Convolutional Deep Networks , 2017, MLMI@MICCAI.

[29]  Ing Ren Tsang,et al.  Multiclass Weighted Loss for Instance Segmentation of Cluttered Cells , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[30]  Lisa Tang,et al.  Deep Convolutional Encoder Networks for Multiple Sclerosis Lesion Segmentation , 2015, MICCAI.

[31]  Xiao Xiang Zhu,et al.  Vehicle Instance Segmentation From Aerial Image and Video Using a Multitask Learning Residual Fully Convolutional Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[32]  K. Rajasekhar,et al.  Microstructural evolution during solidification of austenitic stainless steel weld metals: A color metallographic and electron microprobe analysis study , 1997 .

[33]  Pierre Gouton,et al.  Evaluating creep in metals by grain boundary extraction using directional wavelets and mathematical morphology , 2001 .

[34]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[35]  Geert J. S. Litjens,et al.  Structure Instance Segmentation in Renal Tissue: A Case Study on Tubular Immune Cell Detection , 2018, COMPAY/OMIA@MICCAI.

[36]  Bülent Yener,et al.  Image driven machine learning methods for microstructure recognition , 2016 .

[37]  Samuel Luz Gomes,et al.  New Analysis Method Application in Metallographic Images through the Construction of Mosaics Via Speeded Up Robust Features and Scale Invariant Feature Transform , 2015, Materials.

[38]  Aashish Rohatgi,et al.  A METALLOGRAPHIC AND QUANTITATIVE ANALYSIS OF THE INFLUENCE OF STACKING FAULT ENERGY ON SHOCK- HARDENING IN Cu AND Cu-Al ALLOYS , 2001 .