Optimizing convolutional neural networks to perform semantic segmentation on large materials imaging datasets: X-ray tomography and serial sectioning

Abstract Machine learning was used to segment large materials science datasets resulting from synchrotron-based x-ray computed tomography (XCT) images of dendrite growth, and serial sectioning (SS) images of dendrite coarsening. Both neural networks (NNs) yielded quantitatively more accurate outputs than conventional segmentation techniques using only 30 XCT or 6 SS training images. We show that performance can be improved if NNs are trained using a large number of small images that are sampled from the fixed amount of training data. The optimal image size and number of training images was identified for the XCT and SS datasets. NN transferability was also tested by applying the highest performing XCT and SS NNs to related datasets. While the initial segmentations were successful, applying simple transformations to the raw images further improved NN performance. These results show the great predictive ability and promising future of using machine learning for segmentation of large materials science datasets.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Masahiko Demura,et al.  Pattern recognition with machine learning on optical microscopy images of typical metallurgical microstructures , 2018, Scientific Reports.

[3]  Gabriela Csurka,et al.  What is a good evaluation measure for semantic segmentation? , 2013, BMVC.

[4]  M. Mattson,et al.  Oxidative stress activates a positive feedback between the γ‐ and β‐secretase cleavages of the β‐amyloid precursor protein , 2007 .

[5]  Rodrigo Nakamura,et al.  Computer techniques towards the automatic characterization of graphite particles in metallographic images of industrial materials , 2013, Expert Syst. Appl..

[6]  Jeff Heaton,et al.  Ian Goodfellow, Yoshua Bengio, and Aaron Courville: Deep learning , 2017, Genetic Programming and Evolvable Machines.

[7]  Francesco De Carlo,et al.  Automated correlative segmentation of large Transmission X-ray Microscopy (TXM) tomograms using deep learning , 2018, Materials Characterization.

[8]  Bülent Yener,et al.  Image driven machine learning methods for microstructure recognition , 2016 .

[9]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[10]  Edward J. Delp,et al.  Parameter estimation and segmentation of noisy or textured images using the EM algorithm and MPM estimation , 1994, Proceedings of 1st International Conference on Image Processing.

[11]  P. Voorhees,et al.  The morphological evolution of dendritic microstructures during coarsening , 2006 .

[12]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[13]  M. De Graef,et al.  The Three-Dimensional Morphology of Growing Dendrites , 2015, Scientific Reports.

[14]  Brian K. Spears,et al.  Contemporary machine learning: a guide for practitioners in the physical sciences , 2017, ArXiv.

[15]  Michael J. Kulis,et al.  A quantifiable and automated volume fraction characterization technique for secondary and tertiary γ′ precipitates in Ni-based superalloys , 2018, Materials Characterization.

[16]  Yang Leng,et al.  Materials characterization : introduction to microscopic and spectroscopic methods , 2013 .

[17]  Mario Fritz,et al.  Advanced Steel Microstructural Classification by Deep Learning Methods , 2017, Scientific Reports.

[18]  Brian L. DeCost,et al.  Exploring the microstructure manifold: Image texture representations applied to ultrahigh carbon steel microstructures , 2017, 1702.01117.

[19]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[20]  P. Voorhees,et al.  Quantitative serial sectioning analysis , 2001, Journal of microscopy.

[21]  Synho Do,et al.  How much data is needed to train a medical image deep learning system to achieve necessary high accuracy , 2015, 1511.06348.

[22]  Sabrina Rashid,et al.  An improved method for the removal of ring artifacts in high resolution CT imaging , 2012, EURASIP Journal on Advances in Signal Processing.

[23]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[25]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  David Brandon,et al.  Microstructural Characterization of Materials , 1999 .

[27]  Elizabeth A. Holm,et al.  A computer vision approach for automated analysis and classification of microstructural image data , 2015 .

[28]  Antonio Criminisi,et al.  TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context , 2007, International Journal of Computer Vision.

[29]  Charles A. Bouman,et al.  TIMBIR: A Method for Time-Space Reconstruction From Interlaced Views , 2015, IEEE Transactions on Computational Imaging.

[30]  Matthew Q. Hill,et al.  Improving Transferability of Deep Neural Networks , 2018, Domain Adaptation for Visual Understanding.

[31]  W. Kurz,et al.  Fundamentals of Solidification , 1990 .

[32]  Lipo Wang,et al.  Deep Learning Applications in Medical Image Analysis , 2018, IEEE Access.

[33]  I. Karakaya,et al.  The Pb−Sn (Lead-Tin) system , 1988 .

[34]  Peter W Voorhees,et al.  Measurement of Interfacial Evolution in Three Dimensions , 2012 .

[35]  Peter W Voorhees,et al.  Growth and Coarsening , 2002 .

[36]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[37]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.