Large‐scale time‐lapse scanning electron microscopy image mosaic using a smooth stitching strategy

Due to the trade-off between the field of view and resolution of various microscopes, obtaining a wide-view panoramic image through high-resolution image tiles is frequently encountered and demanded in numerous applications. Here, we propose an automatic image mosaic strategy for sequential 2D time-lapse scanning electron microscopy (SEM) images. This method can accurately compute pairwise translations among serial image tiles with indeterminate overlapping areas. The detection and matching of feature points are limited by geographical coordinates, thus avoiding accidental mismatching. Moreover, the nonlinear deformation of the mosaic part is also taken into account. A smooth stitching field is utilized to gradually transform the perspective transformation in overlapping regions into the linear transformation in non-overlapping regions. Experimental results demonstrate that better image stitching accuracy can be achieved compared with some other image mosaic algorithms. Such a method has potential applications in high-resolution large-area analysis using serial microscopy images. RESEARCH HIGHLIGHTS: An automatic image mosaic strategy for processing sequential scanning electron microscopy images is proposed. A smooth stitching field is applied in the image mosaic. Improved stitching accuracy is achieved compared with other conventional mosaic methods.

[1]  G. Jursich,et al.  The interface of atomic layer deposited ZrO2 on Si/SiO2 from an alkoxide zirconium precursor and ethanol: A transmission electron microscopy‐focused study , 2023, Surface and Interface Analysis.

[2]  N. Bassim,et al.  Making the Stitching Process of Montaged SEM Images Automatic Using Fourier Transform Properties , 2021, Microscopy and Microanalysis.

[3]  K. Nakamae Electron microscopy in semiconductor inspection , 2021 .

[4]  Dženan Zukić,et al.  ITKMontage: A Software Module for Image Stitching , 2021, Integrating Materials and Manufacturing Innovation.

[5]  L. Rundo,et al.  A quantum-inspired classifier for clonogenic assay evaluations , 2021, Scientific Reports.

[6]  Junjun Jiang,et al.  Image Matching from Handcrafted to Deep Features: A Survey , 2020, International Journal of Computer Vision.

[7]  Giancarlo Mauri,et al.  ACDC: Automated Cell Detection and Counting for Time-Lapse Fluorescence Microscopy , 2020, bioRxiv.

[8]  Hugo Jair Escalante,et al.  Split and merge watershed: A two-step method for cell segmentation in fluorescence microscopy images , 2019, Biomed. Signal Process. Control..

[9]  Fan Guo,et al.  Sequential Far Infrared Image Mosaic Using Coarse-to-Fine Scheme , 2019, IEEE Access.

[10]  M. Badaroglu,et al.  Metrology for the next generation of semiconductor devices , 2018, Nature Electronics.

[11]  Giancarlo Mauri,et al.  Area-based cell colony surviving fraction evaluation: A novel fully automatic approach using general-purpose acquisition hardware , 2017, Comput. Biol. Medicine.

[12]  K. Bhadriraju,et al.  MIST: Accurate and Scalable Microscopy Image Stitching Tool with Stage Modeling and Error Minimization , 2017, Scientific Reports.

[13]  T Meyer,et al.  Seamless stitching of tile scan microscope images , 2015, Journal of microscopy.

[14]  Yoichi Sato,et al.  Shape-Preserving Half-Projective Warps for Image Stitching , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Michael S. Brown,et al.  As-Projective-As-Possible Image Stitching with Moving DLT , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Fan Yang,et al.  A method for fast automated microscope image stitching. , 2013, Micron.

[17]  Juan Zhang,et al.  Assessment approach for image mosaicing algorithms , 2011 .

[18]  Radu Hristu,et al.  Influence of confocal scanning laser microscopy specific acquisition parameters on the detection and matching of speeded-up robust features. , 2011, Ultramicroscopy.

[19]  Tolga Tasdizen,et al.  Automatic mosaicking and volume assembly for high-throughput serial-section transmission electron microscopy , 2010, Journal of Neuroscience Methods.

[20]  Joachim M Buhmann,et al.  Fully automatic stitching and distortion correction of transmission electron microscope images. , 2010, Journal of structural biology.

[21]  H Burkhardt,et al.  XuvTools: free, fast and reliable stitching of large 3D datasets , 2009, Journal of microscopy.

[22]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[23]  Feng He,et al.  Use of Autostitch for automatic stitching of microscope images. , 2007, Micron.

[24]  Michael Unser,et al.  User‐friendly semiautomated assembly of accurate image mosaics in microscopy , 2007, Microscopy research and technique.

[25]  Changming Sun,et al.  Mosaicing of microscope images with global geometric and radiometric corrections , 2006, Journal of microscopy.

[26]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[27]  Borivoj Vojnovic,et al.  An Algorithm for image stitching and blending , 2005, SPIE BiOS.

[28]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[29]  C Wählby,et al.  Combining intensity, edge and shape information for 2D and 3D segmentation of cell nuclei in tissue sections , 2004, Journal of microscopy.

[30]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[31]  Uwe D. Hanebeck,et al.  Template matching using fast normalized cross correlation , 2001, SPIE Defense + Commercial Sensing.

[32]  Gregory Randall,et al.  Neuro3D: an interactive 3D reconstruction system of serial sections using automatic registration , 1998, Photonics West - Biomedical Optics.

[33]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[34]  Jianhu Zhao,et al.  Automatic Overlapping Area Determination and Segmentation for Multiple Side Scan Sonar Images Mosaic , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Damon L. Woodard,et al.  REFICS: Assimilating Data-Driven Paradigms Into Reverse Engineering and Hardware Assurance on Integrated Circuits , 2021, IEEE Access.

[36]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.