Enhanced EDX images by fusion of multimodal SEM images using pansharpening techniques

The goal of this paper is to explore the potential interest of image fusion in the context of multimodal scanning electron microscope (SEM) imaging. In particular, we aim at merging the backscattered electron images that usually have a high spatial resolution but do not provide enough discriminative information to physically classify the nature of the sample, with energy‐dispersive X‐ray spectroscopy (EDX) images that have discriminative information but a lower spatial resolution. The produced images are named enhanced EDX. To achieve this goal, we have compared the results obtained with classical pansharpening techniques for image fusion with an original approach tailored for multimodal SEM fusion of information. Quantitative assessment is obtained by means of two SEM images and a simulated dataset produced by a software based on PENELOPE.

[1]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[2]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[3]  Aleksandra Pizurica,et al.  Processing of Multiresolution Thermal Hyperspectral and Digital Color Data: Outcome of the 2014 IEEE GRSS Data Fusion Contest , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[4]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  J. Sempau,et al.  Experimental benchmarks of the Monte Carlo code penelope , 2003 .

[6]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[7]  J. Sempau,et al.  PENELOPE-2006: A Code System for Monte Carlo Simulation of Electron and Photon Transport , 2009 .

[8]  L. Reimer,et al.  Scanning Electron Microscopy , 1984 .

[9]  Ivo W. Rangelow,et al.  Six-axis AFM in SEM with self-sensing and self-transduced cantilever for high speed analysis and nanolithography , 2016 .

[10]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[11]  Jon Atli Benediktsson,et al.  Spectral–Spatial Hyperspectral Image Classification With Edge-Preserving Filtering , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[12]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[13]  Roger L. King,et al.  A wavelet based algorithm for pan sharpening Landsat 7 imagery , 2001, IGARSS 2001. Scanning the Present and Resolving the Future. Proceedings. IEEE 2001 International Geoscience and Remote Sensing Symposium (Cat. No.01CH37217).

[14]  F. Durand,et al.  Flash photography enhancement via intrinsic relighting , 2004, ACM Trans. Graph..

[15]  S. Sides,et al.  Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic , 1991 .

[16]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[17]  Gemma Piella,et al.  A general framework for multiresolution image fusion: from pixels to regions , 2003, Inf. Fusion.

[18]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  L. Wald,et al.  Fusion of high spatial and spectral resolution images : The ARSIS concept and its implementation , 2000 .

[20]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[21]  Mary Ellen Miller,et al.  Image fusion combining SEM and ToF‐SIMS images , 2015 .

[22]  Lucien Wald,et al.  Some terms of reference in data fusion , 1999, IEEE Trans. Geosci. Remote. Sens..

[23]  Tim Wortmann,et al.  Fusion of AFM and SEM scans , 2009, 2009 International Symposium on Optomechatronic Technologies.

[24]  Jocelyn Chanussot,et al.  Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[25]  F. Scholze,et al.  Modelling the response function of energy dispersive X-ray spectrometers with silicon detectors , 2009 .

[26]  S. Limandri,et al.  Experimental study of the efficiency of a SDD X‐ray detector by means of PIXE spectra , 2013 .

[27]  Richard Bauer,et al.  Computer analysis of X-ray spectra (EDS) from thin biological specimens† , 1978 .

[28]  L. Reimer.  Scanning Electron Microscopy: Physics of Image Formation and Microanalysis , 1984 .

[29]  Jesús Angulo,et al.  Ordering on the Probability Simplex of Endmembers for Hyperspectral Morphological Image Processing , 2015, ISMM.

[30]  J. Baró,et al.  An algorithm for Monte Carlo simulation of coupled electron-photon transport , 1997 .

[31]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Pierre Soille,et al.  Morphological Image Analysis: Principles and Applications , 2003 .

[33]  Te-Ming Tu,et al.  A new look at IHS-like image fusion methods , 2001, Inf. Fusion.