A Fast and Massively-Parallel Inverse Solver for Multiple-Scattering Tomographic Image Reconstruction

We present a massively-parallel solver for large Helmholtz-type inverse scattering problems. The solver employs the distorted Born iterative method for capturing the multiples-cattering phenomena in image reconstructions. This method requires many full-wave forward-scattering solutions in each iteration, constituting the main performance bottleneck with its high computational complexity. As a remedy, we use the multilevel fast multipole algorithm (MLFMA). The solver scales among computing nodes using a two-dimensional parallelization strategy that distributes illuminations in one dimension, and MLFMA sub-trees in the other dimension. Multi-core CPUs and GPUs are used to provide per-node speedup. We demonstrate a 76% efficiency when scaling from 64 GPUs to 4,096 GPUs. The paper provides reconstruction of a 204.8?×204.8? image (4M unknowns) executed on 4,096 GPUs in near-real time (almost 2 minutes). To the best of our knowledge, this is the largest full-wave inverse scattering solution to date, in terms of both image size and computational resources.

[1]  A. D. Greenwood,et al.  Adaptive and Parallel Surface Integral Equation Solvers for Very Large-Scale Electromagnetic Modeling and Simulation (Invited Paper) , 2015 .

[2]  Kan Xu,et al.  Multilevel fast multipole algorithm enhanced by GPU parallel technique for electromagnetic scattering problems , 2010 .

[3]  Siyuan Chen,et al.  Inverse scattering of two-dimensional dielectric objects buried in a lossy earth using the distorted Born iterative method , 2001, IEEE Trans. Geosci. Remote. Sens..

[4]  W. Chew,et al.  Reconstruction of two-dimensional permittivity distribution using the distorted Born iterative method. , 1990, IEEE transactions on medical imaging.

[5]  M. Born.  Principles of Optics : Electromagnetic theory of propagation , 1970 .

[6]  Weng Cho Chew,et al.  Numerical accuracy of multipole expansion for 2D MLFMA , 2003 .

[7]  Ignace Bogaert,et al.  Full-Wave Simulations of Electromagnetic Scattering Problems With Billions of Unknowns , 2015, IEEE Transactions on Antennas and Propagation.

[8]  Weng Cho Chew,et al.  Large inverse-scattering solutions with DBIM on GPU-enabled supercomputers , 2017, 2017 International Applied Computational Electromagnetics Society Symposium - Italy (ACES).

[9]  S. Velamparambil,et al.  10 million unknowns: is it that big? [computational electromagnetics] , 2003, IEEE Antennas and Propagation Magazine.

[10]  Weng Cho Chew,et al.  A frequency-domain formulation of the Fréchet derivative to exploit the inherent parallelism of the distorted Born iterative method , 2006 .

[11]  Rio Yokota,et al.  Petascale turbulence simulation using a highly parallel fast multipole method on GPUs , 2011, Comput. Phys. Commun..

[12]  Zhen Peng,et al.  Solving Problems With Over One Billion Unknowns by the MLFMA , 2012, IEEE Transactions on Antennas and Propagation.

[13]  E. Michielssen,et al.  A Multiplicative Calderon Preconditioner for the Electric Field Integral Equation , 2008, IEEE Transactions on Antennas and Propagation.

[14]  Qing Huo Liu,et al.  Reconstruction of 3D objects from multi-frequency experimental data with a fast DBIM-BCGS method , 2009 .

[15]  Weng Cho Chew,et al.  Scalable parallel DBIM solutions of inverse-scattering problems , 2017, 2017 Computing and Electromagnetics International Workshop (CEM).

[16]  Jiming Song,et al.  Fast Illinois solver code (FISC) , 1998 .

[17]  L. Gurel,et al.  Singularity of the magnetic-field Integral equation and its extraction , 2005, IEEE Antennas and Wireless Propagation Letters.

[18]  Roberto J. Lavarello,et al.  Tomographic Reconstruction of Three-Dimensional Volumes Using the Distorted Born Iterative Method , 2009, IEEE Transactions on Medical Imaging.

[19]  Jukka Sarvas,et al.  TRANSLATION PROCEDURES FOR BROADBAND MLFMA , 2005 .

[20]  Levent Gurel,et al.  An MPIxOpenMP implementation of the hierarchical parallelization of MLFMA , 2015, 2015 Computational Electromagnetics International Workshop (CEM).

[21]  Weng Cho Chew,et al.  Inversion of real transient radar data using the distorted-Born iterative algorithm , 1992, IEEE Antennas and Propagation Society International Symposium 1992 Digest.

[22]  Susan C. Hagness,et al.  Sensitivity of the Distorted Born Iterative Method to the Initial Guess in Microwave Breast Imaging , 2015, IEEE Transactions on Antennas and Propagation.

[23]  Ozgur Ergul,et al.  Fast and accurate analysis of large metamaterial structures using the multilevel fast multipole algorithm , 2009 .

[24]  Levent Gurel,et al.  Parallel out-of-core MLFMA on distributed-memory computer architectures , 2015, 2015 Computational Electromagnetics International Workshop (CEM).

[25]  Özgür Ergül,et al.  Hierarchical parallelization of the multilevel fast multipole algorithm (MLFMA) , 2013, Proceedings of the IEEE.

[26]  I. Bogaert,et al.  A Nondirective Plane Wave MLFMA Stable at Low Frequencies , 2008, IEEE Transactions on Antennas and Propagation.

[27]  K. Sertel,et al.  Multilevel fast multipole method solution of volume integral equations using parametric geometry modeling , 2001, IEEE Transactions on Antennas and Propagation.

[28]  M. Oelze,et al.  Density imaging using a multiple-frequency DBIM approach , 2010, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[29]  Jian-Ming Jin,et al.  An OpenMP-CUDA Implementation of Multilevel Fast Multipole Algorithm for Electromagnetic Simulation on Multi-GPU Computing Systems , 2013, IEEE Transactions on Antennas and Propagation.

[30]  A. Broquetas,et al.  Processing microwave experimental data with the distorted Born iterative method of nonlinear inverse scattering , 1993, Proceedings of IEEE Antennas and Propagation Society International Symposium.

[31]  Henrik Wallén,et al.  Broadband multilevel fast multipole algorithm for acoustic scattering problems , 2006 .

[32]  S. Velamparambil,et al.  Analysis and performance of a distributed memory multilevel fast multipole algorithm , 2005, IEEE Transactions on Antennas and Propagation.

[33]  W. Chew,et al.  Fast inverse scattering solutions using the distorted Born iterative method and the multilevel fast multipole algorithm. , 2010, The Journal of the Acoustical Society of America.

[34]  V. Okhmatovski,et al.  Low-Frequency MLFMA on Graphics Processors , 2010, IEEE Antennas and Wireless Propagation Letters.

[35]  M. Schmid.  Principles Of Optics Electromagnetic Theory Of Propagation Interference And Diffraction Of Light , 2016 .

[36]  Lexing Ying,et al.  Fast Directional Multilevel Algorithms for Oscillatory Kernels , 2007, SIAM J. Sci. Comput..

[37]  Qing Huo Liu,et al.  Three-dimensional reconstruction of objects buried in layered media using Born and distorted Born iterative methods , 2004, IEEE Geosci. Remote. Sens. Lett..

[38]  Jiming Song,et al.  Multilevel fast multipole algorithm for electromagnetic scattering by large complex objects , 1997 .

[39]  V. Rokhlin.  Diagonal Forms of Translation Operators for the Helmholtz Equation in Three Dimensions , 1993 .

[40]  Vinh Dang,et al.  GPU Cluster Implementation of FMM-FFT for Large-Scale Electromagnetic Problems , 2014, IEEE Antennas and Wireless Propagation Letters.

[41]  D. W. Winters,et al.  A Sparsity Regularization Approach to the Electromagnetic Inverse Scattering Problem , 2010, IEEE Transactions on Antennas and Propagation.

[42]  L. Gurel,et al.  A Hierarchical Partitioning Strategy for an Efficient Parallelization of the Multilevel Fast Multipole Algorithm , 2009, IEEE Transactions on Antennas and Propagation.

[43]  Makoto Taiji,et al.  42 TFlops hierarchical N-body simulations on GPUs with applications in both astrophysics and turbulence , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[44]  Weng Cho Chew,et al.  Seeing the invisible: Limited-view imaging with multiple-scattering reconstruction , 2018, 2018 United States National Committee of URSI National Radio Science Meeting (USNC-URSI NRSM).

[45]  Johannes Markkanen,et al.  Broadband Multilevel Fast Multipole Algorithm for Electric-Magnetic Current Volume Integral Equation , 2013, IEEE Transactions on Antennas and Propagation.

[46]  Weng Cho Chew,et al.  Scaling Analysis of a Hierarchical Parallelization of Large Inverse Multiple-Scattering Solutions , 2017 .

[47]  Levent Gurel,et al.  Shape reconstruction of three-dimensional conducting objects via near-field measurements , 2014, 2014 IEEE Antennas and Propagation Society International Symposium (APSURSI).

[48]  B. F. Logan,et al.  The Fourier reconstruction of a head section , 1974 .

[49]  Caicheng Lu.  A fast algorithm based on volume integral equation for analysis of arbitrarily shaped dielectric radomes , 2003 .

[50]  M. Oelze,et al.  A study on the reconstruction of moderate contrast targets using the distorted born iterative method , 2008, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[51]  Ramani Duraiswami,et al.  Fast multipole methods on graphics processors , 2008, J. Comput. Phys..