Adorym: A multi-platform generic x-ray image reconstruction framework based on automatic differentiation

We describe and demonstrate an optimization-based X-ray image reconstruction framework called Adorym. Our framework provides a generic forward model, allowing one code framework to be used for a wide range of imaging methods ranging from near-field holography to fly-scan ptychographic tomography. By using automatic differentiation for optimization, Adorym has the flexibility to refine experimental parameters including probe positions, multiple hologram alignment, and object tilts. It is written with strong support for parallel processing, allowing large datasets to be processed on high-performance computing systems. We demonstrate its use on several experimental datasets to show improved image quality through parameter refinement.

[1]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[2]  J. M. Cowley,et al.  The scattering of electrons by atoms and crystals. I. A new theoretical approach , 1957 .

[3]  A. Engström,et al.  X-Ray Microscopy , 1961, Nature.

[4]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[5]  J. Goodman.  Introduction to Fourier optics , 1969 .

[6]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[7]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[8]  W. Hoppe.  Beugung im inhomogenen Primärstrahlwellenfeld. I. Prinzip einer Phasenmessung von Elektronenbeungungsinterferenzen , 1969 .

[9]  D. DeRosier,et al.  The reconstruction of a three-dimensional structure from projections and its application to electron microscopy , 1970, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[10]  K. Ishizuka,et al.  A new theoretical and practical approach to the multislice method , 1977 .

[11]  J R Fienup,et al.  Reconstruction of an object from the modulus of its Fourier transform. , 1978, Optics letters.

[12]  Paul Lagasse,et al.  Beam-propagation method: analysis and assessment , 1981 .

[13]  J. D. O'Sullivan,et al.  A Fast Sinc Function Gridding Algorithm for Fourier Inversion in Computer Tomography , 1985, IEEE Transactions on Medical Imaging.

[14]  C. Werner,et al.  Satellite radar interferometry: Two-dimensional phase unwrapping , 1988 .

[15]  Jorge Herbert de Lira,et al.  Two-Dimensional Signal and Image Processing , 1989 .

[16]  B. L. Henke,et al.  X-Ray Interactions: Photoabsorption, Scattering, Transmission, and Reflection at E = 50-30,000 eV, Z = 1-92 , 1993 .

[17]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[18]  Arnold Neumaier,et al.  Solving Ill-Conditioned and Singular Linear Systems: A Tutorial on Regularization , 1998, SIAM Rev..

[19]  Ning Qian,et al.  On the momentum term in gradient descent learning algorithms , 1999, Neural Networks.

[20]  P. Cloetens,et al.  Holotomography: Quantitative phase tomography with micrometer resolution using hard synchrotron radiation x rays , 1999 .

[21]  Lisa Axe,et al.  Developments in synchrotron x-ray computed microtomography at the National Synchrotron Light Source , 1999, Optics & Photonics.

[22]  J. Miao,et al.  Extending the methodology of X-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens , 1999, Nature.

[23]  Andreas Griewank,et al.  Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation , 2000, TOMS.

[24]  M. Glas,et al.  Principles of Computerized Tomographic Imaging , 2000 .

[25]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[26]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[27]  S. Marchesini,et al.  X-ray image reconstruction from a diffraction pattern alone , 2003, physics/0306174.

[28]  Veit Elser.  Phase retrieval by iterated projections. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[29]  J. Rodenburg,et al.  Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm. , 2004, Physical review letters.

[30]  D Paterson,et al.  X-ray phase imaging: Demonstration of extended conditions for homogeneous objects. , 2004, Optics express.

[31]  J. Rodenburg,et al.  A phase retrieval algorithm for shifting illumination , 2004 .

[32]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[33]  Mario A. Storti,et al.  MPI for Python , 2005, J. Parallel Distributed Comput..

[34]  P. Cloetens,et al.  Optimization of phase contrast imaging using hard x rays , 2005 .

[35]  David M. Paganin,et al.  Coherent X-Ray Optics , 2006 .

[36]  A. G. Cullis,et al.  Hard-x-ray lensless imaging of extended objects. , 2007, Physical review letters.

[37]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[38]  S Marchesini,et al.  Invited article: a [corrected] unified evaluation of iterative projection algorithms for phase retrieval. , 2006, The Review of scientific instruments.

[39]  J. Rodenburg.  Ptychography and Related Diffractive Imaging Methods , 2008 .

[40]  J. Fienup,et al.  Phase retrieval with transverse translation diversity: a nonlinear optimization approach. , 2008, Optics express.

[41]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[42]  M. Grasmair,et al.  Anisotropic Total Variation Filtering , 2010 .

[43]  O. Bunk,et al.  Ptychographic X-ray computed tomography at the nanoscale , 2010, Nature.

[44]  John Shalf,et al.  Tuning HDF5 for Lustre File Systems , 2010 .

[45]  David George Voelz,et al.  Computational Fourier Optics: A MATLAB Tutorial , 2011 .

[46]  David A. Bader,et al.  Hierarchical Data Format , 2011, Encyclopedia of Parallel Computing.

[47]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[48]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[49]  J. Rodenburg,et al.  Ptychographic transmission microscopy in three dimensions using a multi-slice approach. , 2012, Journal of the Optical Society of America. A, Optics, image science, and vision.

[50]  M. Stampanoni,et al.  Regridding reconstruction algorithm for real-time tomographic imaging , 2012, Journal of synchrotron radiation.

[51]  J. Rodenburg,et al.  An annealing algorithm to correct positioning errors in ptychography. , 2012, Ultramicroscopy.

[52]  J. Rodenburg,et al.  Noise models for low counting rate coherent diffraction imaging. , 2012, Optics express.

[53]  Stephen C. Cain,et al.  Phase retrieval and Zernike decomposition using measured intensity data and the estimated electric field. , 2013, Applied optics.

[54]  Andreas Menzel,et al.  Reconstructing state mixtures from diffraction measurements , 2013, Nature.

[55]  Surendra Byna,et al.  A framework for auto-tuning HDF5 applications , 2013, HPDC.

[56]  Franz Pfeiffer,et al.  Near-field ptychography: phase retrieval for inline holography using a structured illumination , 2013, Scientific Reports.

[57]  A. Diaz,et al.  Translation position determination in ptychographic coherent diffraction imaging. , 2013, Optics express.

[58]  Francesco De Carlo,et al.  TomoPy: a framework for the analysis of synchrotron tomographic data , 2014, Journal of synchrotron radiation.

[59]  Ian K Robinson,et al.  Continuous scanning mode for ptychography. , 2014, Optics letters.

[60]  James R Fienup,et al.  Applications of algorithmic differentiation to phase retrieval algorithms. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[61]  Tom Peterka,et al.  Parallel ptychographic reconstruction. , 2014, Optics express.

[62]  T. Ishikawa,et al.  High-resolution multislice x-ray ptychography of extended thick objects. , 2014, Physical review letters.

[63]  O. Bunk,et al.  X-ray ptychographic computed tomography at 16 nm isotropic 3D resolution , 2014, Scientific Reports.

[64]  B. Lai,et al.  The Bionanoprobe: hard X-ray fluorescence nanoprobe with cryogenic capabilities , 2013, Journal of synchrotron radiation.

[65]  J. F. van der Veen,et al.  Diffraction-limited storage rings - a window to the science of tomorrow. , 2014, Journal of synchrotron radiation.

[66]  A. Menzel,et al.  On-the-fly scans for X-ray ptychography , 2014 .

[67]  Tony Lanzirotti,et al.  Scientific data exchange: a schema for HDF5-based storage of raw and analyzed data. , 2014, Journal of synchrotron radiation.

[68]  I. McNulty,et al.  Ptychographic overlap constraint errors and the limits of their numerical recovery using conjugate gradient descent methods. , 2014, Optics express.

[69]  Tom Peterka,et al.  Continuous Motion Scan Ptychography: Characterization for Increased Speed in Coherent X-ray Imaging References and Links Chemical Composition Mapping with Nanometre Resolution by Soft X-ray Microscopy, " Nature , 2022 .

[70]  Yambem Jina Chanu,et al.  Image Segmentation Using K -means Clustering Algorithm and Subtractive Clustering Algorithm , 2015 .

[71]  Michael Unser,et al.  Learning approach to optical tomography , 2015, 1502.01914.

[72]  Xiaojing Huang,et al.  Fly-scan ptychography , 2015, Scientific Reports.

[73]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[74]  Siu Kwan Lam,et al.  Numba: a LLVM-based Python JIT compiler , 2015, LLVM '15.

[75]  Pengtao Xie,et al.  Strategies and Principles of Distributed Machine Learning on Big Data , 2015, ArXiv.

[76]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[77]  Demetri Psaltis,et al.  Optical Tomographic Image Reconstruction Based on Beam Propagation and Sparse Regularization , 2016, IEEE Transactions on Computational Imaging.

[78]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[79]  Feng Yan,et al.  Phase calibration unwrapping algorithm for phase data corrupted by strong decorrelation speckle noise. , 2016, Optics express.

[80]  Tim Salditt,et al.  Regularized Newton methods for x-ray phase contrast and general imaging problems. , 2015, Optics express.

[81]  A. Diaz,et al.  X-ray ptychography with extended depth of field. , 2016, Optics express.

[82]  Peter Norvig,et al.  Deep Learning with Dynamic Computation Graphs , 2017, ICLR.

[83]  V. Morozov,et al.  Early Evaluation of the Cray XC 40 Xeon Phi System ‘ Theta ’ at Argonne , 2017 .

[84]  Kai He,et al.  Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection , 2017, Scientific Reports.

[85]  Wah Chiu,et al.  GENFIRE: A generalized Fourier iterative reconstruction algorithm for high-resolution 3D imaging , 2017, Scientific Reports.

[86]  Youssef S. G. Nashed,et al.  Distributed Automatic Differentiation for Ptychography , 2017, ICCS.

[87]  D. Gursoy.  Direct coupling of tomography and ptychography , 2017 .

[88]  Tim Salditt,et al.  Three-dimensional single-cell imaging with X-ray waveguides in the holographic regime , 2017, Acta crystallographica. Section A, Foundations and advances.

[89]  Doğa Gürsoy.  Direct coupling of tomography and ptychography. , 2017, Optics letters.

[90]  Denis Foley,et al.  Ultra-Performance Pascal GPU and NVLink Interconnect , 2017, IEEE Micro.

[91]  Peng Li,et al.  Multi-slice ptychographic tomography , 2018, Scientific Reports.

[92]  Priya Dwivedi,et al.  Lateral position correction in ptychography using the gradient of intensity patterns. , 2018, Ultramicroscopy.

[93]  M. Ge,et al.  Multi-slice ptychography with large numerical aperture multilayer Laue lenses , 2018 .

[94]  Ming Du,et al.  Tomosaic: efficient acquisition and reconstruction of teravoxel tomography data using limited-size synchrotron X-ray beams , 2018, Journal of synchrotron radiation.

[95]  Stefan M. Wild,et al.  3D X-Ray Imaging of Continuous Objects beyond the Depth of Focus Limit. , 2018, Optica.

[96]  J. Miao,et al.  Correlative 3D x-ray fluorescence and ptychographic tomography of frozen-hydrated green algae , 2018, Science Advances.

[97]  Manuel Guizar-Sicairos,et al.  Iterative least-squares solver for generalized maximum-likelihood ptychography. , 2018, Optics express.

[98]  Jorge Nocedal,et al.  Optimization Methods for Large-Scale Machine Learning , 2016, SIAM Rev..

[99]  Bart van Merrienboer,et al.  Automatic differentiation in ML: Where we are and where we should be going , 2018, NeurIPS.

[100]  Maik Kahnt,et al.  Coupled ptychography and tomography algorithm improves reconstruction of experimental data , 2019, Optica.

[101]  Charles C. Margossian,et al.  A review of automatic differentiation and its efficient implementation , 2018, WIREs Data Mining Knowl. Discov..

[102]  Michael T. McCann,et al.  Biomedical Image Reconstruction: From the Foundations to Deep Neural Networks , 2019, Found. Trends Signal Process..

[103]  David Blinder,et al.  Efficient algorithms for the accurate propagation of extreme-resolution holograms. , 2019, Optics express.

[104]  Yi Jiang,et al.  Instrumentation and method developments of x-ray ptychography at the Advanced Photon Source , 2019, Optical Engineering + Applications.

[105]  E. D. Cubuk,et al.  JAX, M.D.: End-to-End Differentiable, Hardware Accelerated, Molecular Dynamics in Pure Python , 2019, 1912.04232.

[106]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[107]  Avid,et al.  Efficient algorithms for the accurate propagation of extreme-resolution holograms , 2019 .

[108]  Marcus Carlsson,et al.  Four-Dimensional Tomographic Reconstruction by Time Domain Decomposition , 2018, IEEE Transactions on Computational Imaging.

[109]  Andrea Vedaldi,et al.  Small Steps and Giant Leaps: Minimal Newton Solvers for Deep Learning , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[110]  Yao-Lung L. Fang,et al.  Resolving 500 nm axial separation by multi-slice X-ray ptychography , 2019, Acta crystallographica. Section A, Foundations and advances.

[111]  Chris Jacobsen,et al.  Using automatic differentiation as a general framework for ptychographic reconstruction. , 2019, Optics express.

[112]  Marc Niethammer,et al.  Region-specific Diffeomorphic Metric Mapping , 2019, NeurIPS.

[113]  Weiwei Sun,et al.  Linearized Multi-Sampling for Differentiable Image Transformation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[114]  C. David,et al.  Towards optimized illumination for high-resolution ptychography. , 2019, Optics express.

[115]  Aik,et al.  Coupled ptychography and tomography algorithm improves reconstruction of experimental data , 2019 .

[116]  M. Guizar‐Sicairos,et al.  Ab initio nonrigid X-ray nanotomography , 2019, Nature Communications.

[117]  A. Demortière,et al.  X-ray Nanocomputed Tomography in Zernike Phase Contrast for Studying 3D Morphology of Li–O2 Battery Electrode , 2020 .

[118]  Mark F. Adams,et al.  Comparison of distributed memory algorithms for X-ray wave propagation in inhomogeneous media. , 2020, Optics express.

[119]  Henry N. Chapman,et al.  Ptychographic X-ray speckle tracking with multi-layer Laue lens systems1 , 2020, Journal of applied crystallography.

[120]  Ming Du,et al.  Near, far, wherever you are: simulations on the dose efficiency of holographic and ptychographic coherent imaging , 2019, Journal of applied crystallography.

[121]  Youssef S. G. Nashed,et al.  Three dimensions, two microscopes, one code: Automatic differentiation for x-ray nanotomography beyond the depth of focus limit , 2019, Science Advances.

[122]  Viktor Nikitin,et al.  Evaluation of modified uniformly redundant arrays as structured illuminations for ptychography , 2020 .

[123]  Joel Nothman,et al.  SciPy 1.0-Fundamental Algorithms for Scientific Computing in Python , 2019, ArXiv.

[124]  Mahdi Soltanolkotabi,et al.  3D Phase Retrieval at Nano-Scale via Accelerated Wirtinger Flow , 2020, 2020 28th European Signal Processing Conference (EUSIPCO).