Physics informed deep learning to super-resolve and cross-calibrate solar magnetograms

Super-resolution techniques aim to increase the resolution of images by adding detail. Compared to upsampling techniques reliant on interpolation, deep learning-based approaches learn features and their relationships across the training data set to leverage prior knowledge on what low resolution patterns look like in higher resolution images. As an added benefit, deep neural networks can learn the systematic properties of the target images (i.e.\ texture), combining super-resolution with instrument cross-calibration. While the successful use of super-resolution algorithms for natural images is rooted in creating perceptually convincing results, super-resolution applied to scientific data requires careful quantitative evaluation of performances. In this work, we demonstrate that deep learning can increase the resolution and calibrate space- and ground-based imagers belonging to different instrumental generations. In addition, we establish a set of measurements to benchmark the performance of scientific applications of deep learning-based super-resolution and calibration. We super-resolve and calibrate solar magnetic field images taken by the Michelson Doppler Imager (MDI; resolution ~2"/pixel; science-grade, space-based) and the Global Oscillation Network Group (GONG; resolution ~2.5"/pixel; space weather operations, ground-based) to the pixel resolution of images taken by the Helioseismic and Magnetic Imager (HMI; resolution ~0.5"/pixel; last generation, science-grade, space-based).

[1]  Steven C. H. Hoi,et al.  Deep Learning for Image Super-Resolution: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Y. Moon,et al.  Super-resolution of SDO/HMI Magnetograms Using Novel Deep Learning Methods , 2020, The Astrophysical Journal.

[3]  Jaime Fern'andez del R'io,et al.  Array programming with NumPy , 2020, Nature.

[4]  Yoshua Bengio,et al.  HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery , 2020, ArXiv.

[5]  Kevin Reardon,et al.  The SunPy Project: Open Source Development and Status of the Version 1.0 Core Package , 2020, The Astrophysical Journal.

[6]  Joel Nothman,et al.  SciPy 1.0-Fundamental Algorithms for Scientific Computing in Python , 2019, ArXiv.

[7]  Kamal Jain,et al.  A Technical Review on Image Super-Resolution Techniques , 2020 .

[8]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[9]  Wei Wang,et al.  Deep Learning for Single Image Super-Resolution: A Brief Review , 2018, IEEE Transactions on Multimedia.

[10]  K. Mursula,et al.  Photospheric and coronal magnetic fields in six magnetographs III. Photospheric and coronal magnetic fields in 1974–2017 , 2019 .

[11]  Miguel de Val-Borro,et al.  The Astropy Project: Building an Open-science Project and Status of the v2.0 Core Package , 2018, The Astronomical Journal.

[12]  A. Asensio Ramos,et al.  Enhancing SDO/HMI images using deep learning , 2017, ArXiv.

[13]  Mohammad Norouzi,et al.  Pixel Recursive Super Resolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Xiaogang Wang,et al.  Learnable Histogram: Statistical Context Features for Deep Neural Networks , 2016, ECCV.

[15]  T. Duvall,et al.  Observables Processing for the Helioseismic and Magnetic Imager Instrument on the Solar Dynamics Observatory , 2016, 1606.02368.

[16]  Thuy Mai,et al.  Solar Dynamics Observatory (SDO) , 2015 .

[17]  Emmanuelle Gouillart,et al.  scikit-image: image processing in Python , 2014, PeerJ.

[18]  Jesper Schou,et al.  The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: Overview and Performance , 2014, 1404.1881.

[19]  M. Ben‐Nun,et al.  A Multi-Observatory Inter-Comparison of Line-of-Sight Synoptic Solar Magnetograms , 2014 .

[20]  Luca Bertello,et al.  The Solar Cycle , 2012 .

[21]  Yang Liu,et al.  Comparison of Line-of-Sight Magnetograms Taken by the Solar Dynamics Observatory/Helioseismic and Magnetic Imager and Solar and Heliospheric Observatory/Michelson Doppler Imager , 2012 .

[22]  C. J. Wolfson,et al.  Design and Ground Calibration of the Helioseismic and Magnetic Imager (HMI) Instrument on the Solar Dynamics Observatory (SDO) , 2012 .

[23]  J. T. Hoeksema,et al.  The Helioseismic and Magnetic Imager (HMI) Investigation for the Solar Dynamics Observatory (SDO) , 2012 .

[24]  J. Borrero,et al.  Magnetic Structure of Sunspots , 2011, Living reviews in solar physics.

[25]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[26]  David R. Chesney,et al.  Space Weather Modeling Framework: A new tool for the space science community , 2005, Journal of Geophysical Research.

[27]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[28]  J. Linker,et al.  Magnetohydrodynamic modeling of the solar corona during Whole Sun Month , 1999 .

[29]  V. Domingo,et al.  The SOHO mission: An overview , 1995 .

[30]  Alan M. Title,et al.  The solar oscillations investigation - Michelson Doppler Imager. , 1992 .

[31]  Irwin Sobel,et al.  An Isotropic 3×3 image gradient operator , 1990 .

[32]  William Charles Livingston,et al.  The Global Oscillation Network Group (GONG) , 1988 .