A Machine-Learning-Ready Dataset Prepared from the Solar and Heliospheric Observatory Mission

11 We present a Python tool to generate a standard dataset from solar images that allows for user-defined selection criteria and a range of pre-processing steps. Our Python tool works with all image products from both the Solar and Heliospheric Observatory (SoHO) and Solar Dynamics Observatory (SDO) missions. We discuss a dataset produced from the SoHO mission’s multi-spectral images which is free of missing or corrupt data as well as planetary transits in coronagraph images, and is temporally synced making it ready for input to a machine learning system. Machine-learning-ready images are a valuable resource for the community because they can be used, for example, for forecasting space weather parameters. We illustrate the use of this data with a 3-5 day-ahead forecast of the north-south component of the interplanetary magnetic field (IMF) observed at Lagrange point one (L1). For this use case, we apply a deep convolutional neural network (CNN) to a subset of the full SoHO dataset and compare with baseline results from a Gaussian Naive Bayes classifier. 12 1 Background & Summary 13 Studies based on physics models have shown that solar magnetic field captured with magnetograms contain crucial information 14 for estimating the speed of the solar wind, while the dynamical features of CMEs (angular width, initial speed, etc.) are 15 routinely inferred from coronagraph images1, 2. Hence, it is expected that applying machine learning (ML) techniques to 16 the high temporal coverage data of both the Solar and Heliospheric Observatory3 (SoHO) mission and the Solar Dynamics 17 Observatory4 (SDO) mission is a feasible venture, that can potentially improve the space weather forecasting capability of 18 current models5–7. With the quality of input data remaining paramount to the success of these ML-methods8–10 and to ensure 19 reproducible scientific research11, 12, the preparation of a community-wide standard dataset with a standard software is crucial. 20 At present, SoHO has provided more temporal coverage of the Sun than its successor, NASA’s SDO, and has also fully 21 covered Solar Cycle 23 and 24 with a suite of on-board instruments3 including those specific to solar imaging: the Michelson 22 Doppler Imager13 (MDI) for the solar photosphere, the Extreme ultraviolet Imaging Telescope14 (EIT) for the stellar atmosphere 23 to low corona, and the Large Angle and Spectrometric Coronagraph15 (LASCO) covering the corona from 1.5−30 Rs, detailed 24 in Table 1. Recently, a white-light coronal brightness index (CBI), constructed from the full LASCO C2 mission archive, 25 was used to explore correlations between the solar corona and several geophysical indices16. Although SDO has even higher 26 resolution and cadence, SoHO continues to uniquely provide coronagraph products and serves as a mission critical backup to 27 the SDO for solar flare and CME forecasting in the event of SDO failure. 28 Stanford University’s Joint Science Operation Center (JSOC) stores data from SoHO MDI, SDO HMI and AIA, and various 29 other solar instruments. The SunPy-affiliated package DRMS enables querying of these images17, 18. All of these individual 30 image products from JSOC are at the same processing level and are supplied in a Flexible Image Transport System (FITS) 31 format that contains only scalar values. The NASA Solar Data Analysis Center’s (SDAC) Virtual Solar Observatory19 (VSO) 32 tool enables data queries from a number of individual data providers. SDAC’s terabytes of available EIT and LASCO images 33 are also in FITS format. However, the SDAC data is highly heterogeneous. Not only are there intrinsic differences among 34 these SoHO products (e.g., individual cadence as shown in Table 1), but there is also an irregular assortment of image file 35 sizes and processing levels. These varying file sizes can correspond to different image resolutions, calibrations and orbital 36 maneuvers, and multi-frame recordings. In addition, all four of the publicly available EIT products require calibration to 37 ar X iv :2 10 8. 06 39 4v 1 [ as tr oph .S R ] 4 A ug 2 02 1