© M. Núñez et al., Published by EDP Sciences 2019

1 Introduction

Solar Electromagnetic (EM) emissions from the sun during the impulsive phase of flare-Coronal Mass Ejection (CME) events are important for space weather. These emissions are manifestations of the energy released and particle acceleration during the beginning of the flare, and the eruption of the CME and its corresponding shock (Temmer et al., 2010), which finally accelerate solar protons and ions. A well-connected Solar Energetic Proton (SEP) event takes place when these particles propagate along the interplanetary magnetic field lines, and reach the Earth. Forecasting these SEP events helps to improve mitigation of adverse effects on humans and technology in space (Hoff et al., 2004; Durante & Cucinotta, 2011; Shea & Smart, 2012) and on passengers and flight crews on polar airline routes (Beck et al., 2005).

The use of flare data for predicting CME-driven shock related Interplanetary (IP) phenomena, such as gradual SEP event onset and shock arrival times, is supported by a close relationship between flares and CMEs, which has been reported in a number of studies: Chen & Kunkel (2010) concluded that the poloidal flux injection, the driver of CME’s flux rope eruptions, is also physically related to X-ray signatures; Yashiro & Gopalswamy (2009) reported that the fraction of flares accompanied by CMEs increases with flare energy, until X-class (peak GOES 1–8 Å flux >10⁻⁴ W m⁻² as seen at Earth) when essentially all flares are accompanied by a CME; Pick & Vilmer (2008) and Reeves & Moats (2010) found that, for a particular reconnection rate, the CME peak acceleration and the peak GOES flux are well correlated by a power-law relationship. Jain et al. (2010) showed that the speed of CMEs increases with the plasma temperature of X-ray flares; Núñez et al. (2016) presented empirical evidence that the speed of CME-driven IP shocks are correlated with soft X-ray flares.

At present, the empirical and operational SEP forecasting methods rely on solar EM radiation as a basic ingredient rather than the CME observations, to predict well-connected SEP events. The most common EM emissions used for SEP event forecasting are Soft X-rays (SXR) and/or radio emissions (Kahler et al., 2007; Balch, 2008; Laurenza et al., 2009; Núñez, 2011, 2015; Marsh et al., 2014; Dierckxsens et al., 2015; Kahler & Ling, 2015, Papaioannou et al., 2015; Winter & Ledbetter 2015; Alberti et al., 2017; Núñez et al., 2017; Laurenza et al., 2018). The use of in-situ particle data has also been useful for predicting SEP events. The RELEASE model (Posner, 2007) and the Poorly Connected Prediction (PCP) model of the UMASEP-10 tool (Núñez, 2011) make SEP event predictions from in-situ particle data only, electrons and protons, respectively. The UMASEP scheme (Núñez, 2011, 2015) was originally proposed to make its predictions from SXR and in-situ proton data. This paper studies, for the first time, the use of EUV data and proton data with the purpose of possibly improving the forecasting performance for predicting well-connected >10 MeV SEP events at Earth; this scheme would also allow using STEREO observations for off Sun–Earth line predictions.

Currently, EUV observations are not used in any real-time SEP event prediction system; however, several studies have found close relationships between SEP events and EUV waves, which are considered as the lateral expansion of CME-driven shocks. Park et al. (2015) found that faster EUV waves are related to the acceleration of SEPs of higher fluxes and energies, and concluded that EUV wave speeds represent the strengths of the lateral coronal disturbances in CME-driven shocks. Kozarev et al. (2015) combined remote EUV observations with data-driven models in order to deduce coronal shock properties relevant to the local acceleration of SEPs and their heliospheric connectivity to near-Earth space, focusing on the evolution of the EUV waves. On the other hand, Lario et al. (2014) showed that the extent of the EUV wave cannot be used reliably as a proxy for the longitudinal extension of the SEP events in the heliosphere. In this regard, Park et al. (2015) also found that the longitudinal extent of SEPs is not always consistent with EUV waves because they are affected by the ambient of the low corona region.

In this paper we study the use of EUV observations in the UMASEP scheme and analyze the possibility of using them in real-time operations. The UMASEP scheme has been used to develop several real-time prediction tools: UMASEP-10 (Núñez, 2011), UMASEP-100 (Núñez, 2015) and HESPERIA UMASEP-500 (Núñez et al., 2017; Núñez, 2018) which predict >10 MeV, >100 MeV and >500 MeV SEP events, respectively. Since 2010, the UMASEP-10’s forecasts are disseminated by NASA’s integrated Space Weather Analysis system (iSWA), and the model has shown promise on an operational level (Tsagouri et al., 2013). The UMASEP-10 tool was also included as a module in the European Space Agency’s SEPsFLAREs system (García-Rigo et al., 2016).

The paper is organized as follows: Section 2 presents the input data and the target SEP events used in this study. Section 3 summarizes the functioning of the UMASEP scheme. Section 4 presents the results from the developed models for predicting well-connected >10 MeV SEP events using GOES EUV and SDO AIA data. Section 5 studies the convenience of using AIA EUV data in the UMASEP-10 tool. The conclusions are presented in Section 6.

2 The data

This section presents the input data and targets events that are used in this study. The UMASEP-based prediction models developed for this study use 5-min EUV and energetic proton data. These models use data from the Extreme Ultraviolet Sensor (EUVS) on board GOES 13 and 15, and from the Atmospheric Imaging Assembly (AIA) instrument on board the Solar Dynamics Observatory (SDO). The proton data are from six differential energy channels in the range 9–500 MeV of the instrument Electron, Proton, Alpha Detector (EPEAD) aboard the GOES satellites.

We built the EUV-based SEP forecasting models by using calibration data from May 2010 to October 2014; then we validated the models using out-of-sample data from November 2014 to December 2017. Regarding the calibration phase, we decided to divide it into two steps. The first calibration step was to develop three UMASEP-based predictors using data from the EUVS instrument on board GOES 13 and 15 satellites, with the purpose of identifying the most promising wavelength intervals; and in the second calibration step, taking into account the results of the first step, we selected three wavelengths in AIA data to develop other three UMASEP-based predictors.

The EUVS instrument (Evans et al., 2010) aboard the GOES satellites provides five broadband EUV spectral irradiance channels in the range 50–1250 Å. Currently, there are publicly available data from the EUV-A (50–150 Å), EUV-B (250–340 Å) and EUV-E (1180–1270 Å) channels. Although there are gaps in each of the versions (2 and 4) of 5-min EUV data, we selected the interval from May 2010 to October 2014 because on each day of the interval at least one satellite provided EUV data.

The SDO/AIA instrument images the Sun in seven EUV and three UV–visible-light channels (Boerner et al., 2012). The temporal cadence of the EUV channels is 12 s. For each image, the EUV intensity mean over the disk is available in the file header. In this study, we build a 5-min time history from SDO AIA images from May 2010 to December 2017, by calculating 5-min EUV intensity averages over 12-s EUV intensity means. The obtained EUV time series show a slowly fluctuating background which was removed. This background is calculated from the available information (i.e., the 5-min EUV time history) at the time of the prediction (see Sect. 3.1 for details).

During the period from May 2010 to December 2017, 42 SEP events surpassed the threshold of J (E > 10 MeV) > 10 pfu, according to the SWPC/NOAA SEP list (ftp://ftp.swpc.noaa.gov/pub/indices/SPE.txt). Richardson et al. (2014) show that there is a continuum of delays of event onsets that extends well behind the western solar limb, up to a longitude of ~W140. During this period, five events were associated with behind-the-limb flares. For this reason, this paper has refined the definition proposed in Núñez (2011) for prompt events such as those SEP events where either the associated flare took place in the range W60–W140, or whose promptness was less than 8 h. As in Núñez (2011), the promptness of an SEP event is defined as the temporal distance from the flare peak time, PT (peak in SXR flux by GOES), to the SEP event Start Time (ST). In this paper, the SEP event start time is the time when the >10 MeV proton flux exceeds a threshold of 10 pfu (1 pfu = 1 pr cm⁻² s⁻¹ sr⁻¹) for three consecutive 5 min points. Table 1 lists the 32 prompt >10 SEP events (all of which are well-connected events, so we use the term “well-connected” as an equivalent of “prompt”) from the NOAA/NASA SEP list from proton data measured by GOES spacecraft at Geosynchronous orbit. For the aforementioned five events, the approximated location and time of the associated flare is presented. Column 1 lists the SEP event start times; column 2 the event promptness; column 3 the SXR peak times of the associated flare; column 4 the flare class and column 5 the flare location.

3 The model

Within the UMASEP scheme, there is the Well-Connected event Prediction (WCP) approach that tries to identify precursors of well-connected events by empirically estimating the magnetic connectivity from the associated flare/CME in the solar corona to the near-Earth environment. In order to estimate the magnetic connectivity, this approach makes a lag-correlation of bit-based transformations (explained in Sect. 3.1) of solar EM data and the first derivatives of differential particle fluxes in the near-Earth environment. If the solar EM peak flux of the associated flare is greater than a certain threshold (independently of the flare’s heliolongitude), an SEP event prediction is issued. This approach, henceforth called the UMASEP/WCP scheme, is summarized in this section.

Regarding solar EM data, this scheme has been used with SXR flux (Núñez, 2011, 2015; Núñez et al., 2017), and microwave (MW) flux density at 5 and 9 GHz (Zucca et al., 2017). Regarding in-situ particle data, this scheme has been used with differential proton fluxes; recently, it has also been used with relativistic electron data (Núñez, 2018).

Based on the UMASEP/WCP scheme, each tool uses its own WCP model. It is important to say that, although the first versions of the WCP model of the UMASEP-10 tool (Núñez, 2011) used a continuous-based WCP model, the current version of this tool (v1.5) uses the bit-based model (summarized in Sect. 3.1); this model is henceforth called WCP-sxr and its forecasting results are compared with those of the WCP-euv models in Section 4.

3.1 The UMASEP/WCP scheme

In general, the UMASEP/WCP scheme works as follows: Firstly, it generates a bit-based time series from the solar EM data and several bit-based time series from the time derivatives of each of the differential in-situ particle data. The solar EM-based “1s” are the occurrences of sufficiently large EM fluxes, which are considered in this scheme as signatures of particle acceleration. A sufficiently large value is that which surpasses a percentage p of the maximum value in the present time series of size L, below which no SEP event prediction will be made; otherwise, the flux level is transformed into a “0”. To avoid false alarms due to relatively strong fluctuations during periods of low solar activity, a threshold d is necessary as a minimum solar EM flux, which is the minimum value needed to consider it a positive fluctuation (i.e., a “1”). This forecasting approach creates a list of cause-consequence pairs as follows: it takes the first “1” of the bit-based solar EM-based time series, and the first posterior “1” of the bit-based particle-based time series, to create a pair; it then takes the second pair of “1s” in each time series, and thus successively, until all the “1s” of the solar EM-based time series are inspected.

Figure 1 illustrates the identification of pairs from the time series of solar EM-based “1s” (i.e., ⁺A time series) and the time series of differential-proton flux-based “1s” (i.e., ⁺B time series). Note that i, the first “1” (also called fluctuation) in ⁺A (in red), is paired to j (also in red). Note that the last fluctuation in ⁺A cannot be paired because there is nothing left in ⁺B to pair it with, and the first two fluctuations in ⁺B are unpaired fluctuations, because there are no possible causing fluctuations in ⁺A. Once all pairs have been discovered, we need to calculate the mean and standard deviations of all pair separations to calculate the fluctuation correlation at the current time t, as follows:

(1)where, Pairs_t is the number of pairs; Mean_t is the mean temporal separation in all pairs (i.e., between the solar EM “1” and the corresponding proton-based “1s”); SD_t is the standard deviation of the temporal separations in all pairs; and, Odds_t is the sum of the number of unpaired solar EM-based “1s” and the number of unpaired proton-based “1s”. The fluctuation correlation is a value between 0 and 1. An ideal magnetic connection is detected by the UMASEP/WCP scheme, when a sequence of solar EM-based “1s” in a row is followed by a sequence of particle-based “1s” in a row in a window of length L. We say that this ideal magnetic connection would have a fluctuation correlation of 1.

Fig. 1 This figure shows two bit-based series. The “1s” of the top time series shows extreme values from 5-min averaged EUV-based data. The “1s” of the bottom time series is extreme time derivatives from a 5-min differential proton flux. This figure also shows the identification of three “cause-consequence” pairs. A pair, shown in red, is composed of a “1” at time (i) of the top time series and a “1” at time (j) in the bottom time series, and the corresponding pair separation.

Finally, an SEP event prediction is triggered when both the fluctuation correlation and the associated flare are large. A large fluctuation correlation is when it is greater than a threshold r. The threshold r is the minimal fluctuation correlation required to infer that there is empirical evidence that the EM-proton fluxes correlation is due to a magnetic connection between a solar flaring region and the near-Earth environment. The r threshold is empirically found to obtain a high Probability of Detection (POD) and a low FAR for predicting prompt SEP events (see Sect. 3.2). For the case of the EUV-based models, the associated flare is large when the associated solar EM intensity peak is greater than a threshold f, after having removed the recent L-size background. The background at a time t, is calculated as the average of the 5-min EUV intensities from t to t − L, where t is the time when the SEP forecast is issued, without considering those EUV fluxes associated with solar EM “1s”, which are the largest EUV values in the interval. In this paper, we simulate the functioning of the WCP models in real-time; therefore, all estimations are calculated from the available information at the time of the prediction.

It is important to say that all available GOES proton measurements are considered in the analysis explained in this section, regardless on any gap or quality problem; fortunately, GOES measurements are data with very good quality, so gaps or spikes are very scarce. A GOES satellite has six proton channels in the range 9–500 MeV; therefore, if two GOES satellites are available at certain time (e.g., GOES 13 and GOES 15), 12 proton-based time series are correlated with the (single) solar EM time series (e.g., the 94 Å-based time series); so, any prediction triggered in any of the 12 bi-time-series analyses, triggers an SEP event prediction.

It is also important to mention that, as in all WCP approaches, the EUV-based approaches can predict the integral proton flux that will be attained 7 h after the time of the prediction. The procedure is summarized as follows: the >10 MeV integral proton flux 7 h after the time of the prediction, called I _7h , is calculated as:

(2)where a and b are linear regression factors that were empirically found with observed I _7h values in historical well-connected SEP events that took place in solar cycles 22 and 23; FC_max is the maximum fluctuation correlation value calculated from the EUV flux and proton fluxes (see above), and F is the time-integral of the recent EUV flux calculated from near the flare onset to the flare peak. For more information about the aforementioned formula, see Núñez (2011). As in the WCP approaches, the EUV-based approaches cannot predict the SEP peak intensity; however, the predicted I _7h may be analyzed by a larger system for the purpose of predicting SEP event peak intensities; García-Rigo et al. (2016) present the use of the UMASEP/WCP of UMASEP-10 as a component of a larger system for predicting the SEP intensity time profile.

3.2 Calibration of WCP-euv models

Since we use different EUV data sources, we have to find a set of threshold values (summarized in Sect. 3.1) for each of the six WCP-euv models that are evaluated in Section 4. As explained in Section 3.1, the solar EM-based “1s” should be related with signatures of particle acceleration. Hard X-ray (HXR) and MW observations provide direct diagnostics of energy release and particle acceleration in solar flares (Warmuth et al., 2009); for this reason, in Zucca et al. (2017) the solar EM-based “1s” are the extreme MW density flux levels. On the other hand, SXRs are measurements of thermal emissions from the hot corona, which are not signatures of particle acceleration; however, according to Neupert (1968), the time derivatives of SXRs show an intensity-time profile that is similar to that in MWs or HXRs in most flares (mainly during the impulsive phase); for this reason, in the WCP-sxr (Núñez, 2011, 2015; Núñez et al., 2017) model, the “1s” are extreme values of time derivatives of SXRs.

In order to obtain the solar EM-based “1s” (i.e., those extreme EM-related values presumably associated with particle acceleration processes), we could use either the largest time derivatives of EM flux (as done using SXRs in the WCP scheme e.g., Núñez, 2011, 2015; Núñez et al., 2017) or the largest EM flux (as done using MWs in Zucca et al., 2017). Thus, we decided to discover the best use of EUV data by evaluating both approaches: the largest time derivatives of the EUV flux and the largest EUV flux values. We found that the use of time derivatives obtained very poor results, which contrasted with the very good results obtained using the largest EUV flux values. For this reason, this study refers to the results using the extreme values of the EUV time series for obtaining the solar EM “1s” in the WCP-euv models.

In this study, each WCP model calibration is done as an optimization process, whose purpose was to obtain a set of thresholds that maximizes the POD prompt SEPs, and minimizes the FAR. In general, the POD = A/(A + C) and FAR = B/(A + B), where A is the number of successful prompt SEP event forecasts, B is the number of false forecasts using the prompt-oriented forecasts, and C is the number of missed prompt SEP event events.

For this purpose, this study uses the Critical Success Index (CSI), applied to prompt SEP events, which is a combination of POD and FAR as follows: CSI_prompt = [POD_prompt ⁻¹ + (1 − FAR_prompt)⁻¹ – 1]⁻¹. CSI is a commonly used performance metric in atmospheric forecasting studies. A CSI of 100% is the indication of an excellent predictor with POD = 100% and FAR = 0%.

To find a highly effective set of parameters L, p, d, f, and r (although not necessarily the best one), we first run the WCP model using data of the training period (2014–2017), with all sets of parameters resulting by varying the values with low resolution steps, by selecting a number of values (10) equally distributed in the whole range of each parameter. For the best two configuration sets with the highest CSI found, we applied a new search by using higher resolution steps nearby the solutions found in the previous step, by selecting values (10) in each parameter separated by a distance that is 50% shorter than that used in the previous iteration (avoiding the repetition of tests with very similar sets of parameter values). We repeated the process until the highest CSI_prompt was reached over the studied time interval. As a result of each model calibration, we obtain a set of calibration values for each of the developed WCP models. We empirically found the following parameters and thresholds for all the EUV-based models: the size of L was 7 h, the percentage p was 91%, and the threshold d was 0.0001 DN s⁻¹. The f thresholds using 94 Å, 171 Å and 304 Å were 0.037 DN s⁻¹, 0.6 DN s⁻¹ and 0.03 DN s⁻¹, respectively.¹ Note that an SEP event prediction from each EUV dataset is triggered when both the fluctuation correlation is ≥r and the EUV intensity is ≥f.

3.3 Forecast output of WCP-euv models

Figure 2 shows the forecast graphical output that an operator would have seen if the WCP-euv304 model, which uses a EUV time series (from 5-min averages over 12-s EUV intensity means), and the WCP-sxr model, which uses 5-min GOES SXR data, had processed the event that started at 6:35 UT on August 4, 2011, with real-time data. Figure 2a and b shows the WCP-euv304 predictions before and after the SEP event, respectively. Figure 2c and d shows the WCP-sxr predictions before and after the same event. The upper time series of these figures shows the observed integral proton flux with energies greater than 10 MeV. The current flux is indicated below the label “now” at each image. The forecast of integral proton flux is presented to the right of this label. The yellow/orange-colored band indicates the expected evolution of the integral proton flux derived from the prediction of the proton flux. Note that Figure 2a shows the WCP-euv304 prediction at 4:40 on August 4. Figure 2c shows that WCP-sxr issued the prediction at the same time as WCP-euv304. For this event, these two models issued the prediction 1 h 55 min before the SEP event start time. The central curve in each panel displays the solar EM flux, and the lower time series shows the magnetic connectivity estimation time series. When a forecast is issued, the graphical output also shows the inferences about the associated flare, heliolongitude and active region.

Fig. 2 Outputs of WCP-euv304 and WCP-sxr models after processing data at 6:35 UT on August 4, 2011. (a) shows the prediction of WCP-euv304 at 4:40 UT, (b) the subsequent evolution of the >10 MeV integral proton flux after WCP-euv304 prediction, (c) the prediction of WCP-sxr, which was issued at the same time, and (d) the subsequent evolution of the >10 MeV integral flux after the WCP-sxr prediction.

4 Evaluation

We built the EUV-based SEP forecasting models by using calibration data and validated them using out-of-sample data. Section 4.1 presents the forecasting results using GOES EUVS and SDO AIA calibration data from May 2010 to October 2014. Section 4.2 presents the forecasting results using out-of-sample SDO AIA data from November 2014 to December 2017.

4.1 Evaluation results using calibration data

The calibration phase was carried out in two steps; firstly, we calibrated and evaluated three WCP models using 52 months of GOES EUV and proton data (i.e., the presented evaluation results are those obtained with the same calibration data); these models were WCP-euvA, WCP-euvB and WCP-euvE, which used data from the channels EUV-A (50–150 Å), EUV-B (250–340 Å), and EUV-E (1180–1270 Å), respectively. We tuned the model parameters explained in Section 3.2. The best results using GOES EUV data were obtained by the WCP-euvA and WCP-euvB models, as shown below in Tables 3 and 4. In the second calibration step, we selected the 94 Å, 171 Å and 304 Å SDO/AIA channels because they are in the same wavelength range (or very near) that showed the best forecasting performance using GOES EUV data. From the AIA images, we constructed three 5-min time series of EUV data (see Sect. 2 for details). Then we constructed the WCP-euv94, WCP-euv171 and WCP-euv304 models for predicting all SEP events in Table 1. We reused the WCP models developed in the first calibration step as follows: the calibration thresholds of WCP-euv94 and WCP-171 were the same as those in WCP-euvA (which were obtained from EUV data in the range 50–150 Å), with the exception of the f threshold. The calibration thresholds of WCP-euv304 were the same of those in WCP-euvB, which were obtained using EUV data in the range 250–340 Å, with the exception of the f threshold. In general, the f thresholds of all WCP models must be different because they depend on the EUV irradiance levels of the source data, which are different for several reasons, the most important being that the sun emits different EUV irradiance fluxes at different wavelengths. The quiescent Sun emits more EUV at longer wavelengths; on the other hand the signal-to-noise for eruptive-to-quiescent tends to higher at lower wavelengths.

Table 4 presents the forecasting results of the six aforementioned WCP models using calibration data for predicting those events in Table 1 that took place from May 2010 to October 2014. Column 1 gives the SEP event start times (ST), columns 2–13 the forecast results (in terms of “hits” and “misses”) and warning times of the WCP-euvA, WCP-euvB, WCP-euvE, WCP-euv94, WCP-euv171 and WCP-euv304, respectively.

It is important to note that there is a very low probability that SEP events are associated with solar parent events that take place between E20–E90. Table 2 lists the false alarms issued by using 5-min EUV and SXR data during the calibration period. Column 1 presents the data and time of the false alarms. Column 2 lists the times at which the false alarms were issued and the EM data (EUV or SXR wavelength) used shown in brackets. Column 3 and 4 shows the peak time and location of the EUV or SXR flare that caused the corresponding false alarm. It is important to mention that well-connected SEP events are associated with flaring regions in the central and western parts of the Sun. According to the SEP list in ftp://ftp.swpc.noaa.gov/pub/indices/SPE.txt, only 0.4% of prompt SEP events (i.e., see definition of event promptness in the footnote of Table 1) are associated with flaring regions east of E20. Note that false alarm #3 was triggered by an AIA flare at 171 Å that took place east of E20 and, therefore it may be filtered out,² as shown in the corresponding results in Table 3. Regarding Table 2, it is important to say that during the first and fourth time stamps (i.e., March 21st, 2011 and December 14, 2012) there were particle enhancements; however, they did not reach the SWPC SEP threshold for >10 MeV. The existence of particle enhancements is the reason why those flares gave a false alarm.

Table 3 shows the summary of the forecasting performance of the SXR-based and the EUV-based models in terms of POD_prompt, FAR_prompt, Average Warning Time (AWT_prompt), Median Warning Time (MWT_prompt) and CSI_prompt using calibration data from May 2010 to October 2014. The POD_prompt = A _prompt/(A _prompt + C _prompt) and FAR_prompt = B _prompt/(A _prompt + B _prompt), where A _prompt is the number of successful prompt SEP event forecasts, B _prompt is the number of false forecasts using the prompt-oriented forecasts, and C _prompt is the number of missed prompt SEP event events. The warning time is the temporal distance from the prediction time to the prompt SEP event start time (i.e., the time when the >10 MeV integral proton flux surpasses 10 pfu for three consecutive 5 min). The forecasting performance for all the >10 MeV SEP events (e.g., POD_all) is analyzed in Section 5. Although the current version of the software component corresponding to the SXR-based WCP model is 1.5, an old version (v1.3) of this component may be compared with the AIA EUV-based models, because it was calibrated with data up to October 2014; for this reason, the last column of Table 3 presents the forecasting results of the WCP-sxr (v1.3) model. Table 3 shows that the CSI_prompt of the AIA EUV-based models in the range 50–340 Å were the range 67.9–78.6% and the CSI_prompt of the SXR-based model (v1.3) was 63% using calibration data for the period 2010–2014. The MWT_prompt of the AIA EUV-based models were in the range 34–45 min and the MWT_prompt of the SXR-based model was 45 min.

4.2 Evaluation using out-sample data

In this section we took the WCP-euv94, WCP-euv171 and WCP-euv304 models obtained in Section 4.1 and make them predict the SEP events in Table 1 using out-of-sample data from November 2014 to December 2017. Table 6 lists the corresponding forecasting results compared with those of the WCP-sxr model (v1.3), using out-of-sample data from the same period. Column 1 gives the SEP event start times (ST), columns 2–9 the forecast results (in terms of “hits” and “misses”) and warning times of the WCP-euv94, WCP-euv171, WCP-euv304 and WCP-sxr models, respectively. Table 7 lists the false alarms issued during this period. Column 1 presents the date and time of the false alarms. Column 2 lists the times at which the false alarms were issued and EM data (EUV or SXR wavelength) used, shown in brackets. Column 3 and 4 shows the peak time and location of the EUV/SXR flare that caused the corresponding false alarm. Since all these false alarms were triggered by an EUV flare that took place west of E20, no false alarm may be filtered out during the validation period. Note that the flare that took place at N11W50 around 18:00 on September 20, 2015, made three models (WCP-euv94, WCP-euv304 and WCP-sxr) trigger false alarms, mainly because it was a medium-sized flare (M2.1) and strong proton enhancements were observed by GOES in the near-Earth environment. These predictions were not successful because the posterior >10 MeV integral proton flux did not surpass 10 pfu; this flux reached 3 pfu only.

Table 8 presents the summary of the validation results of the WCP-euv94, WCP-euv171 and WCP-euv304 models in terms of POD_prompt, FAR_prompt, MWT_prompt, AWT_prompt and CSI_prompt. Note that WCP-euv304 obtained good results in terms of POD_prompt (100%) but the worst performance in terms of FAR_prompt (40%). The WCP-euv171 and the WCP-sxr obtained the same results in terms of POD_prompt (83.3%), FAR_prompt (16.67%) and AWT_prompt (2 h 10 min). The best result in terms of CSI_prompt using out-of-sample data was obtained by WCP-euv94, a model that was also in the leading group in forecasting performance using calibration data. In summary, the validation phase, which was carried out using out-of-sample SDO AIA only, allowed us to conclude that the use of EUV data in the range 50–340 Å in the UMASEP scheme yields higher (better) or similar PODs than those obtained by the SXR-based model, and higher (worse) or similar FAR compared to that obtained by the SXR-based model. Although the statistical confidence presented in Table 8 is low using out-of-sample data because the performance is calculated using six events, these results are consistent with those using calibration data (see Table 3), which are results with higher statistical confidence, because they used 26 events.

It is important to mention that the FAR obtained by SEP event prediction models relying on solar data only (Kahler et al., 2007; Balch, 2008; Laurenza et al., 2009) or in-situ particle data only (Posner, 2007), is in the range 40%-45%; however, the FAR of the UMASEP/WCP approaches presented in Table 8, is in the range 16.7–40%, using either SXR or EUV out-of-sample data. The reason for the comparatively-low FAR is due to the fact that in the WCP scheme both solar and in-situ particle data are analyzed; that is, more evidence is taken into consideration to predict that proton flux will surpass 10 pfu. On the other hand, the drawback of the WCP models is a lower warning time on average with respect to other techniques (e.g., Posner, 2007, Laurenza et al., 2009; St. Cyr et al., 2017).

4.3 Analysis of forecasting results of the WCP-euv models

According to Table 1, during the period from May 2010 to December 2017, 25% (8/32) of the prompt SEP events were associated with either front-side <C4 class flares or behind-the-limb flares, which are difficult to predict by current forecasting approaches because the observed solar EM intensities are very faint (if any). This study has found that the main reason for the high number of hits using both calibration and out-of-sample EUV data is due to the prediction of this subset of SEP events. This section analyzes these cases in more detail.

4.3.1 SEP events associated with front-side <C4 class flares

The current WCP-sxr model and some solar EM-based SEP event predictors (Balch, 2008, Kahler & Ling, 2015) require the occurrence of a ≥C4 class flare to issue an SEP event prediction. Other EM-based SEP event prediction approaches (e.g., Kahler et al., 2007; Laurenza et al., 2009) require the occurrence of a larger flare (≥M2) to issue a prediction. Currently the prediction of SEP events associated with <C4 class flares has been a challenge because of the very low probability of these flares of being associated with SEP events.

Note that in three of the 26 SEPs in Table 1 associated with front-side flares, the SXR class of the associated flare is lower than C4, the WCP-sxr’s f threshold (i.e., the minimum solar EM flux of the associated flares, as explained in Sects. 3.1 and 3.2); therefore WCP-sxr is not able to predict them. In contrast, these three SEPs were predicted by WCP-euv304 and WCP-euv171 and two of them were predicted by WCP-euv94.

To illustrate this finding, Figure 3 presents the predictions of WCP-euv304 (Fig. 3a) and WCP-sxr (Fig. 3b) for the event that took place at 5:05 UT on September 30, 2013. The class of the associated flare was C1, which is too small for the WCP-sxr model to trigger a prediction. The WCP-euv304 model successfully triggered the SEP event prediction at 2:35 (i.e., 2 h 30 min before the SEP event start time) because the EUV intensity of the associated flare was higher than the f threshold of this model.

Fig. 3 This figure presents the predictions of (a) the WCP-euv304 and (b) the WCP-sxr (v1.3) models for the event that took place at 5:05 UT on September 30, 2013. Note that both models correctly detected a magnetic connection. Unlike the WCP-euv304 model, WCP-sxr missed the event because the condition regarding the threshold f (i.e., the minimum EM flux necessary to trigger a prediction) was not met. The SXR peak flux of the associated flare was 1 × 10−6 W m−2, which was not higher than the f threshold of the WCP-sxr model, which is 4 × 10−6 W m−2.

4.3.2 Forecasting SEP events associated with behind-the-west-limb flares

Figure 4 shows a summary of the POD as a function of the heliolongitude in three ranges: E20–W30, W30–W90 and W90–W140. Regarding the prediction capability of the UMASEP/WCP-scheme for predicting SEP events associated with behind-the-limb flares (i.e., those that took place in the range W90–W140), the PODs of all studied WCP models are notably lower than those in the rest of ranges because the observed SXR and EUV intensities decrease as the structure of the associated flare is partially or totally occulted by the west limb.

Fig. 4 Distribution of POD of the WCP scheme using AIA EUV and GOES SXR data for the period 2010–2017.

Table 1 shows that during the analyzed period (2010–2017), 32 prompt SEP events were associated with flares in the range E15–W135, among which five events (i.e., 15.6%) took place behind the limb.

When the flare structure is partially or totally occulted, SXR sensors detect an attenuated brightening (If any). Figure 4 shows that SXRs were useful for predicting one of five events in the range W90–W140. Therefore, the POD of WCP-sxr for SEPs associated with W90–W140 flares was 20% (1/5). For the event occurring on June 18, 2015, which took place at W91, the flare structure was partially occulted; the flare class (M1.2) was more than enough for all studied EUV-based and SXR-based WCP models to predict the SEP event (see Table 6) with a warning time of 305–340 min.

When the flare structure is totally occulted, current solar SXR-based prediction approaches cannot predict the associated SEP events; however, depending on the EUV wavelength, some faint brightening may be observed and used by the WCP-euv approaches to correlate them with the arrival of 9–500 MeV proton enhancements near-Earth. As an example of this, Figure 5 shows the predictions using SDO EUV 94 Å and GOES SXR data of the SEP event that took place at 5:50 UT on October 29, 2015. There is no reference about the location of the associated behind-the-limb flare. Augusto et al. (2016) and Miteva et al. (2018) reported that at 2:24 UT, 2:36 UT, respectively, the associated CME was observed in the coronagraph imagery LASCO C2 instrument in the south-west sector of the Sun. The middle time series of Figure 5a presents the 5-min EUV 94 Å flux, which shows a faint AIA flare at ~1:30 UT (i.e., ~1 h before the CME observation). The bottom time series shows that a magnetic connection was detected (i.e., a correlation with the GOES′ 9–500 MeV protons near-Earth). The prediction was issued at 3:35, therefore the anticipation of this prediction (i.e., warning time) was 2 h 15 min. Note that the GOES SXR data (see the middle time series of Fig. 5b) did not detect any magnetic connection, and for this reason, the WCP-sxr missed this event.

Fig. 5 Graphical output of WCP-euv94 and WCP-sxr (v1.3) for the SEP event on October 29, 2015, which was associated with a behind-the-limb flare: (a) Prediction output using EUV 94 Å data. (b) Prediction using SXR data.

5 Study of the convenience of using AIA EUV data in the UMASEP-10 tool

The goal of the current UMASEP-10 tool is the prediction of all >10 MeV SEP events. The prompt events are the target of the component WCP-sxr (Núñez, 2011), and the non-prompt events are the target of the component PCP model (Núñez, 2011). The non-prompt SEP events are poorly connected SEP events. The PCP model does not analyze solar EM data; it makes its predictions by analyzing the very gradual rises of proton fluxes of these poorly-connected events by using a model that was constructed by using a supervised machine learning technique by an algorithm that learns from a set of labeled data. In our case, the labels were “SEP event” and “no SEP event”. After processing the data, the algorithm automatically discovers temporal patterns in the data (if any) and creates a model that may determine which label should be given to new data based on the discovered patterns. A learning model is able to process new data in order to provide a label. In our case, the algorithm constructed an ensemble of regression trees (Quinlan, 1992; Wang & Witten, 1997; Fidalgo-Merino & Núñez, 2011). Each regression tree was trained from the differential proton fluxes that took place in the beginning phases of past >10 MeV integral proton enhancements from solar cycles 22 and 23. Although the PCP model was trained with gradual proton enhancements of tens of hours, it may trigger predictions with less gradual proton enhancements, some of them are false alarms, and some are SEP events with proton enhancements of a few hours. Table 5 presents the forecasting results of the PCP model for all >10 MeV SEPs that took place during the period 2010–2017, which are 32 prompt and 10 non-prompt events. Column 1 lists the SEP event start date and time. Column 2 lists the event type (prompt or non-prompt). Columns 3 and 4 present the forecasting results of the PCP and WCP-euv94 models, respectively; and, column 5 highlights which SEP events are predicted by both WCP-euv94 and PCP models and which by none.

In this section, we analyze the possibility of using the AIA/EUV-based WCP models in the UMASEP-10 tool, which includes the aforementioned PCP model. Before presenting this analysis, we present the forecasting results (in terms of hits, misses and false alarms) of the PCP model for the period 2010–2017.

Since the PCP model analyzes very gradual rises, it is able to issue predictions with several hours of anticipation. Figure 6 presents several predictions of the PCP model. Figure 6a presents a successful prediction of the poorly-connected event that took place at 3:50, on June 26, 2015. Note that the PCP prediction was issued 13 h 50 min before the occurrence of the SEP event; that is, at 14:00 on June 25, 2015. The drawback of this approach is that it may lead to a large FAR (44.4% for the period 2010–2017, see Tables 9 and 10) compared with that of the WCP models. Figure 6b presents two false alarms issued during March 15th and 16th, 2015, a period of high solar activity; during this period, also the WCP-euv94 model issued two false alarms (See Table 7). Table 9 lists the false alarms issued by the PCP model for the period 2010–2017. Column 1 lists the date and time at which the false alarm is issued and column 2 lists the observed proton flux after the prediction.

Fig. 6 Examples of predictions made by the PCP model. (a) Presents a successful prediction of the SEP event that took place at 3:50 on June 26, 2015. The warning time was 13 h 50 min. (b) Shows two false alarms issued during March 15 and 16, 2015, a period of high solar activity, as shown in the EUV time series (bottom panel).

In the rest of this section, we study the convenience of using the AIA/EUV-based WCP models in the UMASEP-10 tool (which includes the PCP model). Table 10 presents the summary of the forecasting results of each EUV-based model jointly with the PCP model in terms of POD_all (i.e., the POD evaluated over all >10 MeV SEP events in the NOAA list), as well as the rest of forecasting performance metrics (i.e., FAR_all, AWT_all, MWT_all, and CSI_all) obtained for the period from May 2010 to December 2017. Table 10 also shows the results of the UMASEP-10 tool (v1.3), which was re-calibrated using data up to October 2014, so their results with calibration and out-of-sample can be compared with the EUV-based models shown in this study.

Regarding Table 10, it is important to emphasize that the POD_all is the fraction of the total number of SWPC SEP events (i.e., all >10 MeV SEP events in the SWPC SEP list) for the period from May 2010 to December 2017, that were successfully predicted. The FAR_all is the fraction of the total number of predictions issued by a tool, which were not successful. As we mentioned in Section 3.2, the POD_all is calculated as Hits/(Hits + Misses) and FAR_all = falseAlarms /(falseAlarms + Hits). The term Hits is the number of SWPC SEP events successfully predicted. The term Misses is the number of >10 MeV SEP events that were not predicted. The term falseAlarms is the number of predictions that were not successful. Note that the denominator of the POD formula (i.e., Hits + Misses) is the total number of all SWPC SEP events for the period 2010–2017, and the denominator of the FAR formula (i.e., falseAlarms + Misses) is the total number of predictions issued by the tool from a continuous data stream for the period 2010–2017. Also note that these metrics do not count the number of intermediate events that are analyzed to make predictions, such as the number of flares. For more information about how FAR_all and POD_all are calculated for assessing >10 MeV SEP event predictors, please consult Balch (2008), Laurenza et al. (2009) and Núñez et al. (2018).

Table 10 shows that the FAR_all of the UMASEP-10’s PCP model is high (44.4%) and the POD_all is very low (35.7%); however, the POD_non-prompt is 100% (10/10) for the analyzed period, making the PCP model a very good complement of all the WCP models, which is the main reason why the ensemble WCP + PCP has satisfactory results in terms of the POD_all in all the UMASEP-based tools presented in Table 10. Note that the best overall results using EUVs in the range 50–340 Å were obtained by the UMASEP-10euv94 and UMASEP-10euv304 tools, with a CSI_all of 67.2% and 65.0%, respectively, which are better than the CSI_all of the current tool (60.7%). The MWT_all provided by UMASEP-10euv94 and UMASEP-10euv304, were 69 min and 68 min, respectively, which were worse than the MWT_all provided by current tool (82 min). This study shows that the replacement of the WCP-sxr model by either UMASEP-10euv94 or UMASEP-10euv304, would provide a notably better POD_all, a similar FAR_all, but worse MWT_all.

In search for a better combination of models, instead of replacing the SXR-based model, we also tested the addition of either WCP-euv94 or WCP-euv304 to the UMASEP-10 tool; that is, we tested two combined tools (see the last two columns of Table 10): UMASEP-10sxr94, which uses the WCP-euv94, WCP-sxr and PCP models; and, UMASEP-10sxr304, which uses the WCP-euv304, WCP-sxr and PCP models. Each combined tool issues a forecast when any of the individual models emits a forecast. The best resulting tool was UMASEP-10sxr94, which obtained a FAR_all and MWT_all that are very similar to those of the current tool (31.6% and 85 min, respectively), and a notably better POD_all of 92.9% compared with 81% of the current tool. Note that the MWT of UMASEP-10sxr94 is higher (better) than that of UMASEP-10euv94. The UMASEP-10sxr94 tool has an additional advantage: In real-time operations, the data availability is a very important factor to provide real-time services; we know that the availability of GOES data is very high (there are at least two GOES satellites that provide SXR data). So, having the GOES SXR data is a warranty of a continuous prediction service. From all the above, we conclude the simultaneous use of GOES SXR and SDO AIA EUV 94 Å data would improve the performance of the current tool, and continue providing a robust solution for SEP event forecasting. For this reason, we plan to provide an UMASEP-10 Version 2.0 composed of the WCP-euv94, WCP-sxr and the PCP models for making real-time predictions of >10 MeV SEP events.

6 Conclusions

In this study, we explore the use of the UMASEP/WCP scheme by correlating 5-min EUV time history from GOES EUV and SDO AIA data with GOES proton data for predicting well-connected >10 MeV SEP events. The time history from SDO AIA images was obtained from 5-min averages over 12-s EUV intensity means. This study shows the forecasting results in terms of POD, FAR, AWT and CSI (which combines POD and FAR and provides an overall measure of performance), and presents, for the first time, a quantitative assessment of the use of EUV data in the prediction of well-connected SEP events. The WCP-euv models were calibrated using GOES and SDO AIA data from May 2010 to October 2014, and were validated using out-of-sample SDO AIA data from November 2014 to December 2017.

Regarding the performance with calibration data, the best models in terms of CSI are those that used EUV data in the range 50–340 Å (i.e., WCP-euvA, WCP-euvB, WCP-euv94, WCP-euv171 and WCP-euv304), which yielded a CSI_prompt in the range 67.9%–78.6%, compared with a CSI_prompt of 63% of the SXR-based model (v1.3) model. Regarding the performance with out-of-sample SDO AIA data, WCP-euv94, WCP-euv171 and WCP-sxr obtained a similar performance with a CSI_prompt in the range 71.4%–75%. In general, we may conclude that that the use of EUV data in the range 50–340 Å in the UMASEP scheme yields higher (better) or similar POD_prompt compared to the SXR-based model, and similar or higher (worse) FAR_prompt than the SXR-based model. The best overall results using both datasets (see Tables 2, 3, 6, and 7) were obtained by the WCP-euv94 model. These conclusions are consistent with the forecasting results using calibration and out-of-sample data.

As shown in Table 1, during the period from May 2010 to December 2017, 25% (8/32) of the prompt SEP events were associated with either front-side <C4 class flares or behind-the-limb flares. These events, although less hazardous (having generally low peak fluxes), are difficult to predict from current forecasting approaches. This study found that the main reason for the high number of hits using both calibration and out-of-sample EUV data in the range 50–340 Å, is due to the prediction of these events.

• Regarding those events associated with <C4 flares that took place in the front side, in 3 of the 26 SEPs associated with front-side flares in Table 1, the associated flare was too faint for the WCP-sxr model to issue a prediction (i.e., its f threshold, C4, was higher than the SXR peak flux of the associated flare); therefore, this model missed the aforementioned three events; however, the EUV intensity of these events was higher than the corresponding f threshold, with the following results: WCP-euv304 and WCP-euv171 successfully predicted the aforementioned three SEPs, and WCP-euv94 predicted two of them.

• Regarding those SEP events associated with flares that took place behind the west limb, the POD SEP events is lower because the observed SXR and EUV intensities decrease as the flare structure is partially or totally occulted by the west limb; however, the main difference is that when the flare structure is totally occulted, SXR data do not register any activity. Four of five behind-the-limb SEP flares took place in the range W122–W135 during the period 2010–2017; however, of these four far-behind-the-limb SEP flares, two could be recognized in EUVs near-Earth. Of the two SEP events associated with these faint, far-behind-the-limb EUV flares, both could be predicted by WCP-euv304 and WCP-euv171, and one could be predicted by WCP-euv94 (see Fig. 5).

It is important to mention that the WCP-euv models cannot predict the non-prompt SEP events, which amount to 26% of all >10 MeV events that occurred in the period 2010–2017. The non-prompt SEP events are poorly connected SEP events. The set of non-prompt events are the prediction target of the PCP model (Núñez, 2011) in current UMASEP-10 tool.

In order to use EUV data in the current UMASEP-10 tool, we considered two possibilities: the replacement of the current SXR-based WCP model by one of the EUV-based models, and the addition of an EUV-based WCP model to the current tool. We conclude that a combined tool composed of WCP-euv94 and the models of the current UMASEP-10 tool (i.e., the simultaneous use of the WCP-euv94, WCP-sxr and PCP models) obtains a FAR_all and MWT_all that are very similar to those of the current tool (31.6% and 85 min, respectively), and a notably better POD_all of 92.9% compared with 81% of the current tool. Taking into account the high availability of SXR data in the GOES network (which offers a primary and a secondary satellite), we conclude that the simultaneous use of SDO/AIA 94 Å EUV and GOES SXR data would provide a very robust and reliable solution for predicting >10 MeV SEP events for Earth.

Regarding space missions, EUV instruments are equipped in more spacecraft and locations than SXR instruments; therefore, approaches such as the one presented in this study would increment the number of locations where well-connected SEP event predictions may be made in the future (e.g., STEREO, L4, L5), which would allow us to use other higher-level SEP forecasting models (e.g., physics-based models) of the radiation environment surrounding the Earth. Regarding interplanetary missions, Mars is further away from the Sun than Earth, therefore the magnetic connection as viewed from a spacecraft or station at Mars would be even closer to the west limb than for observers at Earth. This would move a larger fraction of well-connected SEP events behind the solar limb (even further than ~W140). This study shows that an EUV-based approach (e.g., UMASEP-10euv94) could be able to predict an important fraction of all well-connected SEP events at Mars. Therefore, future spacecraft and planetary stations could carry devices with current technology in EUV and proton sensor instrumentation, and autonomous SEP event prediction software, that could provide early warnings to astronauts against well-connected events, improving mitigation of their adverse effects.

Acknowledgments

The presented UMASEP/WCP-euv models were funded by the Plan Propio de Investigación of Universidad de Málaga/Campus de Excelencia Internacional Andalucía Tech. The GOES EUV and proton data were taken from the NOAA’s National Centers for Environmental Information (http://satdat.ngdc.noaa.gov/sem/goes/data). The SDO AIA data were taken from the Stanford University’s Joint Science Operations Center (http://aia.lmsal.com/). The authors thank these centers for providing the data used for the calibration and validation of the UMASEP/WCP-euv models presented in this paper. The authors are also grateful with Luiz. F. G. dos Santos (The Catholic University of America) for the preparation of the 5-min AIA EUV time history data. The authors also acknowledge detailed and helpful comments by the referees. The editor thanks Piers Jiggens and an anonymous referee for their assistance in evaluating this paper.

References

All Tables

All Figures

Fig. 1 This figure shows two bit-based series. The “1s” of the top time series shows extreme values from 5-min averaged EUV-based data. The “1s” of the bottom time series is extreme time derivatives from a 5-min differential proton flux. This figure also shows the identification of three “cause-consequence” pairs. A pair, shown in red, is composed of a “1” at time (i) of the top time series and a “1” at time (j) in the bottom time series, and the corresponding pair separation.

In the text

Fig. 2 Outputs of WCP-euv304 and WCP-sxr models after processing data at 6:35 UT on August 4, 2011. (a) shows the prediction of WCP-euv304 at 4:40 UT, (b) the subsequent evolution of the >10 MeV integral proton flux after WCP-euv304 prediction, (c) the prediction of WCP-sxr, which was issued at the same time, and (d) the subsequent evolution of the >10 MeV integral flux after the WCP-sxr prediction.

In the text

Fig. 3 This figure presents the predictions of (a) the WCP-euv304 and (b) the WCP-sxr (v1.3) models for the event that took place at 5:05 UT on September 30, 2013. Note that both models correctly detected a magnetic connection. Unlike the WCP-euv304 model, WCP-sxr missed the event because the condition regarding the threshold f (i.e., the minimum EM flux necessary to trigger a prediction) was not met. The SXR peak flux of the associated flare was 1 × 10−6 W m−2, which was not higher than the f threshold of the WCP-sxr model, which is 4 × 10−6 W m−2.

In the text

	Fig. 4 Distribution of POD of the WCP scheme using AIA EUV and GOES SXR data for the period 2010–2017.
In the text

	Fig. 5 Graphical output of WCP-euv94 and WCP-sxr (v1.3) for the SEP event on October 29, 2015, which was associated with a behind-the-limb flare: (a) Prediction output using EUV 94 Å data. (b) Prediction using SXR data.
In the text

	Fig. 6 Examples of predictions made by the PCP model. (a) Presents a successful prediction of the SEP event that took place at 3:50 on June 26, 2015. The warning time was 13 h 50 min. (b) Shows two false alarms issued during March 15 and 16, 2015, a period of high solar activity, as shown in the EUV time series (bottom panel).
In the text