Predicting well-connected SEP events from observations of solar soft X-rays and near-relativistic electrons

This paper studies the use of electron data from the Electron Proton Alpha Monitor (EPAM) on board the Advanced Composition Explorer (ACE) in the UMASEP (University of Málaga Solar particle Event Predictor) scheme [Núñez, Space Weather 9 (2011) S07003; Núñez, Space Weather 13 (2015)] for predicting well-connected >10 MeV Solar Energetic Proton (SEP) events. In this study, the identification of magnetic connection to a solar particle source is done by correlating Geostationary Operational Environmental Satellites (GOES) Soft X-Ray (SXR) fluxes with ACE EPAM electrons fluxes with energies of 0.175–0.375 MeV. The forecasting performance of this model, called Well-Connected Prediction with electrons (WCP-electrons), was evaluated for a 16-year period from November 2001 to October 2017. This performance is compared with that of the component of current real-time tool UMASEP10, called here WCP-protons model, which predicts the same type of events by correlating GOES SXR with differential proton fluxes with energies of 9–500 MeV. For the aforementioned period, the WCPelectrons model obtained a Probability of Detection (POD) of 50.0%, a False Alarm Ratio (FAR) of 39% and an Average Warning Time (AWT) of 1 h 44 min. The WCP-protons model obtained a POD of 78.0%, a FAR of 22% and an AWT of 1 h 3 min. These results show that the use of ACE EPAM electron data in the UMASEP scheme obtained a better anticipation time (additional 41 min on average) but a lower performance in terms of POD and FAR. We also analyzed the use of a combined model, composed of WCP-electrons and WCP-protons, working in parallel (i.e. the combined model issues a forecast when any of the individual models emits a forecast). The combined model obtained the best POD (84%), and a FAR and AWT (34.4% and 1 h 34 min, respectively) which is in between those of the individual models.


Introduction
Solar energetic particles are accelerated in relationship with eruptive events in the solar corona. Accelerated protons and ions propagate along the interplanetary magnetic field lines, and may reach the near-Earth environment. These energetic particles can cause Single Event Upsets (SEUs), interference with CCD imaging devices, and space radiation for space missions (Shea & Smart 2012). At Earth, they may irradiate passengers and flight crews in aircraft flying at polar latitudes (Lewis et al. 2001;Beck et al. 2005;Mertens et al. 2010).
Given prompt and accurate Solar Energetic Proton (SEP) event predictions, pilots are able to re-route their planes at lower altitude and latitude in order to be better protected by the geomagnetic field and atmosphere (Lewis et al., 2001;Mertens et al., 2010), and launch operators to postpone or re-schedule a rocket launch (Shea & Smart, 2012). Most empirical SEP event forecasters rely on early observations of solar eruptive signatures (Kahler et al., 2007;Balch, 2008;Laurenza et al., 2009;Papaioannou et al., 2016). They have to contend with the fact that strong Soft X-Ray (SXR) bursts located in the western solar hemisphere may not be followed by SEPs, which may increase the number of false alarms. Other empirical SEP forecasters have been proposed to analyze in situ observations of incoming particles solely (Posner, 2007) or both types of data, solar data and in situ particle data (Núñez, 2011(Núñez, , 2015Núñez et al., 2017).
Near-relativistic electrons (1 MeV electrons have 95% of the speed of light) travel faster than slower ions (30 MeV protons have 25% of the speed of light) (Malandraki & Crosby 2018). REleASE (Relativistic Electron Alert System for Exploration) (Posner 2007) uses this effect to predict the 30-50 MeV proton flux by utilizing actual electron fluxes with energies of 0.25-1.0 MeV from the Electron Proton Helium Instrument (EPHIN) on board the Solar and Heliospheric Observatory (SOHO). Since EPHIN data has a limited time coverage in real-time data, the REleASE scheme was used in the HESPERIA project (Malandraki & Crosby 2018;Núñez et al. 2018) with real-time electron intensities in the energy range 0.175-0.315 MeV from the Electron Proton Alpha Monitor (EPAM) on board the Advanced Composition Explorer (ACE).
By analyzing solar SXR data and in-situ proton data, the UMASEP scheme (Núñez 2011(Núñez , 2015 was proposed to predict >10 MeV and >100 MeV SEP events. The evaluation of this scheme has shown good forecasting performance in terms of Probability of Detection (POD) (in the range of 81%-88%) and False Alarm Ratio (FAR) (the range of 23%-34%); however the Average Warning Time (AWT) was not as good as those obtained with the use of electron data by the REleASE scheme. The UMASEP scheme has been used to develop several real-time prediction tools: UMASEP-10 (Núñez 2011), UMASEP-100 (Núñez 2015) and HESPERIA UMASEP-500 (Núñez et al. , 2018 which predicts >10 MeV, >100 MeV and >500 MeV SEP events, respectively. Since 2010, the NASA's integrated Space Weather Analysis (iSWA) system redistributes UMASEP-10's forecasts, which have obtained successful results on an operational level (Tsagouri et al. 2013). The UMASEP-10 tool was also included as a real-time forecasting module in the European Space Agency's SEPs-FLAREs system (García-Rigo et al. 2016).
This paper studies the use of ACE EPAM electron data in the UMASEP scheme, with the purpose of augmenting the AWT for predicting >10 MeV SEP events. The outline of this paper is as follows: Section 2 summarizes the UMASEP's wellconnected prediction (WCP) scheme. Section 3 presents the prediction performance results of the WCP model that uses ACE EPAM electrons, called here the WCP-electrons model, compared with the current UMASEP-10's WCP model, abbreviated here as WCP-protons, for the period from November 2001 to October 2017. The conclusions are presented in Section 4.

Model for predicting well-connected SEP events
The UMASEP's WCP scheme tries to identify precursors of well-connected events by empirically estimating the magnetic connectivity from the associated coronal mass ejection (CME)/flare process zone to the near-Earth environment and identifying a great flare temporally associated with the phenomenon, independently of the flare's heliolongitude. In order to estimate the magnetic connectivity, the WCP scheme makes a lag-correlation of bit-based transformations of solar electromagnetic (EM) data and the first derivatives of differential particle fluxes observed near-Earth. If the solar EM peak flux of the associated flare is greater than a certain threshold, an SEP event prediction is triggered. Regarding solar EM data, the UMASEP's WCP scheme has been used with SXR flux (Núñez 2011(Núñez , 2015Núñez et al. 2017Núñez et al. , 2018, and microwave (MW) flux density at 5 and 9 GHz (Zucca et al. 2017). Regarding in situ particle data, the WCP scheme has been used with proton data only; however, in this paper we evaluate for the first time the use of electron data in this scheme.
Each tool uses its own WCP model, which is based on the WCP scheme. Currently, all UMASEP-based real-time tools use the last version of the WCP scheme (Núñez 2015;Núñez et al. 2017), which is summarized in this section. It is important to say that, although the first versions of the UMASEP-10 tool (Núñez 2011) used a continuous-based lag-correlation, the current version of this tool (Vers. 1.5) also uses the bit-based lagcorrelation.
The WCP scheme works as follows: Firstly, it generates a bit-based time series from the solar EM data and several bitbased time series from the time derivatives of each of the differential in situ particle data. The solar EM-based ''1s'' should be related with signatures of particle acceleration (e.g. if SXR data are used, the ''1s'' are the occurrences of sufficiently large time derivatives of SXR flux; if microwave data are used, the ''1s'' are the occurrences of sufficiently large flux density values). A sufficiently large value is the one that surpasses a percentage p of the maximum value in the present sequence of size L, beyond which no influence is assumed in the SEP event to be predicted; otherwise, the flux level is transformed into a ''0''. To avoid false alarms due to relatively strong fluctuations during periods of low solar activity, a threshold d is necessary as a minimum solar EM flux, which is the minimum value needed to consider it a positive fluctuation (i.e. a ''1''). Figure 1 illustrates the UMASEP/WCP scheme to correlate a solar EMbased time series and a particle flux-based time series. This forecasting approach creates a list of cause-consequence pairs as follows: it takes the first ''1'' of the solar EM-based time series, and the first ''1'' of the particle-based time series, to create a pair; it then takes the second pair of ''1s'' in each time series, and thus successively, until all the ''1s'' of the solar EM-based time series are inspected. Note that i, the first ''1'' (also called fluctuation) in the solar EM time series is paired to j in the particle-based time series. For more information, see Núñez (2015) and Núñez et al. (2017).
An ideal magnetic connection is detected by the WCP scheme when a sequence of solar EM-based ''1s'' in a row is followed by a sequence of particle-based ''1s'' in a row. We say that this ideal magnetic connection would have a Fluctuation Correlation of 1. In general, we need a formula that calculates the Fluctuation Correlation between the bit-valued solar EM-based time series and a particle-based time series. The Fluctuation Correlation formula used by the WCP scheme is explained in detail in Núñez (2015) and Núñez et al. (2017). Finally, an SEP event prediction is triggered when the fluctuation-correlation is greater than a threshold r, and the solar EM intensity of the associated flare is greater than a threshold f (e.g. if SXR data are used, the units of f are W m À2 ).
Based on the WCP scheme explained above, we constructed a model, called WCP-electrons, which correlates GOES SXR fluxes with ACE EPAM electrons fluxes with energies of 0.175-0.375 MeV with purpose of predicting wellconnected >10 MeV SEP events. Note that the WCP-electrons uses SXR flux time derivatives for identifying signatures of particle acceleration and SXR flux (without derivatives) for setting the minimum solar EM intensity to trigger a prediction (i.e. the f threshold), which is a different purpose. It is important to mention that SXR flux does not allow us to directly identify the times of particle acceleration, which is necessary for estimating the solar-EM-based ''1s''; however, according to Neupert (1968), the time derivatives of SXRs show an intensity-time profile that is similar to that in MWs or HXRs in most flares, which are signatures of particle acceleration (mainly during the impulsive phase). The UMASEP-10 tool has a component, called here WCP-protons model, which use the WCP scheme to predict the same type of events by correlating GOES SXR with differential proton fluxes with energies of 9-500 MeV. The cadence of input data of WCP-protons and WCP-electrons is 5 min. Thus, the differences between the WCP models are the specific input data used, and, consequently, the set of the WCP model's calibration values. Each WCP model calibration is done as an optimization process whose purpose was to obtain a high POD and Advance Warning Time (AWT), and a low FAR. The purpose was to obtain a set of thresholds that maximizes the POD and minimizes the FAR. To find a highly effective configuration of parameters (not necessarily the best one), we first searched two optimal threshold configurations using low-resolution steps. For the best two configuration sets found, we applied a new search by using higher resolution steps in the neighbourhoods of the solutions found in the previous step. We repeated the process until the highest general forecasting performance was reached over the studied time interval from November 2001 to October 2017. As a result of each model calibration, we obtain a set of calibration values. Since the target event of WCP-electrons is the same as WCP-protons, we empirically found the same values of L (7 h) and p (91%) for both models; that is, after trying different values of L and p using the aforementioned searching approach, we found that the size of L, beyond which no influence is assumed, was 7 h; and that the percentage p of the maximum value of the time derivative in the present L-size sequence, was 91%.
Since the particle data of both models are different, some thresholds are different. Regarding the threshold f of the WCPprotons, it is 4 · 10 À6 W m À2 (i.e. the minimum SXR class of the associated flare for the WCP-proton model to trigger an SEP event prediction, is C4). During the calibration process of WCP-electrons, we found that the ACE EPAM electron data makes the UMASEP-scheme trigger many more false alarms than the GOES proton data; that is, if we set the same value to the threshold f of the WCP-electrons model, it would have obtained a POD similar to that obtained by WCP-protons, but it would trigger four times more false alarms. We speculate that one of the reasons of the high number of false alarms are the wide-spread solar electron events originated in solar eruptions (even far beyond the limbs), as those reported by Dresing et al. (2014). For this reason, we had to increase the value of WCP-electrons model up to 2.6 · 10 À5 W m À2 to reduce the number of false alarms to operational levels (e.g. a FAR below 40%). That is, the minimum SXR class of the associated flare for the WCP-electrons model to trigger an SEP event prediction, is M2.6). The forecasting results of the final WCP-electrons and WCP-protons models are shown in the next section.

Results
In this section, we present the forecasting results of the WCP-electrons for predicting well-connected >10 MeV SEP events from the correlation of GOES SXR and ACE EPAM electron fluxes for a 16-year period, from November 2001 to October 2017. In this section, we also compare these results with the predictions of the WCP-protons model that is part of the current version of the UMASEP-10 tool (Vers. 1.5). Figures 2 and 3 show the forecast graphical outputs that an operator would have seen if the WCP-electrons and WCP-protons models, respectively, had processed real-time data for the event that took place at 10:55 UTC on April 11, 2013.    Figure 3a shows the prediction of WCP-protons at 9:30, and Figure 3b the subsequent evolution of the >10 MeV integral proton flux after WCP-protons prediction. The yellow/orange band in the proton intensity plots gives the predicted range. Figures 2a and b show the WCP-electrons predictions before and after the SEP event, respectively. Figures 3a and b show the WCP-protons predictions before and after the same event. The upper time series of these figures show the observed integral proton flux with energies greater than 10 MeV. The current flux is indicated below the label ''now'' at each image. To the right of this label, the forecast integral proton flux is presented. The yellow/orange-coloured band indicates the expected evolution of the integral proton flux derived from the prediction of the proton flux. Note that Figure 2a shows the WCP-electrons prediction at 8:15 (April 11, 2013) when no enhancement of the >10 MeV integral proton flux had taken place. Figure 3a shows the WCP-protons prediction at 9:30 for the same event (i.e. 1 h 15 min after the WCP-electrons prediction), when the enhancement of the >10 MeV integral proton flux had already taken place. For this event, the WCP-electrons issued the prediction 2 h 40 min before the SEP event start time. The horizontal orange line at the top right part of Figures 2b  and 3b, means that during that period an SEP event was occurring. The central curve in each panel displays the SXR flux, and the lower time series shows the magnetic connectivity estimation with the best-connected CME/flare process zone. When a forecast is issued, the graphical output also shows the inferences about the associated flare, heliolongitude and active region.
In order to assess the performance of the WCP-electrons model, it was run from November 2010 to October 2017. During this period, in 91 SEP events the Space Weather Prediction Center (SWPC) threshold was surpassed (i.e. J (E > 10 MeV) > 10 pfu); among them 50 SEP events were prompt events. As in Núñez (2011), we say that a prompt >10 MeV SEP event is that event in which the temporal distance from the flare peak time to the SEP event start time (when the aforementioned threshold was surpassed), is less than 8 h. The lapse of 8 h is the time that results in the same number of prompt and delayed events that have occurred in solar cycles 22 and 23 (Núñez 2011). Table 1 lists the prompt SEP events (all of which are well-connected events), and the corresponding forecast results of the WCP-electrons and WCP-protons models. Column 1 gives the SEP event start times (ST), column 2 the peak time (FT) of the associated flare, column 3 the temporal distance ST-FT, column 4 presents the hardness of the SEP event (hard is set when the event also surpassed the SWPC threshold J (E > 100 MeV) > 1 pfu; otherwise the value of this column is soft), columns 5 and 6 the forecast results (in terms of ''hits'' and ''misses'') and warning time of the WCP-electrons model, and columns 7 and 8 the same results of the WCP-protons model.
Taking into account the results in Table 1, Table 2 presents the forecast performance results in terms of POD, FAR and AWT using the WCP-electrons and WCP-protons models. Probability of Detection is the number of the predicted SEP events divided by that of the observed SEP events that actually occurred, that is 50 prompt events in the considered time interval. The number of successful predictions were 25 and 39, and therefore the POD was 50% (25/50) and 78% (39/50) for the WCP-electrons and WCP-protons models, respectively. The false alarm ratio (FAR) is the number of false predictions over the number of predictions; the number of false predictions were 16 and 13, therefore the FAR was 39% and 22%, for the WCP-electrons and WCP-protons models, respectively. The AWT is the average of temporal distances from the time of a successful predictions to the corresponding SEP event start times. The measured AWT were 1 h 44 min and 1 h 3 min for the WCP-electrons and WCP-protons models, respectively. We also analyzed the use of a combined model, composed of WCP-electrons and WCP-protons working in parallel (i.e. the combined model issues a forecast when any of the individual models emits forecasts). The combined model, which is shown in Table 2 as ''WCP-electrons + WCP-protons'', obtained the best POD (84%), and a FAR and AWT (34.4% and 1 h 34 min, respectively) in between those of the individual models.
From Table 2, we note that the use of ACE EPAM electron data in the UMASEP scheme (i.e. WCP-electrons model) obtains a lower forecasting performance in terms of POD and FAR (i.e. lower POD and higher FAR), but a better performance in terms of AWT (i.e. 41 min higher) than the WCPprotons model, which uses GOES protons measurements. The results of the combined model show that both models may be seen as complementary for SEP event forecasting purposes.
It is important to mention that for both models, prompt SEP events originating behind the solar limb may be detectable depending on the SXR peak; if the SXR flux peak measured by GOES during a behind-the-limb solar event is lower than the f threshold (i.e. the minimum flare peak threshold according to Sect. 2), the associated SEP event is unpredictable by the WCP models; however, if the observed SXR flux peak is higher than the f threshold (i.e. M2.6 for WCP-electrons, or C4 for WCP-protons) and there is an enhancement of the flux of the analyzed particles in the near-Earth, the event may be detectable by the WCP models; for example, the >10 MeV SEP event on May 17, 2012, originated behind the western solar limb, was successfully predicted by the WCP-protons model because the measured SXR flux of the behind-the-limb flare was M5, which was sufficiently higher than the corresponding minimum threshold C4. Figure 4 shows the overall performance of the WCP-electrons and WCP-protons model in terms of the number of hits (correct prompt >10 MeV SEP forecasts) and missed prompt SEPs for the period from November 2001 to October 2017. Note that during the analyzed periods of higher activity (i.e. 2002-2003 and 2011-2012), the WCP-protons model obtained more hits and less misses than the WCP-electrons model.

Forecasting performance comparison depending
on the hardness of the SEP energy spectra Table 3 lists the POD and AWT of the WCP-electrons and WCP-protons models, depending on the hardness of the event listed in column 4 of Table 1. In this paper, an >10 MeV SEP event is hard if the event also surpasses the SWPC threshold J (E > 100 MeV) > 1 pfu, otherwise, the event is soft.
Regarding the prediction of hard prompt events, according to Table 3, the combined model and WCP-protons obtained better results than WCP-electrons in terms of POD (i.e. 95%-100% vs. 70%). Regarding AWT, the combined model and WCP-electrons obtained better results than WCP-protons (i.e. 53.6-59 min vs. 41.3 min). Since the combined model obtains better results in both metrics, while individual models obtained better results in one aspect only, we say that the combined model obtains better results than individual models in the prediction of hard prompt events.
Regarding the prediction of soft prompt events, the performance results are mixed. Although the combined model and WCP-protons are better than WCP-electrons in terms of POD (66.7%-73.3% vs. 36.7%), they are worse in terms of AWT (91-134 min vs. 170 min).
3.2 Study of the convenience of using ACE EPAM electron data in the UMASEP-10 tool In addition to using the WCP-protons model for predicting well-connected SEP prediction events, the current real-time UMASEP-10 tool has also the Poorly-Connected Prediction (PCP) model. Poorly-connected SEP events are characterized by a very gradual enhancement of the integral proton flux and are normally associated to eastern parent solar events (Reames 2004). The PCP model makes its predictions by using GOES proton data only with an ensemble of regression trees constructed from the beginning phases of past >10 MeV integral proton enhancements from solar cycles 22 and 23. For more information about the PCP model, please consult Núñez (2011).
This section will compare the forecasting performance for predicting all SWPC SEP events (i.e. well-and poorly connected events) of the current UMASEP-10 tool with the performance of the best of the studied models, which is the combined model. Table 4 shows the performance results in terms of POD, FAR and AWT of these two tools with data from November 2010 to October 2017. The current UMASEP-10 tool (Vers. 1.5), which includes the WCP-protons model, obtained a POD of 82.4%, a FAR of 23.47%, and an average warning time of 4 h 20 min for the same period. Table 4 shows that the use of the combined model (which uses WCP-electrons model) in the UMASEP-10 tool obtained a POD of 83.5%, a FAR of 29.0% and an AWT of 4 h 51 min. Although the combined use of the WCP-electrons model in the current UMASEP-10 tool seems to improve the AWT of the current tool, the differences between the forecasting results of this combination and those of the current UMASEP-10 tool, shown in Table 4, are not statistically significant.

Conclusions
The use of ACE EPAM electrons in the UMASEP prediction scheme was studied. The identification of magnetic connection to a solar particle source is done by a model, called here WCP-electrons, by correlating GOES SXR fluxes with ACE EPAM electron fluxes. This model makes predictions of the time interval within which the >10 MeV proton flux is expected to surpass 1 pr cm À2 sr À1 s À1 (i.e. the SWPC threshold). The EPAM onboard the ACE satellite provides real-time electron intensities in an energy range of 0.175-0.315 MeV  in a time resolution of 5 min. We evaluate the performance of this model for predicting prompt >10 MeV SWPC SEP events (all of them well-connected events), for a 16-year period from November 2001 to October 2017. We compare these results with those of the WCP-protons model, part of the current real-time UMASEP-10 tool, which predicts well-connected events, by correlating GOES SXR with differential proton fluxes. The key findings are the following: -the POD of the WCP-electrons model is lower (worse) than that of the WCP-protons model (50% vs. 78%). We also analyzed the use of a combined model, composed of WCP-electrons and WCP-protons, working in parallel (i.e. the combined model issues a forecast when any of the individual models emits a forecast). The combined model obtained a POD of 84%, which was the best result of the models individually; -the FAR of the WCP-electrons model is higher (worse) than that of the WCP-protons model (39% vs. 22%). The combined model obtained a FAR of 34.4%, which was in between the FARs of both models individually; -the AWT of the WCP-electrons model is higher (better) than that of the WCP-proton model (1 h 44 min vs. 1 h 3 min). The combined model obtained a AWT of 1 h 34 min, which is in between the AWTs of both models individually, closer to the WCP-electrons model.
Regarding the prediction of hard prompt >10 MeV SEP events (i.e. those events which also surpass the SWPC threshold J (E > 100 MeV) > 1 pfu), the combined model and WCP-protons obtained better results than WCP-electrons in terms of POD. Regarding AWT, the combined model and WCP-electrons obtained better results than WCP-protons. Since the combined model obtains better results in both metrics (POD of 100% and AWT of 59 min), we say that it is better than individual models in the prediction of hard prompt events. all well-and poorly-connected events). b UMASEP-10 comprises WCP-protons + PCP models. PCP stands for Poorly-Connected Prediction model (Núñez 2011). c ''UMASEP-10 + WCP-electrons'' may also be seen as the resulting tool obtained by replacing the current WCP-protons in UMASEP-10 with the combined model, which comprises the WCP-protons and WCP-electrons models.