Analysis on selected geo-effective events using observations and models at Space Environment Prediction Center Siqing Liu, Ercha Aa, Qiuzhen Zhong, Bingxian Luo, Zhitao Li, Jingjing Wang, and Jiancun Gong Space Environment Prediction Center, National Space Science Center, Chinese Academy of Sciences
SEPC (http://eng.sepc.ac.cn) Established in 992 to support space exploration Started to issue space environment prediction in 998 7 days/week & 365 days/year Chinese Academy of Sciences National Space Science Center SEPC
Space Weather Forecasting Services SEPC is now an Associate Warning Center of The International Space Environment Service (ISES). General Products Space Weather Indices Space Weather Events Ap, Kp, F07, Dst, AE/AU/AL Solar Flares, SPE, Geomagnetic Storm, Relativistic Electron Enhance Tailored Products Publishing Ways Manned Space Mission, Lunar Exploration Website, Text Message, Mail, Email, Mobile App
Outline Overview of the Space Environment 24 th Solar Cycle Year 204 Verification of SEPC/NSSC Forecasting Single Storm Event Jun, 204 - Jan, 205
Overview of the 24 th Solar Cycle Sunspot # from st -24 th solar cycle Two Peak profile has shown Entering Decreasing Phase Estimated Total Sunspot #: ~4400 Smoothed sunspot # st Peak 2 nd Peak Feb, 202 Apr, 204 74%
204 Overview: Solar Activity 203 204 No sunspot day 0 Active Region # 302 39(+6%) Average Sunspot # 97 2(+25%) Mean F0.7 23 46(+9%) C-class Flares 357 798(+32%) M-class Flares 00 208(+08%) X-class Flares 2 6(+33%) SEP Events 7 6(-4%) X. X.6 X3. X2.0 X2.0 X.0 204 Oct-Nov (AR292: 35M- & 6 X-class flares) Active Region with Maximum Area in the 24 th solar cycle
204 Overview: Geomagnetic Activity Kp Values Kp = 4 Kp = 5 Kp = 6 Kp = 7/8 Kp = 9 Geomagnetic Activity level Active Minor Storm Moderate Storm Strong Storm Severe Storm 203 204 Average AP 8 9(+3%) # of Days for AP 5 4 49(+20%) Geomagnetic Active Days 54 70(+0%) Minor/Moderate Storm Days 49 47(-4%) Strong Storm Days (0%) Maximum AP 5 47(-8%) Maximum Kp 7 7(0%)
Outline Overview of the Space Environment 24 th Solar Cycle Year 204 Verification of SEPC/NSSC Forecasting Single Storm Event Jun, 204 - Jan, 205
SEPC Space Weather Forecasting Products (Index/Flux)
Geomagnetic Storm Lists from Jun, 204-Jan, 205 Time (Y/M/D) Storm Level/Duration Causes Description Remarks 204/06/08-06/09 Minor/ 3 hours CME from Filament 204/06/8 Minor/ 3 hours CIR 204/08/9 Moderate/ 3 hours CME from Filament 204/08/27-08/3 Minor/ 6 hours CME + CIR 204/09/3-09/4 Strong/ 3h+ Moderate/ 3h + Minor/ 9h CME from AR258, X.6 flare 204/09/9 Minor/ 3 hours CME from AR257 + following CIR 204/0/4-0/5 Minor/ 9 hours CME from filament 204/0/20-0/22 Minor/ 3 hours CIR 204//0-/2 Minor/ 6 hours CME from AR2205 + following CIR 204/2/06-2/09 Minor/ 6 hours CIR 204/2/2 Minor/ 3 hours CIR Storm Levels Minor: Kp=5 Moderate: Kp=6 Strong: Kp=7/8 Severe: Kp = 9 204/2/2-2/24 Minor/ 6 hours CME from AR2242 + following CIR 204/2/28-2/30 Minor/ 6 hours CIR 205/0/02-0/03 Minor/ 3 hours CME from filament 205/0/04-0/05 Minor/ 9 hours CME 205/0/07-0/08 Strong/3h + Moderate/3h CME
Geomagnetic storms @ Aug 27-3, 204 6-h Storm caused by CMEs from Aug 22 with C6 flare from AR248 or C2/C6 flares from AR246 Following with multiple CIRs
Forecasting Verification Factors Factor Formulation Reflection Mean Error (ME) Mean Absolute Error (MAE) Root Mean Square Error (RMSE) Correlation Coefficient (CC) MSE Skill Score (MSESS or PE) N ME = i O i N i= (F ) N MAE = i O i N i= F N RMSE = i O i N i= CC = N i= (F F)(O O) i i (F ) N N 2 2 (F i F) (O i O) i= i= MSE MSESS = MSE forecast reference 2 Bias Accuracy Accuracy Association Skill
Geomagnetic Storm@ Aug. 27-3, 204 : Ap forecasting results Model Description Ap forecasting for future 27-days using auto-regression analysis model Ap mod Ap art Reflection Better One Mod ME -6.583-2.75 Bias MAE 6.75 4.083 Accuracy RMSE 8.39 5.346 CC 0.76 0.634 Association MSESS -.00 0.86 Skill Evaluation Ap model forecasting results Ap artificial forecasting results What is the forecasting performance Why such Performance Where it can be used and how to use Art Larger bias during storm time Fair accuracy agreement Higher correlation coefficients but lower skills Focus on the general consistence of the 27- days prediction with details being neglected Smaller bias during storm time Better accuracy agreement Lower correlation coefficients & Higher skills More flexible reaction & adjustment of people based on the level and duration of the current storm Where: Space Weather prediction, research and application, and SEPC/NSSC online forecasting tools How: Forecasting results (future 27-days by model and 3 days by artificial) are updated and published on the website & mobile App simultaneously, and can be browsed & downloaded conveniently by users.
Geomagnetic Storm@ Aug. 27-3, 204 : F 0.7 forecasting results Model Description F 0.7 forecasting for future 27-days using auto-regression analysis model with 54-orders F 0.7 mod F 0.7 art Reflection Better One Mod ME -0.583 0.75 Bias MAE 7.583 6.083 Accuracy RMSE 0.57 9.543 CC 0.78 0.748 Association MSESS 0.0 0.94 Skill Evaluation F 0.7 model forecasting results F 0.7 artificial forecasting results What is the forecasting performance Why such Performance Where it can be used and how to use Art Comparable bias between mod and art results Fair accuracy agreement (~7-8% relative errors) Higher correlation coefficients but lower skills -day delay in grasping inflection point Comparable bias between mod and art results Fair accuracy agreement (~7%-8% relative errors) Lower correlation coefficients & moderate skills -day delay in grasping inflection point Generally stable variation trends, which is readily to grab Historical periodical data cannot reflect the sudden appearance/disappearance of active region Where: Space weather prediction, and empirical/theoretical models with F 0.7 as driven factor How: Forecasting results (future 27-days by model and 3 days by artificial) are updated and published on the website & mobile App simultaneously, and can be browsed & downloaded conveniently by users.
Geomagnetic Storm@ Aug. 27-3, 204 : D st forecasting results Model Description -h forecasting of D st index using real time solar wind data from ACE observation based on the influence of S w dynamic pressure to the decay & injection of ring current D st mod Reflection Evaluation ME -9.32 Bias MAE 0.243 RMSE.665 Accuracy 0%-5% CC 0.946 Association Very High MSESS 0.678 Skill High Evaluation What is the forecasting performance Why such Performance Where it can be used and how to use Good accuracy agreement (~0-5% relative errors) High correlation coefficients and skills (close to ) D st model forecasting results The modeled Dst is a sum of three terms that have growth and decay, a dynamic pressure term, an interplanetary magnetic field term, and some offset terms. Where: Space weather prediction,, operational system in SEPC/NSSC, and web-based forecasting tools How: Forecasting results (future hours) are updated and published on the website & mobile App simultaneously, and can be browsed & downloaded conveniently by users.
Geomagnetic Storm@ Aug. 27-3, 204 : Kp forecasting results Model Description -hour in advance forecasting of Kp index using real time solar wind data from ACE observation based on neural network algorithms. Kp mod Reflection Evaluation ME -0.83 Bias MAE 0.897 RMSE.3 Accuracy ~20%-30% CC 0.83 Association High MSESS 0.340 Skill Moderate Evaluation What is the forecasting performance Why such Performance Where it can be used and how to use Kp model forecasting results Commonly Underestimation of Kp index around 20%-30% High correlation coefficients Moderate skill score Relative stable and subtle fluctuation of the geomagnetic field for the 24 th solar cycle, which as been used as the background values of the input of neural network Where: Space weather prediction, geomagnetic storm alert and warning How: Forecasting results (-hour in advance) are updated and published on the website & mobile App simultaneously, and can be browsed & downloaded conveniently by users.
Geomagnetic Storm@ Aug. 27-3, 204 : Relativistic Electron Forecasting Model Description -day in advance forecasting of daily integrated relativistic electron flux > 2Mev at geosynchronous orbit using linear prediction combined with Kalman filter Model Reflection Evaluation ME -0.57 Bias 3% MAE 0.462 RMSE 0.682 Accuracy ~8%-0% CC 0.86 Association High MSESS 0.436 Skill High Evaluation What is the forecasting performance Why such Performance Where it can be used and how to use Electron flux model forecasting results Sometimes lagged by 24 hours in grasping the sudden variation of the values Less than 0% prediction errors High correlation coefficients and skills Restriction of linear prediction in representing sudden variation Where: Space weather prediction, aircraft alert and warning at GEO orbit How: Forecasting results (-day in advance) are updated and published on the website & mobile App simultaneously, and can be browsed & downloaded conveniently by users.
Geomagnetic Storm@ Aug. 27-3, 204 : AE Index Forecasting Model Description -hour in advance forecasting of AE Index with 0 min resolution based on empirical functions by using solar wind data, IMF parameters, and F0.7 index as input Model Reflection Evaluation ME 2.43 Bias %-2% MAE 43.65 RMSE 75.83 Accuracy ~5%-20% CC 0.84 Association High MSESS 0.8 Skill High Evaluation What is the forecasting performance Why such Performance Where it can be used and how to use Electron flux model forecasting results Very good performance in grabbing the trends of the variation and the level of the peak values 5%-20% prediction errors High correlation coefficients and skills Proper settings of the empirical function in establishing the relationship between solar wind and electro-jet in polar region Where: Space weather prediction, or driven factor for other empirical/theoretical models How: Forecasting results (-hour in advance and 0 min resolution) are updated and published on the website & mobile App simultaneously, and can be browsed & downloaded conveniently by users.
Verification of SEPC Forecasting Jun 204 - Jan 205
Verification of SEPC Forecasting: Accuracy How to choose appropriate factor to represent forecasting accuracy? RMSE for index with low base values, and NRMSE for high base values N RMSE = i O i N i= (F ) 2 NRMSE N Fi Oi = ( ) N O i= i 2 Ap Model 6.76 -- Ap Artificial 5.87 -- Kp.9 -- Dst 4.37 -- F0.7 Model 7. 4.89% F0.7 Artificial 7.60 5.22% AE 72.03 32.43% Relativistic Electron 0.47 (after log0) 6.54%
Verification of SEPC Forecasting: Association and Skill Correlation Coefficients 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0. 0 0.283 Ap mod 0.524 Ap art 0.968 0.968 F07 mod F07 art 0.77 0.89 0.593 0.87 Dst e_flux Kp AE Evaluation: Association Relative high correlation coefficients for various parameters except for Ap index model Skill Score (PE).000 0.800 0.600 0.400 0.200 0.000-0.200-0.400 Ap Ap art F07 F07 Dst e_flux Kp AE mod mod art -0.278 0.037 0.940 0.932 0.20 0.803-0.325 0.780 Evaluation: Skill Score Relative high skill score for F0.7, Relativistic electron, and AE index comparing with persistence model.
Summary and Conclusions Overview of the 24 th solar cycle The current 24 th solar cycle has a relative small activity level, and the solar activity has reached the highest point and began to decrease. Overview of 204 The solar activity is stronger than that of the year before. The geomagnetic activity remains almost the same. The average Ap levels and geomagnetic active days are increased, but the storm days are almost the same with the year before. Verification of SEPC forecasting results The forecasting results of Ap, Kp, F 0.7, Dst, Relativistic Electron, and AE index have been verified for selected event and for period of Jun 203-Jan 204 based on the bias, accuracy, association, and skill respectively. The performance of the model as well as the artificial forecast have been evaluated accordingly. The forecasting results can be used for space weather prediction, operational system in SEPC/NSSC, and web-based forecasting tools. The forecasting results are updated and published on the website & mobile App simultaneously, and can be browsed & downloaded conveniently by users.
Thanks for your attention! Space Environment Prediction Center, NSSC/CAS