Recent Advances in Flares Prediction and Validation

Transcription

1 Recent Advances in Flares Prediction and Validation R. Qahwaji 1, O. Ahmed 1, T. Colak 1, P. Higgins 2, P. Gallagher 2 and S. Bloomfield 2 1 University of Bradford, UK 2 Trinity College Dublin, Ireland r.s.r.qahwaji@bradford.ac.uk

2 Organisation Introduction ASAP SMART-ASAP Evaluation Final Words

3 Automated Flares Prediction Real time performance More feasible as the size of images and datasets increases (The three solar instruments of SDO, AIA, HMI, and EVE, produces about 1-2 TB Data/day exceeding the SOHO data volume by more than a thousand time) Consistent objective performance Ability to associate different data sets Ability to extract knowledge from historical data

4 Challenges for computer-based predictions:? How to represent human s? knowledge and experiences using computerised rules? How to accurately predict flares if the flaring mechanism is still not totally understood? How to make computers learn and become better with time?

5 About this Work Recent advances in solar physics are integrated with recent advances in machine learning and feature selection to study the feasibility of developing the nextgeneration of automated flares prediction technology. This new technology could improve our understanding of the flaring mechanism by highlighting the AR properties that are most related to flare occurrences.

6 Organisation Introduction ASAP SMART-ASAP Evaluation Final Words

7 What is ASAP? Automated Solar Activity Prediction (ASAP) is a web-compliant, fully automated computer to predict solar flares in near real-time. Several solar imaging algorithms and 5 Neural Network Agents. Fully implemented in C++. Available publically at

8 ASAP Features ASAP is always downloading latest images from SOHO MDI (GIF images) and SDO HMI It takes around seconds to detect, group and classify sunspots and then to generate flare predictions. It is written in C++ and can be modified to run under different platforms (currently runs under Windows and Mac)

9 ASAP Modules and Timeline Colak & Qahwaji 2009

10 Automated predictions of solar flares are provided daily in our website

11 ASAP is integrated with SWENET COST ES0803 Alcala 2011

12 ASAP is also integrated with the Integrated Space Weather Analysis System of NASA

13

14

15 Colak T and Qahwaji R (2009): "ASAP: A Hybrid Computer Platform Using Machine Learning and Solar Imaging for Automated Prediction of Significant Solar Flares" Space Weather, 2009

16 Comparison of QR measures for 24, 48, and 72 hour time windows from ASAP and SWPC. SWPC results can be accessed from their website

17 Solar Monitor Active Region Tracker (SMART) Magnetic feature (MF) detections were generated using SMART ( Higgins et al., 2010) SMART is a recently developed feature extraction algorithm to detect, characterise, and catalogue MFs using 96 minute SoHO/MDI images. MFs are detected in magnetograms by segmenting quiet-sun and feature pixels using a combination of image processing techniques. SMART detects MFs automatically and is independent from NOAA s ARs. AR records were generated by SMART [Higgins et al. 2010]. To minimise the error caused by projection effects, only MF detections located within 45 from solar disk centre are considered for this work. Flares catalogues were obtained from the National Geographical Data Centre (NGDC) database. Period: from April Dec 2010

18 MF properties generated by SMART No AR Properties Description 1 Type-Polarity Unipolar/Multipolar 2 Type-Size Big/Small 3 Type-Evolution Emerging/Decaying 4 Area_Mmsq Area of the region [Megameters squared]. 5 Bflux_Mx 6 Bfluxp_Mx 7 Bfluxn_Mx 8 Bfluximb 9 DBfluxDt_Mx 10 Bmin_G Minimum B value in the region [Gauss]. 11 Bmax_G Maximum B value in the region [Gauss]. 12 Bmean_G Mean B value in the region [Gauss]. 13 Lnl_Mm Neutral Line Length in the region [Mega meters]. 14 Lsg_Mm High Gradient Neutral Line Length in the region [Mega meters]. 15 MxGrad_GpMm Maximum Gradient along the Neutral Line [Gauss / Megameter]. 16 MeanGrad Mean Gradient along the Neutral Line [Gauss / Megameter]. 17 MednGrad Median Gradient along the Neutral Line [Gauss / Megameter]. 18 Rval_Mx Schrijver R-Value [Maxwells], (Schrijver, 2007). 19 WLsg_GpMm Falconer WLsg value [Gauss / Megameter], (Falconer et al., 2008). 20 R_Str Schrijver R-Value with a lower threshold for summing flux [Maxwells]. 21 WLsg_Str A modified version of WLsg.

19 Generation of Associated Data Sets Algorithms are developed to associate SMART MF detections with flares from the NGDC catalogue for the period April 1996 till December The purpose of this association is to classify MF detections as flaring or non-flaring. For every MF, the association algorithm determines the times of all C-, M-, and X-class flares in the following 24-hour period. Flare locations provided in the NGDC catalogue are remapped back to the time of the MF detection (Colak and Qahwaji, 2010) and the location compared to the MF spatial coverage. MFs are defined as flaring if the remapped flare location falls within the boundary of the SMART MF contour.

20 Two forms of association used: segmented and operational. Different criteria are used for non-flaring MF detection. In the segmented set, a non-flaring MF is not associated with any C-, M-, or X-class flares in a +/-48-hour period around its observation time, while a flaring MF is associated with a flare within 24 hour period after observation. In the operational set, a non-flaring detection is defined as a MF with no C-, M-, or X-class flare associations in the 24-hour period following its observation. Only MFs, which are located within 45 from the solar disk centre are considered.

21 NGDC Flares Data

22 The total number of flare events and MF detections in the input records (pre-association) during Input Data Total Number of MF Detections Total Number of Flares C M X 663,340 4, Total: 4,915 Total number of flaring and non-flaring MF detections in the segmented and operational sets. Segmented Association Output Flaring MF Detections 27,428 (Within +24 hrs.) Non-Flaring MF Detection 432,930 (Within 48 hrs.) Operational Association Output Flaring MF Detections 27,428 (Within +24 hrs.) Non-Flaring MF Detection 491,259 (Within +24 hrs.)

23 Total Flare Events (with Location) SMART AR Catalogue Input 4,915 Total MF Samples 663,340 C 4,301 M 562 X 52 Flare Catalogue AR Properties Flare Details. No. of Flares Samples No. of AR Samples Associating ARs to Flares Associated Data AR Properties Flare/ No-Flare No. of AR Samples Association Output (Segmented Set) Total MF to Flare Associations 27,428 Total MF to No-Flare Associations 432,930 Total Associated Data 460,358

24 Feature Selection The Feature Selection (FS) process aims to study the significance of the input features (MF properties in our case) with respect to the prediction classes (flaring probability here). FS select features that are highly correlated with the class but uncorrelated with each other. FS enhances the efficiency and usability of a data set by removing features that are irrelevant, redundant, and/or noisy FS is widely used with machine learning to enhance data analysis and prediction. Here, the FS process is coupled with machine learning to highlight the AR properties that are more relevant to flaring/no-flaring. No. of AR Samples Associated Data AR Properties Feature Selection MF Properties that are mostly correlated with Flares/No-Flares Flare/ No-Flare

25 Advantages of the FS process 1. Reduce the number of input features by removing features that are irrelevant, redundant, and/or noisy. 2. Enable machine learning or predictors to be cost effective and faster. 3. Could improve the prediction accuracy of machine learning systems. 4. Provide a physical insight onto the importance of input features.

26 Feature Selection COST ES0803 Alcala 2011 Two different FS algorithms applied: Correlation-based Feature Selection (CFS) and Minimum Redundancy Maximum Relevance (MRMR). We consider all possible combinations of both to create 11 different FS processes.. Initially, cross-validation is applied to the segmented data set (consisting of 21 MF properties). 50% of the data is selected randomly and then FS is applied. This is repeated 20. CFS determines the best (but un-ranked) MF properties while MRMR ranks the most significant MF properties. The predictive capabilities of the selected MFs are compared against the full set (21 MF properties) using CCNN.

27 SMART MF properties selected by each feature selection process. COST ES0803 Alcala 2011 Evaluation Output Search Method Data Type Method Type Selected SMART MF Property IDs CFS BestFirst Backward Normalised Subset v7, v9, v13 CFS BestFirst Bidirectional Normalised Subset v7, v9, v13 CFS BestFirst Forward Normalised Subset v7, v9, v13 CFS Greedy Stepwise Backward Normalised Subset v5, v6, v7, v13, v14, v15, v19, v20, v21 CFS Greedy Stepwise Forward Normalised Subset v7, v9, v13 CFS BestFirst Backward Discretised Subset v9, v13, v14, v18, v19, v20 CFS BestFirst Bidirectional Discretised Subset v9, v13, v14, v18, v19, v20 CFS BestFirst Forward Discretised Subset v9, v13, v14, v18, v19, v20 MRMR-MIQ Forward Discretised Weighted v13, v21, v20, v19, v18, v17, v16, v15, v14, v11 MRMR-FCQ Forward Normalised Weighted v13, v4, v14, v15, v21, v7, v6, v20, v11, v19 MRMR-FCD Forward Normalised Weighted v13, v21, v14, v20, v15, v4, v5, v19, v7, v18 SMART MF properties selected by feature selection processes

28 SMART MF properties most relevant to flaring COST ES0803 Alcala 2011 The SMART MF properties most relevant to flaring Property ID SMART Property Description v4 A Area of the region v5 Total unsigned magnetic flux of the region v6 + Total positive flux in the region v7 Total negative flux in the region v11 B MAX Maximum magnetic field in the region v13 L NL Neutral line length in the region v14 L SG High gradient neutral line length in the region v15 MAX Maximum gradient along the neutral line v18 R Schrijver R value v19 WL SG Falconer WL SG value v20 R * Schrijver R value with a lower threshold v21 WL SG * A modified version of WL SG

29 The Evaluation Strategy Segmented Data Set Operational Data Set Evaluation using CCNN and Cross- Validation Evaluation using CCNN and Timedependant data Evaluation using CCNN and Cross- Validation Evaluation using CCNN and Timedependant data Benchmark Metrics

30 Machine Learning Machine learning enables the creation of computerised learning rules. Cascade Correlation Neural Networks (CCNN) [Qahwaji & Colak, 2007] were applied as our major learning technique. No. of AR Samples Associated Data 21 AR Properties Flare/ NoFlare No. of AR Samples Associated Data 8 AR Properties Flare/ NoFlare Machine learning was applied to the full-set of MF properties (21 properties) and the selected MF properties (12 properties), to compare their prediction performance. CCNN ML Evaluate Prediction Performance CCNN ML Evaluate Prediction Performance Training and testing vectors were organized based on the statistical Jack-Knife technique [Fukunaga 1990]. Compare

31 Machine Learning For all the experiments, 80% of the input samples were randomly selected for training, and the remaining 20% were used for testing. Every experiment was repeated 10 times and the average performance was recorded. CCNN s performance is evaluated by comparing the actual target values with the generated values. The metrics proposed in Fawcett (2006) and Balch (2008) were used. The evaluation strategy is applied on the full MF data-sets (segmented and operational) and FS data sets.

32 Benchmark Metrics: Fawcett (2006) & Balch (2008) TPR=TP/(TP+FN); FPR=FP/(FP+TN); TNR=TN/(TN+FP); FNR=FN/(TP+FN); FAR=FP/(FP+TP); ACC=(TP+TN)/((TP+FN)+(FP+TN)); SPC=TN/(FP+TN); PPV=TP/(TP+FP); NPV=TN/(TN+FN); (Negative Prediction Value) FDR=FP/(FP+TP); (False Discovery Rate) HSS= (2*((TP*TN)-(FP*FN))) / (((TP+FN)*(FN+TN)) + ((TP+FP)*(FP+TN))); (Heidke Skill Score) MCC= ( (TP*TN)-(FP*FN) ) / ((TP+FP)*(TP+FN)*(TN+FP)*(TN+FN)) 2 ; (Mathews Correlation Coefficient)

33 Input using CV Cross-Validation on the Segmented Dataset MSE TPR FPR TNR FNR FAR ACC HSS Cross-Validation on the Operational Dataset Input using CV MSE TPR FPR TNR FNR FAR ACC HSS

34 Train: Segmented Test: Segmented Input/Test Train MSE TPR FPR TNR FNR FAR ACC HSS , , Train: Operational Test: Operational Input/Test Train MSE TPR FPR TNR FNR FAR ACC HSS , , Train: Segmented Test: Operational Input/Test Train MSE TPR FPR TNR FNR FAR ACC HSS , ,

35 Training Data Comparison with ASAP COST ES0803 Alcala 2011 ASAP was trained on a dataset covering a longer time period, it used a different association technique to compare ARs and flares, and did not discard ARs above 45 from the solar centre. It therefore has more data samples in the training and testing sets. Jan1982 Dec2006 (1Feb Dec2002) ASAP Flare Prediction Capability No. of Flaring Testing Data ARs in the TPR ACC FAR MSE HSS Testing Set 1Feb Dec2002 5,

36 Evaluation of FS data sets Benchmark Machine Learning using Cross Validation on the 21 MF Properties MSE TPR FPR TNR FNR FAR ACC HSS Machine Learning using Cross Validation on the Selected MF Properties Group No MSE TPR FPR TNR FNR FAR ACC HSS G G G G

37 Organisation Introduction ASAP SMART-ASAP Evaluation Final Words

38 Early days still, but results so far seems very promising FS not applied to enhance the flaring prediction but to provide better physical insight. SMART and ASAP are both automated. The execution time of SMART is about 20-60, while its about 10 seconds for ASAP on a computer with 2.66 GHz Intel core 2 duo with 2GB of 800MHz DDR2 SDRAM. Hence, both systems can be run integrated to run on realtime.

39 But, Can the prediction performance be improved further (other technologies could be investigated such as HMM, SVM, RBF)? More MFs properties needed? All clear prediction test? SDO s Challenge: ASAP s technology can handle SDO. SMART is to be updated. Both to be tested again!

40 MDI 1K 1K HMI 1K 1K HMI 2K 2K HMI 4K 4K

41