Automatic Solar Filament Detection Using Image Processing Techniques

Transcription

1 Automatic Solar Filament Detection Using Image Processing Techniques Ming Qu and Frank Y. Shih College of Computing Sciences, New Jersey Institute of Technology Newark, NJ U.S.A. Ju Jing, Carsten Denker and Haimin Wang Center for Solar-Terrestrial Research, New Jersey Institute of Technology Newark, NJ U.S.A. Big Bear Solar Observatory, New Jersey Institute of Technology North Shore Lane, Big Bear City, CA U.S.A. Aug. 18th, 2004 Abstract. We present an automatic solar filament detection procedure using advanced image enhancement, segmentation, pattern recognition and mathematical morphology methods. This procedure not only provides the detection results of filaments, but also identifies the spines, footpoints and disappearances of filaments. A generic procedure consists of the following steps: 1) Filaments are emphasize and sharpen by the stabilized inverse diffusion equation (SIDE) which was introduced by Pollak et al. (2000). 2) To set the global and local thresholds for segmenting filaments from the background, a new algorithm for automatic threshold selection is proposed. 3) An efficient feature-based classifier, the Support Vector Machine (SVM), is utilized to distinguish sunspots from filaments. 4) Filament identification is achieved by morphological thinning, pruning and adaptive edge linking methods. 5) Finally, we propose a filament matching method to detect the filament disappearances. We have experimented our procedure with a large amount of Hα full-disk images observed at the Big Bear Solar Observatory (BBSO) in California, and obtained better results comparing to the results of Gao et al. (2002) and Shih and Kowalski (2003). We believe our work will lead to the real-time solar filament detection using advanced image processing techniques. 1. Introduction Filaments are located in the corona, but have temperatures only one hundredth of the corona and densities one hundred times greater than the local corona values. In Hα images, filaments are seen as dark ribbons against the bright solar disk which is shown in Figure 1. Filament eruptions, flares and Corona Mass Ejections (CMEs) are the most important solar events as far as space weather effects are concerned linking to solar eruptions, major interplanetary disturbance and geomagnetic storms (Gosling et al., 1991). Increasing observational evidence that there is an association between filament eruptions, flares and CMEs confirms that they are different manifestations of one physical process at different evolutionary stages (Gilbert et al., 2000; Gopalswamy et al., 2003). In order to gain a better understanding of CMEs and furthermore, to develop the geomagnetic storm predictions, it is essential to find early manifestations of CMEs in the solar atmosphere (i.e., filament eruptions, flares) (Jing et al., 2004). Previous work on the detection of filaments is based on thresholding methods. Gao et al. (2002) detected the filament using global thresholds which are chosen by median values of the image intensity. This method could not handle the low contrast filaments and produce unstable results of filaments. Shih and Kowalski (2003) adapt local thresholds which are chosen by median c 2005 Kluwer Academic Publishers. Printed in the Netherlands. filament6.tex; 13/01/2005; 17:58; p.1

2 2 a Figure 1. Filaments in an Hα image. values of the image intensity to extract filaments. The local method is effective in measuring the low contrast filaments and small features. However, the observational background may change from time to time. It is difficult to set the local thresholds for each sub-image. The median criterion for the threshold selection cannot guarantee robust results in that the bright features on images can significantly affect the value of thresholds associated with the median value. In order to distinguish sunspots from filaments, Shih and Kowalski (2003) proposed the sunspot removal method based on darkness. But the single darkness classifier is not able to yield accurate result, especially when the sunspots occur near the limb of the sun. We propose a procedure for the automatic detection of filaments and their disappearance which are shown in Figure 2 & 3. We use full-disk Hα images as the data set for our procedure. During the last few years, the Big Bear Solar Observatory (BBSO) has developed a new generation of well-calibrated, photometric Hα full-disk observations (Denker et al., 1999), which include limb darkening correction to enhance features on the disk as well as above the limb. Generally speaking, the procedure consists of the following steps. 1. In the first step, the stabilized inverse diffusion equation (SIDE), that preserves the high frequencies of the image via the evolution of nonlinear partial differential equations (PDEs) (Pollak et al., 2000), was applied to emphasize and sharpen the features (i.e., filaments) for the further processing. filament6.tex; 13/01/2005; 17:58; p.2

3 2. In the second step, we proposed a new algorithm for automatic threshold selection from the result of edge detection. The edges of the filaments were detected by Sobel operator. Then we compute the segmented regions for each potential threshold. The difference regions made by two successive thresholds were computed iteratively. The optimal threshold is the one that best matches with the edge detected previously. 3. In the third step, a new and efficient feature-based classifier, the Support Vector Machine (SVM), was used for distinguishing sunspots from filaments. For the present study, a sunspot is represented by nine features. As far as we know, it is the first time that SVM is applied for sunspot detection. Experimental results show that this method significantly improved the classification rate in comparison with other methods (Shih & Kowalski, 2003). 4. Next, because some isolated regions might be the parts of the main body of one filament, we checked the relationship between the segmented regions to determine if they should be integrated. This was achieved by morphological closing, thinning, pruning and adaptive edge linking methods. Consequently, the spines and the foot-points of filaments were obtained. 5. Finally, we detect the filament disappearance by comparing the results, obtained from the step 4, on two successive days. This paper is organized as follows. Section 2 shows system algorithms. Section 3 demonstrates our results. Section 4 gives the conclusions Image Enhancement 2. System Algorithms A SIDE is a dynamical system that has been inspired from a stabilized limiting case of a spacediscrete nonlinear diffusion filter (Pollak et al., 2000). The SIDE filter is an approach to enhance edges and remove unimportant small features. Let us consider a discrete signal f = (f i ) N 1 i = 0. Then its SIDE evolution produces a sequence of filtered images u(t) = (u i (t)) N 1 i = 0 with u(0) = f. Increasing the time t leads to a consecutive merging of regions. The evolution between two merging events is governed by a dynamical system with discontinuous right hand side. The SIDE algorithm for 2D images proceeds as: 1. Start with the trivial initial segmentation: each pixel n i is regarded as a region of size m ni. 2. Evolve Equation (1) until the values in some neighboring regions become equal. u ni = 1 m ni n j A ni F (u nj u ni )p ij, (1) where m ni is the number of pixels in the region n i ; A ni is the set of indices of all the neighbored regions of n i ; n j is a neighbored region of n i ; p ij is the number of neighbored filament6.tex; 13/01/2005; 17:58; p.3

4 4 Obtain full-disk images from the BBSO Preprocess: remove the background of the sun Image enhancement by the SIDE Image segmentation Edge detection Adaptive thresholding based on the edge detection Sunspot removal by the SVM classifier with nine features Directional morphological closing Component labeling Morphological thinning and pruning Adaptive edge linking Save the result of filaments Figure 2. The procedure of the filament detection. pixels between regions n i and n j. The following force function F is proposed by Pollak et al. (2000): F (v) = sgn(v) v (2) L where sgn is defined as the sign of the value. sgn(v) returns + if v is positive and if v is negative. In order to satisfy the conditions of a force function, L is larger than the dynamic range of the image to be processed. 3. Merge the neighboring regions with equal gray levels. 4. Stop the evolution if the predefined number of regions or times of the evolutions are met; otherwise, go to step 2. Two neighboring regions n 1 and n 2 are merged by replacing them with one region n of mass m n = m n1 + m n2 and the set of neighbors A n = A n1 A n2 \ {n 1, n 2 }. filament6.tex; 13/01/2005; 17:58; p.4

5 5 Yesterday s image: A Today s image: B Apply differential rotation to A for corresponding B Rotated A Filament detection Filament result of Rotated A Filament result of B Match filaments and detect disappearances Provide the filament disappearance report Figure 3. The procedure of the filament disappearance detection Image Segmentation Adaptive Thresholding Based On Edge Detection We utilize the Sobel edge detector to obtain the edges. The Sobel operator performs a 2D spatial gradient measurement on an image and emphasizes regions of high spatial gradient that correspond to edges. Typically it is used to find the approximate absolute gradient magnitude at each point in an input gray-scale image (Jähne, 1997). The operator consists of a pair of 3 3 convolution masks and one mask is simply the other rotated by 90. Our adaptive thresholding method contains two steps. In the first step, the best global threshold for an image is calculated using edge information. A sequence of self-adaptive thresholds t 1 < t 2 <... < t n ranging from 0 to the median of intensity value of the image are chosen for segmenting dark regions. The region r i is computed by the threshold t i. The best global threshold t max is computed by Equation (2). r 1 = r 1, r i = r i r i 1, g i = Sobel(r i ) n r i, filament6.tex; 13/01/2005; 17:58; p.5

6 6 i max = index of max(g i ), t img = t imax, (3) where n r i is the number of pixels in the region r i. For example, we utilize thresholds t 1 < t 2 <... < t n to get regions r 1 < r 2 <... < r n respectively. New added regions are r 1, r 2,..., r n which r i = r i r i 1 and r 1 = r 1. The average spatial gradient g i for each pixel on a new added region is computed by Sobel edge detector. When the new added region r i meets edges, the gradient g i can reach the maximum. By finding the maximum average gradient for each pixel on the new added region g i, the best global threshold t img for an solar image is obtained. In the second step, the local thresholds are calculated for the pixels sub-images on the pixels Hα image. The same threshold selection method are used with the following three criteria: 1. The local threshold t loc = t img ± 20; 2. The new segmented region is less than three times of the old region; 3. The gradient for each pixel of the new added region is greater than the half of the global average pixel gradient g imax. These criteria are used to speed up the processing time and obtain the robust results correspond to the results by the global threshold. After the segmentation, the small regions which are less than 10 pixels are considered as noises and are removed Sunspot Removal Segmented dark features include filaments and sunspots. It is necessary to remove sunspots from the results to improve accuracy. In our sunspot detection, there are two classes: sunspot and nonsunspot. Nine input features are proposed to represent the sunspots, and SVM is used to separate the two classes in the nine feature space using a hyperplane. SVM is a learning system based on advances in statistical learning theory. The basic idea of a support vector machine is to separate the fixed given input pattern vectors into two classes using a hyperplane with the maximal margin which is the distance between the decision plane and the closest sample points (Vapnik, 1998). Points on the decision plane are called support vectors. The support vectors can be found by the classical method of Lagrange multipliers. SVM is based on the structural risk minimization (SRM) inductive principle. The SRM principle is intended to minimize the risk functional with respect to the empirical risks and the confidence interval (Vapnik and Chervonenkis, 1974). For the nonlinear case, SVM use the kernel functions to transfer input data to feature space. For example, Polynomial support vector classifier and Gaussian Radial Basis Function (RBF) kernel classifier are two of nonlinear classifiers. Considering the complexity of the sunspot detection method, linear SVM is used to minimize the empirical risks and confidence interval. SVM has been successfully applied in the solar flare detection and has been proved that the SVM classifier is better than the neural network classifier (Qu, 2004). In our experiment, we adopt the SVM classifier with the linear kernel. The comparisons between the linear kernel filament6.tex; 13/01/2005; 17:58; p.6

7 and the other kernels can be found from Qu et al. (2003). For further information about SVM, readers can refer to Guyon and Stork (2002) and Vapnik (1998). These nine features for sunspots can be computed from the result of segmented regions and the corresponding regions on the original Hα image. Let us denote the whole segmented regions as A; a segmented region as s; a small pixels window, where the region s is on the center, as W 1; a small window, which just enclose s, as W 2. The nine features of s are described below. Feature 1: number of pixels of s denotes as n s. Feature 2: radial position of the center of s. Feature 3: distance from the center of the s to the equator of the solar disk. We define the distance d by d = y y c (4) where y is the y coordinate position of s and y c is the Y coordinate position of the equator of the solar disk. Feature 4: mean ratio is defined by m 1,2 = x 1 x 2 (5) where x 1 represents mean brightness of s and x 2 represents mean brightness of A. This factor is used to measure the brightness differences between sunspots and other features on an image. Feature 5: standard deviation ratio is defined by std 1,2 = std 1 std 2 (6) where std 1 denotes standard deviation of s and std 2 denotes standard deviation of A. Feature 6: mean brightness of a small window W 1. This feature considers the effect of the local neighbors of s such as active regions and flares. Feature 7: standard deviation of W 1. Feature 8: shape feature (1) is defined as h 1 by where m W 2 is the number of pixels of W 2. Feature 9: shape feature (2) is defined as h 2 by h 1 = n s /m W 2 (7) h 2 = x W 2 /y W 2 (8) where x W 2 is the length size of W 2 and y W 2 is the height size of W Component Labeling by Morphology methods Directional morphological closing After sunspot removal, morphological closing is used to eliminate small gaps. Shih and Kowalski (2003) introduced the eight directional linear structuring elements with 0, 22.5, 45, 7 filament6.tex; 13/01/2005; 17:58; p.7

8 8 67.5, 90, 112.5, 135, slopes, respectively (Shih and Gaddipati, 2003). The closing of A by B denoted by A B, is defined as A B = (A B) B (9) where denotes the morphological erosion and denotes the morphological dilation Morphological thinning and pruning The morphological thinning is defined as: A B = A (A B) (10) where A B denotes the morphological hit-and-miss transformation of A by B (Gonzalez and Woods, 2002). For thinning A symmetrically, use a sequence of structuring elements: B = B 1, B 2, B 3,..., B n, where B i is a rotated version of B i 1. The entire process is repeated until no further changes occur. There are four steps in the morphological pruning which uses four steps to yield X 1, X 2, X 3 and final pruning result X 4 by Equation (10), (11), (12) and (13) respectively (Gonzalez and Woods, 2002). X 1 = A {B} (11) where A is an input set and B is pruning structuring element sequence. X 2 = (X 1 B) (12) where H is a 3 3 structuring element of 1 s. X 3 = (X 2 H) A (13) X 4 = X 1 X 3 (14) The original pruning method is used to remove the fix length of a parasitic component. For our filament pruning, we propose an adaptive pruning method by monitoring X 3. Repeat step 1 to step 3 to remove pixels until X 3 has two points left. Then the fourth step is invoked to compute the final results. The pruned results are lines which represent the spines of the filaments. The final two points in X 3 denote the footpoints of the filaments Adaptive edge linking Some isolated regions might be parts of the main body of one filament. We call the isolated regions as broken filaments. Big gaps of broken filaments could not be eliminated by directional morphological closing. Therefore we adopt adaptive edge linking method which is introduced by Shih and Cheng (2004). The basic idea of adaptive edge linking is to connect edges based on the orientation of spines. For each footpoint of a spine, the direction of the spine is obtained by computing the edge slope direction of the footpoint. The edge linking operation is performed in an iterative manner, so that the broken filaments can be linked up gradually and smoothly filament6.tex; 13/01/2005; 17:58; p.8

9 while the details of the object shape are preserved. When two regions are connected, the spines and footpoints are updated. The final results of the spines and footpoints of filaments are saved for the detection of filament disappearance Filament disappearances We have developed a computer program to detect filament disappearances automatically and send the filament disappearance report to our researchers automatically. First, download two consecutive Hα images A and B on successive days. Second, rotate A to match B using differential solar rotation program called drot map in the Solar SoftWare tree (SSW; Freeland and Handy, 1998). Our enhancement, segmentation and morphology methods are applied to obtain the segmented filaments and their spines and footpoints. To detect the disappeared filaments on two consecutive images automatically, we propose the following component matching method. 1. The spines of filaments are dilated with a pixels structuring element. 2. When two filaments are overlapped, match them according to the size of spines and the intensity of filaments. The intensity of a filament is calculated by the mean value of the intensity of the filament on the original image corresponding to the spine of the filament. The size of the spines is matched if the size of the spine on the current day is greater than 30 percent of the size of the spine on the previous day. The intensity of the filaments is matched if the intensity of the filament on the current day is less than 2 times of the intensity of the filament on the previous day. 3. Filaments are matched if the size of the intensity are both matched. Unmatched filaments on the previous image are reported as disappeared or significantly shrinked filaments Results The solar Hα images of pixels were obtained from the BBSO. 50 images are randomly selected starting from January 1, 1999 to September 1, To achieve the goal of automatic segmentation, images are selected with high and low contrast filaments and other features. In this paper, we present the detection results of the image observed on Oct. 24, 2003 which contains high contrast and low contrast filaments, and sunspots. Before our procedure, we adopt some preprocessing methods which can be found from SSW tree (Freeland and Handy, 1998). They include the method for removing background of the sun and differential solar rotation Filament Enhancement In this section we present the image enhancement results of filaments. In our experiment, the resolution of images is 8 bits and the maximum neighbor difference L for force function F, described in Section 2, is 100. We compare the results of recursive soft morphological filters filament6.tex; 13/01/2005; 17:58; p.9

10 10 a b c d Figure 4. a. Original images, b. Result of the recursive soft morphological filters, c. Result of the Perona-Malik filters, and d. Result of the SIDEs. (Shih and Puttagunta, 1995), Perona-Malik filters and SIDEs which are shown in Figure 4. From experiments, we observe that some low contrast filaments are better enhanced by SIDEs rather than other filters. Background is suppressed without blurring the edges of solar features. After the SIDE enhancement, the edge of features are very clear and the features are easy to be segmented by edge-based methods. filament6.tex; 13/01/2005; 17:58; p.10

11 11 Table I. Filament and sunspot detection rate using different methods. Method LF (%) SF (%) CS (%) Gao s method Shih s method Our method a LF - Large filaments; b SF - Small filaments; d CS - Classified sunspots Filament segmentation and sunspot removal The enhancement and segmentation result for full-disk Hα images are shown in Figure 5. After the segmentation, we apply the SVM classifier to distinguish sunspots from filaments. For sunspots classification, 50 sunspots and 50 non-sunspot features are collected by our segmentation procedure. We randomly select 25 sunspots and 25 non-sunspot features as training samples to train the SVM classifier. Other features are used as testing samples to test the SVM classifier. The sunspot removal result is shown in Figure 5. The classification rate is shown in Table I Component labeling and filament disappearances Component labeling method is used to connect broken filaments and find spines and footpoints of filaments. Figure 6 shows the result of directional morphological closing, thinning, pruning, and directional edge linking. Disappeared filaments are detected based on the result of component labeling. Since the research on filament disappearances is focused on large filaments, disappearances of small filaments whose spines are less than 60 pixels are not reported. Figure 7 shows the result of filament disappearances. The gray and dark spines are dilated with pixel structuring elements for matching the filaments. The result shows a filament disappearance on the right side of the region Computation Time We develop our programs using Interactive Data Language (IDL) by Research Systems, Inc. The programs run on a computer with a 1.7 GHz Pentium 4 processor and memory of 512 Mbytes under Windows We use pixels solar images with 8-bit gray levels. The computational time of our detection method is given in Table II. filament6.tex; 13/01/2005; 17:58; p.11

12 12 a b c d Figure 5. a. Original Hα image observed on Oct. 24, 2003; b. Result of image enhancement and background removal; c. Image segmentation result using our adaptive thresholding method; d. Result of removing the sunspots and removing small regions which are less than 10 pixels. The sunspots are indicated by the rectangle boxes. filament6.tex; 13/01/2005; 17:58; p.12

13 13 a b c d Figure 6. a. Result of filament segmentation; b. Result of directional morphological closing; c. Result of morphological thinning and pruning; d. Result of adaptive edge linking. 4. Summary In this paper we have presented a new automatic procedure for the filament detection using the advanced image enhancement, segmentation, pattern recognition and morphology methods. We conclude the procedure as follows. 1. The SIDE can best enhance filaments and minimize small feature effects. filament6.tex; 13/01/2005; 17:58; p.13

14 14 a b c d Matched Unmatched e f Figure 7. a. Differential rotated image obtained on Oct 24, 2003 whose original image has been shown in Figure 1; b. The same area of (a) obtained on Oct 25, 2003; c. The corresponding result of the filament detection for (a) ; d. The corresponding result of the filament detection for (b); e. Overlaid (c) with (d); f. The result of matching the filaments. filament6.tex; 13/01/2005; 17:58; p.14

15 15 Table II. Computational time of segmentation. Method Computational Time (seconds) Image enhancement with SIDEs 21. Adaptive thresholding with region detection 5.9 Sunspot removal 1.47 Directional Morphological closing and small region removal 1.4 Morphological thinning and pruning 12.3 Directional edge linking 16.9 Filament disappearance detection Filaments are segmented using our adaptive threshold selection method. The results of the filament segmentation show our method are efficient and robust comparing to the previous two methods. 3. Sunspots are distinguished from filaments using the SVM classifier with the nine features. 4. The directional morphological closing, thinning, pruning and directional edge linking methods are used to connect filaments and measure the spines and footpoints of filaments. 5. The filament disappearance is detected by component matching method. The automatic solar filament detection methods allow us to study the evolutions of a large number of filaments automatically and efficiently. Furthermore, The filament disappearance report is helpful for analyzing filaments promptly and forecasting the space weather. In the future, the detection program for filaments and filament disappearances will be one of our web-based monitoring tools. Acknowledgment We thank the referee for providing constructive comments and help in improving the contents of this paper. This work is supported by National Science Foundation (NSF) under grants IIS , ATM and ATM References Denker, C., Johannesson, A., Goode, P. R., Marquette, W., Wang, H., and Zirin, H.: 1999, Solar Phys. 184, 87. filament6.tex; 13/01/2005; 17:58; p.15

16 16 Freeland, S. L. and Handy, B. N.: 1998, Solar Phys. 182, 497. Gao, J., Wang, H., and Zhou, M.: 2002, Solar Phys. 205, 93. Gilbert, H. R., Holzer, T. E., Burkepile, J. T., and Hundhausen, A. J.: 2000, Astrophys. 537, 503. Gopalswamy, N., Shimojo, M., Lu, W., Yashiro, S., Shibasaki, K., and Howard, R. A.: 2003, Astrophys. 586, 562. Gosling, J. T., McComas, D. J., Phillips, J. L., and Bame, S. J.: 1991, J. Geophys. Res. 96, Guyon, I. and Stork, D. G.: 2000, Linear discriminant and support vector machine, The MIT Press, Cambridge, p Gonzalez, R.C. and Woods, R.E.: 2002, Digital Image Processing, Prentice Hall. Jing, J., Yurchyshyn, V. B., Yang, G., Xu, Y., and Wang, H.: 2004, Astrophys. 614, Jähne, B.: 1997, Digital Image Processing, Springer, p Kichenassamy, S.: 1997, SIAM. J. Appl. Math 57, 5, p Pollak, I., Willsky, A.S. and Krim, H.: 2000, IEEE Transactions on Image Processing 9, 2. Qu, M., Shih, F.Y., Jing, J., and Wang, H.: 2003, Solar Phys. 217, 157. Qu, M., Shih, F.Y., Jing, J. and Wang, H.: 2004, Solar Phys. 222, 137. Shih, F. Y. and Puttagunta, P.: 1995, IEEE Transactions on Image Processing 4, Shih, F. Y. and Kowalski, A. J.: 2003, Solar Phys. 218, 99. Shih, F. Y. and Gaddipati, V.: 2003, Pattern Recognition 36, Shih, F. Y. and Cheng, S. : 2004, Information Sciences 167, 9. Sobotka, M., Brandt, P. N. and Simon, G. W.: 1997, Astron. Astrophys. 328, 682. Vapnik, N. V. and Chervonenkis, A. J.: 1974, Theory of Pattern Recognition., Nauka, Moscow. Vapnik, N. V.: 1998, in S. Haykin (ed.), Statistical Learning Theory, John Wiley & Sons, Inc., NY, USA. filament6.tex; 13/01/2005; 17:58; p.16