Application of automated CB/TCU detection based on radar and satellite data

Transcription

1 Application of automated CB/TCU detection based on radar and satellite data Paul de Valk, and Rudolf van Westhrenen Royal Netherlands Meteorological Institute, Ministry of Infrastruture and Environment Po box AE De Bilt, The Netherlands Abstract The turbulence, precipitation, and lightning associated with Cumulonimbi (Cb) and towering cumuli (Tcu) form a hazard for aviation. Their detection especially in aerodromes is a requirement by ICAO to limit hazardous flying conditions. The Cb and Tcu observations are predominantly done by human observers. In 2007 at two regional civil airports in the Netherlands, Maastricht-Aken (EHBK) and Eelde (EHGG) the visual observations are replaced by automated observations. The automated detection of Cb and Tcu is based on an algorithm using radar signals as input. The Probability of Detection and False Alarm Rate of this algorithm show ample room for improvement. Therefore research was initiated at KNMI to improve the performance of automated Cb/Tcu detection. Two parallel studies were performed, Carbajal-Henken, 2009 and De Valk and Westrhenen, Both studies showed that synergistic use of both radar and satellite data resulted in an improved detection performance. The results of the studies were presented during the Eumetsat User conference, in Cordoba, 2010 As the former study of Carbajal-Henken, 2009, concentrated on summer day only, the latter study of De Valk and Westrhenen, 2010 was used as basis for an operational algorithm, 24/7. The algorithm became operational since March 9, 2011 at three regional civil airports in the Netherlands, Rottterdam (EHRD) Maastricht-Aken (EHBK) and Eelde (EHGG). The paper describes the developments to come from a research version to an operational algorithm. It shows that the set goal of 65 percent probably of detection versus a false alarm ratio of 35 percent is achieved for Summer days Introduction The occurrence of Cb and Tcu and their associated weather phenomena, like lightning, heavy precipitation, and gusts form a hazard for aviation, especially near airports. Here an unexpected vertical movement of an aircraft can have serious consequences. It is therefore a primary requirement by ICAO to include the occurrence of Cb or Tcu in the METAR (Meteorological Aerodrome Report or MÉTéorologique Aviation Régulière) of an airport to limit the risks for aviation. The METAR report is predominantly given by observers. In 2007 an automated Cb/Tcu detection system, hereafter referred to as the operational algorithm-2007, became operational at EHBK and EHGG. The algorithm-2007 uses radar and lightning observations. Its performance is not optimal, a study by The ( 2006), showed a probability of detection of 50 % and a False Alarm ratio of 70 % as averaged values over the whole year. Studies were performed at KNMI to explore the possibility to improve this performance by combining radar and satellite observations. A master thesis study (Carbajal-Henken et al.,2009). Carbajal-Henken studied the daytime cases for the summer season for one airfield for four years. The thesis study indicated logistic regression to be a successful approach in the classification of Cb-Tcu. And a study which included all cases during night and day time, Winter and Summer season, De Valk and Westrhenen, The results of both studies were presented at the EUMETSAT user conference, in Cordoba, Spain, As an operational

2 algorithm runs throughout the year, the latter study was further developed to form the base of the operational algorithm. This paper describes the observation methods,the algorithm development, results and ends with the conclusion. Observations Only Radar and satellite observations have the temporal and spatial coverage required to observe convection. Radar (Radio detection and ranging) In the Netherlands two Doppler radars are operated primarily for precipitation detection. The C-band radar emits and receives pulsed 6 Ghz radio waves with a wave length of around 5 cm. The lowest inclination of the radar beam is 1 degree. Therefore the part of the atmosphere not observed by the radar increases with the distance to the radar position. The observed reflections are obtained from a distance from the Radar site (varying from km) and at moderate altitude (0.8-3 km) above surface of the earth. The reflection signal is proportional to the sixth power of hydrometeor diameter, when the particles are smaller than the wavelength, Holleman (2000). Due to the sixth power the variance of the reflectivity value is huge. Therefore a decibel or logarithmic unit is used to represent the signal. The radar reflections are projected on a grid with grid cells of 2.5 by 2.5 km. Satellites Meteorological satellites provide an instantaneous view of the atmospheric state. The geostationary satellites are an invaluable source of information for nowcasting. The latest generation of operational geostationary satellites provides an image each 15 minutes over Western Europe. They are operated by EUMETSAT. The Spinning Enhanced Visible and Infra red Imager (SEVIRI) on board the METEOSAT 8 and its follow-on observes the world since January SEVIRI is a passive instrument, it does not emit a signal, opposite to the radar. SEVIRI observes the reflection of the earth in spectral bands from 0.5 μm to 3.9 μm and the emission from the earth in spectral bands ranging from 3.9 μm to 13.4μm. Next to the eleven spectral bands, there is a high resolution visible (HRV) channel, 0.4 μm - 1.1μm. The sampling grid distance in the nadir point of the satellite is 3 km for the eleven channels and 1 km for the HRV channel. The observation cycle consists of a 12.5 minutes scan of the earth from south to north. Then the scan mirror returns to its starting position and calibration occurs in 2.5 minutes remaining from the 15 minutes cycle. Further details on the satellite platform and the SEVIRI instrument can be found at Algorithm development The radar nor satellite observations can discriminate between Cb and Tcu. Therefore both cloud types are treated as one category Cb/Tcu further used in the algorithm development. For the development of the algorithm the observations of 2005 were used. For this year the METAR classification of Cb/Tcu by observers was available to serve as the 'truth'. Logistic regression In study of De Valk and Westrhenen, 2010, some two hundred potential predictors are determined to classify the binary predictand: Cb/Tcu or non Cb/Tcu. A successful approach to come to binary results is the Logistic regression, Wilks (1995) and Carbajal-Henken et al (2009). Logistic regression models result to a classification or prediction of a binary predictand while the predictor variables can be of any type. A nonlinear equation can fit the predictand c using a multiple number of predictors x. 1 P c = 1 exp b 0 b 1 x 1 b 2 x 2... b n x n With P(c) the probability that c occurs, b i the regression parameters and x i the predictor variables. The function is bounded between 0 and 1 due to its mathematical form allowing only for properly bounded probability estimates. The function drawn will always result in a S- shape curve. It is not possible a priori to indicate which predictors will lead to the best result in the desired classification.

3 The dependencies and correlations between them are too complex. Commonly used is the forward stepwise regression, Wilks (1995). In each step a predictor is added to the equation and based on the statistical scores it is decided if the additional predictor contributes to the overall performance. It is up to the user to decide how many steps or predictors contribute significantly to the classification performance. Using all predictors may lead to an over-fit regression, Wilks (1995). In an over-fit regression too many predictors are used in the equation to describe the observations. The regression will fit to the used observations but the equation may fail to describe other observations not used for its determination. To assess the performance of the obtained equation it is recommendable to split the data set in two parts: one part is referred to as dependent set, the other part is the independent set. By logistic regression predictor variables and regression parameters, also called coefficients, are determined on the dependent part. The performance of the derived predictors and coefficients are tested and evaluated on the independent set. Verification Cb/Tcu occurrence is a dichotomous phenomenon. The frequency of Cb/Tcu occurrence is relatively low in comparison to the total number of METARs. The value of a forecast or classification can be assessed by comparison to an observation. Frequently used for assessment is the contingency table 1. Here the occurrences of forecast/classification in comparison to observations are represented. observed yes observed no classified yes hits false alarms classified no misses correct negatives Table 1. Contingency table,( Wilks 1995). Relationship between the number of observed and classified cases of a dichotomous phenomenon. The sample size is the sum of the hits, misses, false alarms and correct negatives. From the table a number of scores can be calculated. Given the large number of correct negatives for this specific Cb-Tcu classification it is not incorporated in any of the scores used here. It may lead to an incorrect interpretation of the results. Considered are, the Probability of Detection (POD= Hits/ (Hits + Misses) ), The False Alarm Ratio (FAR= False Alarms / (Hits + False Alarms)) the Critical success index (CSI= Hits/ (Hits + Misses + False Alarms)) or threat score, and the BIAS(= (Hits + False Alarms) / (Hits + Misses). Evaluation of METAR as truth During the studies uncertainties arose on the used collocation area within the METARS. Intermediate results suggested that various radii were used for the circular collocation area. To investigate this a separate study was initiated. An experienced forecaster evaluated all cases of 2005 for the major airports, EHAM, EHRD, EHGG, and EHBK. A classification for CB/Tcu was produced for two collocation areas with 15 and 30 km radius respectively. The result consists of a large number of evaluations: [number of days times number of METARS times number of airports times number of collocation areas]. The forecaster had access to all radar and satellite observations. He had the advantage that he could scroll forward and backward in the time through the observations. An observer at the airports does not have the possibility to scroll forward in the time to interpret developments in the atmosphere. Often the view of the observer is obscured by other clouds, and he has a limited view in night conditions. The forecaster had also access to radio-sonde data and analysed weather maps, this helps to classify CB/Tcu as the likelihood of instability occurrence can be assessed. The forecaster can also discriminate embedded Cb's. The forecaster, however, will not be able to identify small sub pixel phenomena. It is however uncertain whether sub grid cloud phenomena will form a hazard for aviation when they remain smaller than a pixel. Therefore it is assumed that a wrong classification of the undetectable small scale clouds will not hamper this study. The created data set enabled a consistent evaluation of Cb/Tcu occurrence. The used radii for the collocation area were fixed and for the four separate aerodromes an identical identification method was used. A benefit is the possible classification of embedded Cb. An additional benefit was the inclusion of the night time cases for EHGG and EHBK into the study. The night time METARS for these airports were not available in 2005, as these observations were stopped earlier in May The evaluated data set was used as predictand in a logistic regression evaluation instead of the METAR used in De Valk and Van Westrhenen 2010.

4 2005 versus 2009 data Within the study of the 2005 evaluated data set the five predictors with the highest explanatory power were: -HRVmax-HRVmin daytime only, -The maximum radar contour, -The averaged cloud top temperature, -The standard deviation of the cloud top temperature, -The number of pixels with T39-T108< 0 night time only, see also Mecikalski To extend the number of cases the forecaster also evaluated It was anticipated that the selected predictors found in the 2005 study would be applicable for the 2009 results. Using these predictors derived from 2005 for the 2009 cases showed unfortunately a poor performance. While studying potential causes of this poor performance it became apparent that the radar resolution was in 2008 increased and the pseudo capi height was changed from 800 m to 1500 m. Before 2008 the radar results were distributed on 2.5 x 2.5 km 2 grid, in 2008 the observations were distributed on a 1 x 1 km 2 grid. The 2.5 x 2.5 km 2 is still distributed as a number of applications use this format as a basic input, but it is derived from the 1 x 1 km 2 grid. Extended research indicated that these changes had an impact on the derived coefficients for the predictors. An evaluation of the 2009 data set showed a change in the predictor set with explanatory power in comparison to the study based on the 2005 data set. The five predictors with the highest explanatory power for the 2009 study were: number of radar contours with 14 dbz (0.25 mm/hr) the maximum radar contour cloud top temperature average number of pixels T03.9-T10.8 < 0 (night time only) HRV maximum - HRV minimum, contrast (day time only.) The predictors were scrutinized for their logical physical relation to the occurrence of Cb/Tcu. The change in radar resolution and pseudo cappi height showed the sensitivity of the optimized merged systems to changes. Whenever there are instrument changes or additions the whole algorithm has to be recalibrated and evaluated. This also limits the possibility to share the system with other (third) parties, when they use different instrumentation. The coefficients and the predictor selected on base of the 2009 observations were used within the operational algorithm. Results The evaluation by a forecaster is extended to 2010 with the inclusion of more airports and platforms. This allows for a performance evaluation of the operational algorithm. Regrettably only the data for the summer months, June- August 2010 were available at the time of writing this study. The results are evaluated per category: summer, winter, day, night per aerodrome. As an example the POD, FAR, BIAS and CSI for EHAM summer day is given In Figure 1, left part. From the POD, FAR, BIAS and CSI diagrams a threshold can be determined on which the classification can be based. The wide variety occurring within the graphs shows that one fixed value of threshold for all categories cannot be determined. For each season and day/night situation a threshold can be derived. The attributes diagram compares the predicted probability to observed relative frequency. The predicted values are binned into 10 percent bins, the occurrence number is given in the diagram and also in the histogram. The no-resolution line relates to the climatology, in this study the number of Cb occurrences compared to all reported METARS in the category. The no-skill line is halfway the no-resolution line and the perfect reliability line, which is represented by the diagonal. An example for EHAM Summer day is given in figure 1, right part. In figure 2 the results for EHAM summer day and night are shown in a POD-FAR diagram. Here the values of POD, FAR, CSI and BIAS are displayed for a range of probability thresholds from 0.2 to 0.5. The results obtained with coefficients derived from the 2010 data set as dependent data are included for comparison. Also the METAR results compared to the evaluated data set are included.

5 Figure 1 Scores for EHAM summer day (left). POD (open squares), FAR (black squares), CSI (pluses) and BIAS (dashed lines) as a function of the probability threshold. The attributes diagrams for EHAM summer day (right) as a function of the predicted probability. The numbers in the figure indicate the number of cases per 10 percent bin. The no resolution line relates to the climatological Cb occurrence, different for each station and season. The perfect reliability line is the diagonal. The no skill line is halfway the diagonal and the no resolution line. Figure 2 The lines with bullets indicate the variation in performance due to the variation of the probability thresholds of the three cycling independent data sets for EHAM summer day (left) and EHAM Summer night (right ).The red line with bullets indicate the performance of the operational algorithm for the 2010 case. The blue line indicates the performance of 2010 derived coefficients, dependent data set. The black line indicates the inclusion of the CAPE from NWP as a predictor. The performance curves are given for probability threshold values ranging from 0.2, upper right, to 0.5, lower left, in steps of 0.05 as a function of FAR and POD. Note that an increase of probability threshold will result in a lower POD and a lower FAR. Isolines of CSI in dotted black varying in steps of 0.1 from 0.9, the values of 0.1, 0.5, and 0.9 are indicated in the top of the figure. The red line denotes the BIAS is 1, right to this line are higher values of bias, left lower values. The black square denotes the METAR performance versus the evaluated data set.

6 Conclusion and Discussion An automated Cb-Tcu detection algorithm based on the synergy between radar and satellite observations is developed and made operational since March 9, It replaces the formerly used algorithm which used only radar data. Logistic regression forward stepwise is applied to determine the probability of Cb-Tcu occurrence. The predictors with the highest explanatory power for Cb/Tcu occurrence were: number of radar contours with 14 dbz (0.25 mm/hr) the maximum radar contour cloud top temperature average number of pixels T03.9-T10.8 < 0 (night time only) HRV maximum - HRV minimum, contrast (day time only.) In comparison to the Human observer the system has a higher POD but also a higher FAR. Where the observer in the shown results reaches a CSI of close to 0.5, the automated system reaches 0.6. Feedback by end users was received on the performance of the detection algorithm. From the feedback some weaknesses of the algorithm could be identified, which will be addressed in the next version of the algorithm. Intense frontal rain can lead to misclassification. Probably instability information of NWP (CAPE or other instability index) could mitigate this error. Special care is required to avoid a mixture of observation and NWP information which could feedback into the model as observation. As an example the impact of CAPE consideration on the performance of the model using CAPE as a predictor is included in Figure 2. Its impact appears marginal. The studied period in this paper is the summer. More impact is anticipated in the periods with more frequent cases of frontal rain, like occurring in autumn and winter. Extended Mesoscale Convective systems were wrongly classified. An improvement of the radar contour calculation will mitigate this erroneous classification. Coverage was too high. This can be corrected by adjusting the thresholds for coverage. But it will be nearly impossible to verify this. The observation of coverage of Cb can only be done by an human observer and his view can be limited by other cloud layers. Combining various observations methods within one classification algorithm increases the performance of detection. But it is also vulnerable to changes in one of the considered systems. If one system changes a new evaluation of the combined system is required. This requires a continuous monitoring of the performance, as not all changes can be foreseen. Literature. Carbajal Henken, C., Schmeits, M., Wolters, E. and Roebeling, R. Detection of Cb and Tcu clouds using MSG-SEVIRI cloud physical properties and weather radar observations. KNMI WR Holleman, I. Introduction to Radar internal webpage at KNMI. Mecikalski, John R. and Bedka, Kristopher M.. Forecasting convective initiation by monitoring the evolution of moving cumulus in daytime GOES imagery. Monthly Weather Review, Volume 134, Issue 1, 2006, pp Mecicalski (2007) -based Convective Initiation Nowcasting System Improvements Expected from the MTG FCI Meteosat Third Generation Capability, EUM/CO/07/ /JKG Technical Report The H. (2006), Verificatie AUTOMETAR CB/TCU- detectie (herziene versie) (internal KNMI report) P. de Valk, R. van Westrhenen, 2010, Probability of Cb and Tcu occurrence based upon radar and satellite observations, KNMI WR Wilks, D.S., (1995) Statistical Method in the atmospheric sciences, academic press, New york