Kalman filtering based probabilistic nowcasting of object oriented tracked convective storms

Transcription

1 Kalman iltering based probabilistic nowcasting o object oriented traced convective storms Pea J. Rossi,3, V. Chandrasear,2, Vesa Hasu 3 Finnish Meteorological Institute, Finland, Eri Palménin Auio, pea.rossi@mi.i 2Colorado State University, Fort Collins, United States 3Aalto University, Helsini, Finland (Dated: 3 May 22) Pea Rossi. Introduction Due to severe convective storms, many municipalities suer rom economic and ecological losses every year. The ability to predict the movement o the phenomenon up to a ew hours ahead would provide valuable inormation or end-users. A standard approach or nowcasting and analyzing convective weather is so-called radar based object oriented convective storm tracing, which captures the history and movement o an individual convective storm (e.g. Dixon and Wiener 993; Johnson et al. 998; Handwerer 22). However, storm tracing algorithms typically provide short term orecasts and warnings only in deterministic ashion. That is, the orecast model produces only a single orecast with limited guidance o the uncertainties related to the model. This has led to the development o the storm tracing algorithm presented in this paper, which explicitly addresses the issue o uncertainty o the orecasts. In this study, we introduce a Kalman ilter based nowcasting and tracing algorithm or convective storms. The algorithm introduces two major eatures in the ield o convective storm tracing. First, the algorithm provides a compact way to compute smoothed estimates or various storm parameters, such as storm position and velocity, which can be urther used or reliable nowcasting o the convective storm objects. Second, the estimated posterior distribution o the storm parameters can be used or deriving probabilistic nowcast products or the traced convective storms. 2. Kalman iltering In the ield o convective storm tracing algorithms, the concept o probabilistic analysis is still rare. However, in generic engineering tracing tass, probabilistic algorithms have been used or a long time (e.g. Bar-Shalom and Li 995). The most widely used method applied in probabilistic target tracing is deinitely Kalman iltering developed by Rudol E. Kalman in 96s (Kalman 96). Probabilistic approaches, including Kalman ilter, are also widely applied in numerical weather prediction (e.g. Zhou et al. 26). Kalman iltering can be regarded as a statistical inversion method, where the unnown state x o a system is estimated through noisy measurements y,, y N. When applying Kalman iltering with an object oriented convective storm tracing algorithm, each individual traced storm is regarded as the system o interest, and the hidden state x o the system can include, or example, position, velocity and area parameters o the storm. The goal is to provide an optimal estimate o the unnown state vector x at th time step using the previous noisy observations y,, y and a dynamic model or the temporal development o the state parameters. In case o the storm tracing, the observation vector y include variables that can be directly measured rom the radar-detected storm object, such as uncertain measurements o the area and centroid position o the storm. Kalman iltering assumes that the system can be presented with a linear stochastic state-space model with Gaussian noises, ormulated as x F x w y H x v, () where F is the state transition model, H is the observation model and v and w - are additional noise variables. The state transition model F deines how the model state evolves between successive time steps. However, no deterministic model describes any system perectly and thereore the random variable w - is added to describe uncertainties o the modeled system. Thus, w - is also called as the process noise and it is assumed to ollow Gaussian distribution with w ~N(, Q ). The observation model H, in turn, deines how the model state is mapped rom the state variables to the observations. The random vector v - is the measurement noise, which relates to the uncertainty o the measurements y. Lie the process noise, it is assumed to ollow Gaussian distribution with v ~N(, R ). It is also assumed that both v and w - are white noise processes and independent on the initial state x. The Kalman ilter estimate m and its covariance P or the hidden state x at th time step are estimated recursively with the ollowing two-phase algorithm

2 Prediction step: Update step: m F m P F P F Q T. K P H H P H R T T m m K y H m P IK H P. (2) (3) The variables m and P in the prediction step denote the predicted state estimate and covariance, that is, the estimated state estimate and covariance beore the they are updated with an evidence given by the measurement. In the update step, the state estimate and covariance are urther modiied using the noisy measurement y. In (3), K is Kalman gain, which deines the weight given to the observation with respect to predicted state. Furthermore, it is assumed that at the irst time step the system state has an initial state m and covariance P. From Bayesian point o view, the estimates m and P can be regarded as the mean and covariance o the posterior distribution o the state x, given the noisy measurement y and the prior mean m and covariance P. For urther discussion on general theory o Kalman iltering, we reer to various sources in the literature (e.g. Bar-Shalom and Li 995; Simon 26). 3. Kalman iltering model or tracing convective storms 3. Applied Kalman iltering model Kalman iltering model or traced convective storms includes state estimation o variables o storm cells. In here, the initial tracing, i.e. storm identiication and assignment, are perormed with the clustering based tracing algorithm by Rossi and Mäelä (28). Thereater, we apply Kalman iltering to the traced storms, which improves the state estimation o the storm cells. The estimated state inormation can be used, or example, in the storm nowcasting. The initial storm tracing algorithm applies the radar relectivity actor o 35 dbz or identiying storm areas, which enables tracing o ull storm systems, such as multicellular storms or MCS (Dixon and Wiener 993). The identiied storm areas are urther processed using the morphological closing operation with a round structuring element with 3 m diameter (e.g. Gonzalez and Woods 28). In many object oriented storm tracing applications the nowcasting o the storm cells is based on linear extrapolation o the position using previous position or velocity estimates (e.g. Dixon and Wiener 993; Handwerer 22). Thereore, also the state variables o our model are the centroid position (x,y) o an identiied storm and corresponding velocity components x, y, denoted as a state-vector x x x y y T. The only measured variables are the centroid position coordinates o the identiied storms and thereore the elements corresponding to the velocity observations in the measurement matrix H are zero. The velocity inormation is estimated later on using the applied dynamic model and the Kalman iltering equations. Thus, the linear observation model H o the system is and the measurement noise v is zero mean Gaussian white noise with covariance H, (4) 2 r R 2. (5) r In (5), the parameter r [length] describes the noise intensity o the measured centroid position. The uncertainty comes rom various sources, such as attenuation, randomness o the relectivity measurements and resolution limitations deined by the radar contribution volume, which results in random deormation o the identiied storm shape and consequently in luctuations in the centroid position. The measurement noise deviation o both x and y is o the order o r, which helps us to choose a suitable parameter or r. However, it is diicult to estimate the contribution o each noise source objectively and thereore in this wor the noise parameter r is selected intuitively. Moreover, the radar-detected centroid is only a noisy measurement o the centroid o the true storm cell, which includes not only the intense precipitating part observed by the radar, but also the non-measureable updrat and downdrat o air mass. In addition to the measurements and the measurement model, Kalman iltering requires a reasonable dynamic model to estimate the state variables o the traced storms. Here, we adopt the continuous white noise acceleration model, where position and velocity coordinates o a target are modeled with a linear state-space model and the acceleration is assumed to be a zero mean Gaussian random process (e.g. Bar-Shalom and Li 995). This means that the velocity o the target stays

3 unchanged on average, but over each time step, it undergoes a small Gaussian random change. The state transition model F and the process noise covariance Q o the discretized white noise acceleration model are (see e.g. Bar-Shalom and Li 995 or a detailed derivation) 3 2 t t 3 2 t 2 t t 2 F F( t), Q Q ( t) q, (6) t 3 2 t t t t 2 where t is the interval between two time steps and q [length 2 / (time step) 3 ] deines the process noise intensity. Random changes in the velocity over the time step length t are o the order o q t, which helps us to choose a reasonable parameter or the process noise. Lie with the measurement noise, the value o q can be chosen subjectively, based on an educated guess or expert nowledge on the storm velocity behavior. 3.2 Nowcasting with Kalman ilter One o the main applications o the Kalman ilter derived position and velocity is naturally short term prediction o the storm cells. This can be done by estimating the state parameters with the Kalman iltering prediction step (2) extended to the orecast time. Thus, the predicted state mean and covariance on the th step with the lead time o t are m F( t ) m, t P F( t ) sp F( t ) Q( t ), t T (7) where F ( t ) and Q ( t ) are deined as in (6). As illustrated below in Section 5, an optional parameter s > can be used to compensate the overestimation o the probability orecasts in case o decaying or transorming storm cells. It can be also shown that i the scaling is included in the prediction step o the Kalman ilter (2), it corresponds to the Kalman ilter with ading memory, where more weight is given to the recent observations with respect to the older observations (e.g. Simon 26). The inal nowcast or the storm cell is obtained by placing the initial storm to the location o the estimated centroid position. 3.3 Dealing with splitting and merging Important special cases in the convective storm tracing are the storm splitting and merging. A storm can split into two or more new storms, or a cluster o storms can merge to the same storm. Although the initial storm tracing detects splits and mergers, they are a complicated in the iltering, since the original Kalman ilter is not designed to deal with such a problem. In here, we propose the ollowing heuristic approach or dealing with the splitting and the merging. In case o the merging, the predicted state vectors and covariance matrices o the merging storms are simply combined with the areaweighted average o the merging storms. I a storm splits, each split successor storm inherits velocity components rom the predicted state estimate o the predecessor storm. However, the centroid position estimates are taen directly rom the measured centroid positions o the split storms. Each split storm cell also inherits the state covariance o the predecessor cell. 4. Ensemble nowcasting o convective storms In addition to the estimated state parameters, Kalman ilter provides the covariance matrix o the state parameters. Assuming that the state vector ollows a multidimensional Gaussian distribution, the state estimate and its covariance uniquely deine the probability density unction o the state. This probability density unction can be used or deriving dierent probabilistic nowcasting products or the traced convective storms, which is the main motivation o this paper. Here, we estimate the thunderstorm occurrence probability by generating ensembles with orecasted state mean mt and covariance P t.the ensembles are generated in the ollowing way. First, the orecasted state mean m t and covariance P t are calculated with (7). Second, we draw a random sample rom the distribution deined by m and P. t t Third, the orecasted storm at time +t is generated by placing the initial storm to the position coordinates o the random sample. To ease the ensemble production, the identiied storms are approximated with ellipses itted on the identiied cell. The ellipse itting is perormed through the principal component analysis (see e.g. Jollie 2); a covariance matrix C is estimated or the boundary points o the identiied cell, ater which we obtain ellipse main axes by calculating the principal component vectors o the covariance matrix. The length o each principal component vector equals to the variance o the cell boundary points along the corresponding principal axis. Here, the length o each ellipse axis is chosen such that it is two times the corresponding principal component.

4 Dance et al. (2) introduced an analogous method or deriving strie probability nowcasts or traced storm objects, i.e. the probability that a given location will be inluenced by a storm within the orecast time period, using constant predeined deviation parameters or the storm velocity. In our wor, both the speed and the location uncertainties are estimated recursively through Kalman ilter. Moreover, the uncertainty depends on the length o the history; the covariance matrix is larger with storms having shorter history and it converges toward a steady-state covariance as more evidence o the storm movement becomes available. 5. Case examples and algorithm evaluations The algorithm was tested with data obtained rom Finnish Meteorological Institute s eight Doppler C-band weather radars covering almost the whole Finland. Approximate constant altitude PPI images (pseudo-cappi) o 5 m altitude with 5 min temporal and x m spatial resolution were applied or the detection o storm objects. For the overall loo, Figure shows an example o the Kalman iltering tracing on Aug 9 25, where white polygons depict traced storm objects. Kalman iltered trac positions (green lines) are smoother than the non-iltered storm positions (red lines), resulting in a more consistent trac. Stable, smooth behavior o the trac improves the visualization o the storm tracs in end-user products. Moreover, the Kalman ilter based velocity estimates are consistent with the smoothed trac history, although the traced storms split and merge requently. Figure shows also the development o the orecasted position covariance (upper right image). The dashed circle shows the area, in which the orecasted centroid alls with the probability o 95%. As illustrated below, this uncertainty can be used or realistic storm probability orecasting. Lower right image o Figure shows an example o storm occurrence probability orecasting, where probability orecasts o lead times 2, 4 and 6 minutes are overlaid on each other. Fig. An example o the Kalman ilter based tracing on Aug (For discussion, see text) In order to veriy the nowcasting capabilities o the algorithm, it is necessary to evaluate the perormance using wellestablished sill metrics. In this wor, we applied commonly used probability detection (POD), alse-alarm ratio (FAR) and critical success index (CSI) (e.g. AMS glossary o meteorology 22).The veriication was perormed using data rom three cases: August 9 25, August 4 27 and June 2. Overall 26 complex storm tracs, i.e. tracs with splits and mergers (Dixon and Wiener 993), were identiied in the initial tracing. For the POD, FAR and CSI calculation, the resolution was downgraded to 5 by 5 m. A grid pixel was considered active i any point within the grid pixel was inluenced by an identiied storm. Consequently, the orecast was considered successul, i both orecasted and true grid pixels were active. A alse-alarm was assumed, i the orecasted grid pixel was active, but the true grid pixel was inactive. A ailure occurred, i the true grid pixel was active, but the orecasted grid pixel was inactive. Figure 2 shows POD, FAR and CSI values o the trac-by-trac analysis using the parameter settings r = 2 m, q t = 3.8 m /h. In the trac-by-trac analysis, we calculate sill scores only against storms, or which the history exceeds the lead time (Dixon and Wiener 993). This is reasonable as the storm tracing algorithms are developed to trac and orecast movement o existing storms, and they are not able deal with the temporal evolution or anticipate developing convection. Thus, the trac-by-trac analysis measures the nowcasting capability o the algorithm along the trac. The presented results are competent with orecasting methodologies introduced with other storm tracing algorithms. For example, Dixon and Wiener (993) calculated similar trac-by-trac results with TITAN. In terms o CSI, the results calculated with our algorithm are slightly better. However, it is important to note that the data set used or veriication is dierent in this study.

5 Probability o detection (POD) False alarm ratio (FAR) Critical success index (CSI) POD.4 FAR.4 CSI lead time (min) lead time (min) lead time (min) Fig. 2 Probability o detection, alse alarm ratio and critical success index calculated with the test data The probability algorithm was also validated. In order to provide realistic probability orecasts, the estimated probabilities should be in accordance with the true observed requencies o the orecasts. Thereore, we estimated the observed requencies or lead times 2-6 minutes with the ollowing procedure. The predicted probabilities were irst rounded to the closest integer. Thereater, the predicted probability ields were compared against the true observed storm cells. Since the probability orecasts were created by predicting the storms approximated with ellipses, also the observed storm were approximated with the same ellipse itting procedure (see Section 5). For each discrete probability value ranging between - 95 %, both the number o pixels under the area o the storms and the total number o pixels having the same probability value were counted. With a given probability value, the proportion o the pixels under the inluence o the storms to the total number o pixels estimates the observed requency. Figure 3.a shows the reliability diagrams or the probability values -95 %, where predicted probabilities are plotted against the observed requencies. The correspondence between the predicted probabilities and the observed requencies is good until 5 %, ater which the predicted probabilities start to overestimate the true probabilities. With shorter lead times, the correspondence is better. The overestimation o the large probability values is expected, since the algorithm does not explicitly deal with growth or decay o the storms. Individual convective cells have typically a lie time less than 3 minutes, and thereore many storms decay or undergo a signiicant deormation during the orecast time, which is not estimated by the algorithm. However, the additional uncertainty caused by the growth and decay eects can be compensated by increasing the actor s in (7). Figure 3.b presents the reliability diagrams orecasted with s = 2. The overestimation with the large probability values is reduced. On the other, large probabilities are estimated with ewer pixels, maing the estimation less reliable. Still, the additional scaling o the covariance matrix seems to improve the overall reliability. In the inal nowcasting application the probability can be urther calibrated according to the estimated reliability diagram. However, since maximum observed requency is around 8 %, also the maximum ris in the calibrated orecasts cannot exceed this value. Reliability plot q =.3 m 2 /(time step) 3, r = 2.5 m, s = a) Reliability plot q =.3 m 2 /(time step) 3, r = 2.5 m, s = b) Observed requency (%) Forecast time 2 min 3 min 45 min 6 min Observed requency (%) Forecast time 2 min 3 min 45 min 6 min Predicted probability (%) Predicted probability (%) Fig. 3 Reliability plots or the probability orecasts with varying orecast times. a) No covariance scaling (s = ), b) Increased scaling with s = 2

6 6. Discussion and uture directions This wor presented a Kalman iltering based algorithm or tracing convective storms. The algorithm applies the white noise acceleration model to produce a smoothed and consistent estimate or the position and velocity o a storm. When applied to the storm nowcasting, the algorithm produces competent perormance results in comparison to other tracing methods. In addition, the ilter can predict storm occurrence in conjunction with associated uncertainties o the orecast. These ris orecasts are consistent with observed requencies, veriying that the proposed methodology relects the true uncertainties o the orecasts. Kalman iltering provides a well-established and compact way to use the estimation o various storm parameters in the same algorithm. The model presented in this paper includes only storm centroid and velocity components, but it can be extended also with other state variables, such as storm area or echo top altitude, i their temporal behavior and relationship with other parameters can be ormulated with a stochastic state-space presentation. Moreover, the noise parameters o the Kalman ilter have physical quantities, which help us to choose reasonable parameterization or the ilter. In this paper, the noise parameters are selected heuristically, but i the noise behavior o the storm parameter can be estimated objectively, they can be directly included in the model. The Kalman iltering based storm tracing is also a step towards adaptive tracing, since the parameters o the ilter can be changed according to the varying noise levels. For example, in the case o attenuation, beam blocing etc., the measurement noise covariance o the ilter could have increased values. In addition, o dierent storm systems, lie mesoscale convective systems or supercell storms, can behave dierently. Thereore, also the process noise could be tuned adaptively in dierent storms. Kalman ilter provides a convenient way to combine uncertainty contributions rom various sources. Thereore, the iltering algorithm presented in this paper could utilize data rom other data sources. As an example, lightning data provides inormation on the location and movement o individual storms. This inormation could be incorporated as one o the state variables or the storm system. Acnowledgements This study has been supported by the Finnish Funding Agency or Technology and Innovation (Tees) within Heavy Rainall Processes (RAVAKE) project, Väisälä Foundation and Finnish Society o Automation. The participation o V.Chandrasear in this research is supported by the National Science Foundation and the Tees FiDiPro program. Reerences AMS Glossary o Meteorology, cited 22: Sill, [Available online at Bar-Shalom, Y. and Li, X.-R., 995: Estimation with Applications to Tracing and Navigation: theory, algorithms and sotware, New Yor: Wiley Dixon, M. J., and Wiener, G., 993: TITAN: Thunderstorm Identiication, Tracing, Analysis, and Nowcasting A radar-based methodology. J. Atmos. Oceanic Technol.,, Dance, S., E. Ebert, and D. Scurrah, 2: Thunderstorm strie probability nowcasting. J. Atmos. Oceanic Technol., 27, Gonzalez, R., and R. Woods, 28: Digital Image Processing. Prentice Hall Handwerer, J., 22: Cell tracing with TRACE3D--a new algorithm, Atmospheric Research, 6, Pages 5-34 Johnson, J.T., MacKeen, P.L., Witt, A., Mitchell, E.D., Stump, G.J., Eilts, M.D., and Thomas, K.W., 998: The Storm Cell Identiication and Tracing Algorithm: An enhanced WSR-88D algorithm. Wea. Forecasting, 3, Jollie, I.T., 2: Principal Component Analysis, Springer, New Yor Kalman, R, E., 96: A new approach to linear iltering and prediction problems. Transactions o ASME, Journal o Basic Engineering, 82, 55-8 Rossi, P., and Mäelä A., 28: A clustering-based tracing method or convective cell identiication and analysis. Proc. 5th European Con. on Radar Meteorology (ERAD 28), Helsini, Finland. Simon, D., 26: Optimal State Estimation: Kalman, H Ininity, and Nonlinear Approaches, Wiley-Interscience Zhou, Y., D. McLaughlin, and D. Entehabi, 26: Assessing the perormance o the ensemble Kalman ilter or land surace data assimilation. Mon. Wea. Rev., 34,