TESTS OF QUANTITATIVE PRECIPITATION ESTIMATES USING NATIONAL WEATHER SERVICE DUAL-POLARIZATION RADAR IN MISSOURI

Transcription

1 TESTS OF QUANTITATIVE PRECIPITATION ESTIMATES USING NATIONAL WEATHER SERVICE DUAL-POLARIZATION RADAR IN MISSOURI A Thesis Presented to the Faculty of the Graduate School at the University of Missouri In Partial Fulfillment of the Requirements for the Degree of Masters of Science by MALLORY LUMPE Dr. Neil Fox, Thesis Advisor December 205

2 The undersigned, appointed by the dean of the Graduate School, have examined the thesis entitled TESTS OF QUANTITATIVE PRECIPITATION ESTIMATES USING NATIONAL WEATHER SERVICE DUAL-POLARIZATION RADAR IN MISSOURI Presented by Mallory Lumpe, a candidate for the degree of master of science, and hereby certify that, in their opinion, it is worthy of acceptance. Associate Professor Neil Fox Assistant Professor Bohumil Svoma Professor Allen Thompson

3 ACKNOWLEDGEMENTS I would like to thank the professors of the meteorology department for their knowledge and guidance in the past five and a half years of my education at this institution. Without the teachings of Dr. Anthony Lupo, Dr. Patrick Market, Dr. Bohumil Svoma, and especially Dr. Neil Fox, I would never have made it this far. I would also like to thank Dr. Allen Thompson for being part of my committee and offering his expertise from an outside field. The assistance I received from Micheal Simpson in the compilation of this thesis was helpful beyond words, as was the production of mapping images from Joshua Kastman. My fellow graduate students, friends, and family have given me unwavering support and priceless advice over the years, and that deserves special recognition. I am forever grateful to everyone who has helped me along the way; I could never have gotten this far alone. Thank you. ii

4 TABLE OF CONTENTS Acknowledgements... ii List of Tables... v List of Figures... vii Abstract... ix Chapter. Introduction.... Purpose Objectives... 2 Chapter 2. Literature Review Radar Parameters Drop Size Distribution Why Dual-Polarization Algorithm Selection Quality Control Comparison Techniques Sources of Uncertainties Radar Miscalibration Precipitation Attenuation Ground Clutter & Anomalous Propagation Beam Blockage Variability of the Z-R Relationship Range Degradation Vertical Variability of the Precipitation System Vertical Air Motion & Precipitation Drift Temporal Sampling Errors Gauge Sampling Errors Summary of Uncertainties Chapter. Methodology Radar Sites Rain Gauge Sites Event Selection..... Time Conversion Data Downloading and Processing... 5 iii

5 .4. WDSS-II and WDSS-II Processing Overview Statistical Testing Statistical Testing Definitions Bias Mean Absolute Error Fractional Bias Fractional Absolute Difference Fractional Root Mean Square Error Fractional Standard Deviation Statistical Analysis Methodology Radar Algorithm Definitions Chapter 4. Statistical Results and Discussion Statistical Analysis: Rain Gauge Site Williamsburg Vandalia Bradford Farm Sanborn Field Capen Park Jefferson Farm Green Ridge Brunswick Versailles Mount Vernon Lamar Mountain Grove Statistical Analysis: Radar Location Best Equation Type Quality Control vs. Non-Quality Control Statistical Analysis: Function of Range Statistical Analysis: Significance Testing Chapter 5. Conclusions and Future Work... 0 Appendix... 5 References iv

6 TABLES Table. Coordinates of the radar locations Table.2 Coordinates of the rain gauge sites... 0 Table. Event dates and associated precipitation totals for the (a) St. Louis radar, (b) Kansas City radar, and (c) Springfield radar... 2 Table.4 Duration of each event date given in both local and universal time for the (a) St. Louis radar, (b) Kansas City radar, and (c) Springfield radar... 4 Table.5 Statistical tests performed on the data Table.6 Set of equations from Ryzhkov et al. (2005), separated by type Table.7 Groupings of equations and quality control schemes Table 4. Comparison of the values of the six statistical tests for each of the thirty-one equations for the Williamsburg data... 5 Table 4.2 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Vandalia data Table 4. Comparison of the values of the six statistical tests for each of the thirty-one equations for the Bradford Farm data... 6 Table 4.4 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Sanborn Field data Table 4.5 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Capen Park data... 7 Table 4.6 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Jefferson Farm data Table 4.7 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Green Ridge data... 8 v

7 Table 4.8 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Brunswick data Table 4.9 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Versailles data... 9 Table 4.0 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Mount Vernon data Table 4. Comparison of the values of the six statistical tests for each of the thirty-one equations for the Lamar data... 0 Table 4.2 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Mountain Grove data Table 4. Separation of the rain gauge sites by close range and far range Table 4.4 Significance testing results using a paired t-test that compares mean absolute error values from R(Z)-Convective and Eqn. 2b vi

8 FIGURES Figure. Map of the chosen rain gauge sites, NWS radar locations, radar range rings, and the domain used in the radar analysis... Figure 4. Results for the Williamsburg gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC Figure 4.2 Results for the Vandalia gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC Figure 4. Results for the Bradford Farm gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC Figure 4.4 Results for the Sanborn Field gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC Figure 4.5 Results for the Capen Park gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC Figure 4.6 Results for the Jefferson Farm gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC Figure 4.7 Results for the Green Ridge gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC Figure 4.8 Results for the Brunswick gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC Figure 4.9 Results for the Versailles gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC Figure 4.0 Results for the Mt. Vernon gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC Figure 4. Results for the Lamar gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC Figure 4.2 Results for the Mtn. Grove gauge site data: (a) Bias (b) Mean absolute error (c) Best equation type (d) Best QC scheme (e) QC vs. NON-QC... 0 vii

9 Figure 4. Best equation type per radar site for (a) KLSX (b) KEAX (c) KSGF and (d) All radar locations... 4 Figure 4.4 Benefits of QC over NON-QC & best type of QC scheme for (a) & (b) R(KDP) type equations (c) & (d) R(Z) type equations (e) & (f) R(KDP, ZDR) type equations (g) & (h) all equation types... 8 Figure 4.5 Close and far range results for (a) Best equation type for close range (b) Best QC scheme for close range (c) Best equation type for far range (d) Best QC scheme for far range Figure A. A.48 Bar graph results for the remaining four statistical tests for each rain gauge site... Appendix viii

10 ABSTRACT Flash flooding is the most common and widespread threat associated with severe weather. Therefore, it is essential for forecasters to be able to properly assess the risk of flash flooding in order to issue watches and warnings. The underestimation of rainfall accumulation by radar algorithms often leads to undiagnosed flash flooding, so it is necessary to determine which type of quantitative precipitation estimation (QPE) equation best assesses the actual amount of rain that has fallen at a given location (Ryzhkov et al. 2005). By comparing the radar estimated rainfall to the accumulated precipitation measured by rain gauges, the bias and error of the QPE algorithms can be assessed. In the following study, these measurements will be compared for significant rainfall events that occurred across the state of Missouri in 204. The data from twelve individual rain gauge sites, which are considered to provide the ground truth rainfall quantities (Kitchen and Blackall, 992), are measured against the estimated rainfall calculated by the three National Weather Service radars. Also included is an analysis of whether gauge distance from the radar location has an effect on the error and bias. Various quality control (QC) methods are applied to the radar parameters in order to determine whether or not they enhance the outcomes of the statistical testing applied to the radar data. The results show that R(Z, ZDR) type equations produce the best data, as they give error and bias calculations closest to zero. ix

11 Chapter : Introduction According to the National Severe Storms Laboratory (NSSL), the most common and widespread natural disaster that occurs in the United States is flooding. It can happen in any state or U.S. territory at any time of the year. Flooding is also the most costly of all natural disasters, with an annual expense of nearly $5 million. Furthermore, floods cause more fatalities in the U.S. every year than tornadoes, hurricanes, or lightning. Of the numerous types of floods, flash flooding is the most dangerous as it can occur with little or no prior warning. Better quantitative precipitation estimation (QPE) is necessary to improve flood forecasting (Ryzhkov et al. 2005). This study analyzes point-measurements of rainfall from a network of rain gauges across the state of Missouri in comparison with areal rainfall calculations from three National Weather Service radars. Because gauges provide single-point measurements and radar estimates represent a much larger volume of space, there will be significant differences amongst the data sets (Bringi et al. 20). It is largely accepted that large uncertainties exist for radar estimated rainfall totals (Villarini and Krajewski, 2009). However, in the United States, the use of operational weather radars to quantitatively estimate precipitation totals is much more economical than implementing a super-dense network of rain gauge locations across the entire country.

12 . Purpose The work presented in this study is important because it will attempt to select the appropriate radar rainfall estimation algorithm, with the smallest error and bias, to limit the amount of over- or underestimation of rainfall in a given location. Flash flood forecasting requires reliable measurements of rainfall accumulations, regardless of storm intensity, with high spatial resolution (Ryzhkov et al. 2005). The expectation is that polarimetric radar variables will produce more accurate rainfall estimation than the conventional, single polarization reflectivity parameter (Cifelli et al. 200). The underestimation of rainfall could lead to an undiagnosed flash flood, whereas an overestimation could lead to forecasting a false alarm. More accurate estimation of rainfall accumulation over a specific location will improve warning time for flash flood statements as well as correctly forecasting the type of flooding that may occur..2 Objectives The main objective of this study is to answer the following question: Which radar-estimated rainfall algorithm is best? This inquiry is simply stated, but requires a complex analysis to arrive at a conclusion. Meteorological, climatic, and physiographic conditions all factor into the performance of the optimal radar algorithm (Prat and Barros, 2009). By comparing rain gauge measurements of rainfall, which are considered to be the ground truth (Kitchen and Blackall, 992), to the more uncertain calculations derived from radar parameters, the accuracy of these radar algorithms may be assessed. Both 2

13 conventional, or single-polarization (e.g., reflectivity), and polarimetric, or dualpolarization, (e.g., differential reflectivity and specific differential phase) radar parameters are investigated. The rainfall events analyzed in this study all occur in the state of Missouri in the year 204. The rain gauge stations utilized occur at a wide range of distances from the given radar locations, up to 50 km, even though the typical range of an S-band NWS radar has a maximum of 20 km (Fulton et al. 998). Therefore, the distance a gauge site is from its corresponding radar will also be considered. This spatial range may have an effect on appropriate algorithm selection because as distance from the radar increases, beam height and width also increase and this has significant implications on the areal volume measurement of the atmosphere. One objective for this study is to determine if there is a single type of radar algorithm that produces the most accurate rainfall results at various ranges from the radar, at different radar locations, and at numerous rain gauge sites across the state of Missouri. Furthermore, whether or not data quality control has any effect on the rainfall estimation will be analyzed. The hypothesis is that a type of algorithm using a polarimetric variable, or a combination of polarimetric variables, will produce the most accurate rainfall estimation.

14 Chapter 2: Literature Review In order to accomplish the objectives of this research and show support by previous studies, a thorough review of existing literature was completed. The first section, 2., supplies a basic overview of the radar parameters to be analyzed in this thesis. The second section gives a background on how the drop size distribution of a radar sample volume affects the radar parameters performance and thus aids in the algorithm selection. Section 2. defends the use of polarimetric radar data over the conventional singular polarized data. Next, a general overview of the performance of a variety of radar rainfall retrieval algorithms is analyzed, and suggestions are given for when it is best to use a certain type of equation. Here, Sub-Section 2.4. describes the need for qualitycontrolled data to be used in the selected algorithms in order to provide clean inputs for the equations. Techniques for how to compare the radar data against other methodologies are described in Section 2.5, namely the use of rain gauge comparison. Finally, Section 2.6 delves into the many sources of uncertainties surrounding single- and dual-polarization radar data as well as gauge sampling errors. 2. Radar Parameters Remote sensing of the atmosphere can be done by a number of methods. This study focuses on the process of active remote sensing via weather radars; 4

15 specifically, dual-polarized weather radars operated by the National Weather Service. Essentially, the radar works by emitting a short pulse of microwave energy and subsequently measuring the power scattered back by particles in the beam scanning volume (Villarini and Krajewski, 2009). In the case of dualpolarized radars, this pulse is discharged in the form of both horizontally and vertically polarized electromagnetic waves. These perpendicular waves encounter a particle in the atmosphere and are transmitted back to the radar where a receiver detects the two wave dimensions separately. Before the polarimetric upgrade of the radar network, however, there was the installation of over 60 S-band Weather Surveillance Radar 988 Doppler (WSR-88D) radars nationwide between the years of by the NEXRAD (Next Generation Weather Radar) program (Fulton et al. 998). This network of operational radars produced standard products such as reflectivity, Doppler velocity, and spectrum width, as well as value-added hydrometeorological products via automated algorithms that transform the input reflectivity factor (Z) into rainfall accumulations (Klazura and Imy, 99; Seo et al. 2000). In this network, the only input value to the rainfall algorithm was calibrated, quality controlled horizontal reflectivity. The upgrade to dual-polarization, or simply dual-pol, in the early 200s led to the formation of a number of additional radar products that can more accurately assess hydrometeor type, size, and rainfall accumulation. The dualpol parameters focused on in this study include differential reflectivity (ZDR) and specific differential phase (KDP). The differential reflectivity is based on the ratio 5

16 of the horizontally polarized reflectivity to the vertically polarized (Seliga and Bringi, 976). Therefore, because raindrops become increasingly oblate as they grow, this parameter is a good indicator of drop size (Illingworth and Blackman, 200). Positive values of differential reflectivity indicate that the hydrometeors have a larger horizontal axis than vertical, whereas negative values indicate vertically oriented hydrometeors. ZDR is measured in units of decibels (db) and has a typical range of -2 to 8 db. The specific differential phase is the rate of change of the differential phase, ɸDP, with range, where ɸDP is the difference in the phase of the horizontally polarized signal return (ɸH) and vertically polarized return (ɸV). This parameter relies on the lag in signal return between the horizontally and vertically oriented beams as they travel through an area of heavy rainfall with large, oblate drops (Sachidananda and Zrnic, 987). Because specific differential phase depends upon the axis ratio of the hydrometeor, it is a good indicator of drop shape. For example, similarly to ZDR, positive KDP values are indicative of horizontally oriented particles and negative values indicate vertically oriented particles. Additionally, because hail tumbles as it falls, the differential phase ɸDP measures close to 0 km-, and thus hail has minimal impact on the measurement of KDP. KDP is measured in degrees per kilometer and has a typical range of -2 to 7 km -. However, even though KDP is not sensitive to hail contamination, it may still be affected by variations in drop size spectra (Illingworth and Blackman, 200). 6

17 2.2 Drop Size Distribution In order for the polarimetric parameters to outperform the conventional horizontally oriented reflectivity factor, the correct assumptions on the drop size distribution must be made (Illingworth and Blackman, 200). This drop size distribution, or DSD, is based on hydrometeor type, storm type, latitude, season, and many other parameters. For example, in convective rain, there is a clear trend for the mean volume diameter to increase with increasing rainfall amount (Testud et al. 200). In other words, in periods of heavy convective rainfall intensity, there are characteristically fewer small raindrops present. The oblateness of a drop is key for measuring the drop size in order to provide a better representation of the DSD and thus more accurate rainfall estimate from the selected radar algorithm. In short, DSD knowledge is very important for radar rainfall estimation because it determines what algorithm should be used to relate radar parameters to rainfall. Drop diameter is especially important for measuring single-polarization reflectivity (Z), where the sum of the diameter (D) to the sixth power is used to measure Z. Z = N(D)D 6 dd () In Schuur et al. (200), it is shown that there is great DSD variability not only between one rainfall event and another, but also within a single storm. The dual- polarized parameters of ZDR and KDP are less susceptible to variation in the drop size distribution than conventional reflectivity, but they are still somewhat affected by the DSD variability. For example, DSDs containing an uncommonly large number of small (large) drops will typically result in an under (over) 7

18 estimation of the rainfall rate. Therefore, it is important to understand how the drop size distribution will vary naturally in rainfall events (Ryzhkov et al. 2000). In addition to the meteorological, climatic, and physiographic conditions that affect DSD, it is also important to have information on the microphysical processes present in the rain event in question. These processes include details on break up, coalescence, accretion, evaporation, and condensation occurring in the storm (Prat and Barros, 2009). 2. Why Dual-Polarization One aim of this study is to analyze the benefits of polarimetric radar data versus reflectivity inputs alone. Fulton et al. (998) found that while it was possible to use the standard Z-R relationship in order to derive rainfall products in real time, this singular algorithm was not robust enough to be used everywhere and at any time. Numerous sources point to the improvement of dual-pol data over single polarization, and many strive for the same end goal for the data output: improvement of quantitative precipitation estimation, or QPE, by the radar rainfall algorithm (Ryzhkov et al. 2005; Zrnic and Ryzhkov, 996; Cifelli et al. 200; Jameson, 99). The polarimetric parameters, more specifically ZDR and KDP, show significant improvement in the quality of the radar data. For example, they lead to more accurate differentiation between hydrometeors and non-hydrometeors, which is important for this study in order to correctly distinguish non-liquid echoes from hailstones, snow, birds, bugs, and other non-meteorological targets. Furthermore, when these dual-pol products 8

19 are utilized for rainfall estimation, they are not as affected by drop size distribution variability as the conventional Z-R relationships. As previously mentioned, ZDR is useful for measuring the median diameter when analyzing drop size. As raindrops become larger, they tend to grow horizontally, thus leading to positive values of ZDR. Therefore, differential reflectivity is in turn indicative of both drop shape (i.e., longer horizontal axis than vertical axis) as well as drop size (Illingworth and Blackman, 200). Specific differential phase also provides insight to the shape of the hydrometeors present in the radar volume scan; positive values of KDP indicate horizontally oriented targets, and negative KDP points to a vertical orientation. Unlike reflectivity, which is largely affected by the presence of hail, KDP shows a value near zero for the tumbling hailstones and is thus relatively unaffected by the presence of hail in a rainfall event. KDP is also almost immune to miscalibration, precipitation attenuation, and partial beam blockage (Ryzhkov and Zrnic, 996). Reflectivity, on the other hand, is highly susceptible to variability in the drop size spectra. Z is dependent on the DSD and the size of the drops to the sixth power (Marshall et al. 947); therefore, if there is large variation of values in the drop spectra, the calculated Z has the potential to be largely inaccurate when used as input for the reflectivity-rainfall algorithm. However, this study will compare the performance of a number of different Z-R relationships based on rainfall type to algorithms utilizing polarimetric data input to test which equation type is best at a long range from the radar site. 9

20 2.4 Algorithm Selection When the general form of the radar-rainfall equation was first proposed by Marshall et al. (947), the only input variable was the reflectivity factor, Z. However, as mentioned repeatedly throughout Section 2.6, there are many sources of uncertainty associated with this single-polarization parameter. This relationship is extremely transient and highly dependent upon the estimated drop size distribution of the sample volume. This single algorithm is simply not robust enough to be used at any place and at any time (Fulton et al. 998). It is therefore necessary to utilize the more advanced polarimetric radar parameters to more accurately assess the rainfall accumulation totals. As stated in Ryzhkov et al. (2005), the primary benefit that comes from using dual-polarization radar is the improvement in quantitative precipitation estimation. One study in particular, JPOLE (Joint Polarization Experiment), had the main objective of developing the optimal QPE algorithm for operational weather radars (Ryzhkov et al. 2005). Different radar algorithms perform better based on different parameters that vary the drop size distribution such as season, location, storm type, et cetera (see Section 2.2 for in-depth description). Furthermore, DSD can vary not only from storm to storm, but even within the same storm itself. The polarimetric variables utilized in this study are differential reflectivity and specific differential phase, which are good for measuring drop size and shape as well as being relatively immune to many sources of uncertainty that affect the conventional reflectivity parameter. Ryzhkov et al. (2005) tested a set of 6 equations based on various assemblages 0

21 of these parameters with varying constant coefficients and exponents based on a given drop size distribution, as suggested by previous studies. There is an indepth discussion of these equations in Section.6. From this group of radar algorithms, it was shown that for a given reflectivity parameter, large ZDR values correspond to many large drops in a given sample volume, while small ZDR relates to a large number of small drops. It was also concluded that the radar underestimated higher rainrates, which typically correspond to convective rain. However, the presence of hail was still found to be a problem even when KDP equations were used. At distances less than 90 kilometers from the radar site, where DSD variability and possible hail contamination are dominant factors affecting the accuracy of rainfall estimation, the dual-pol algorithms were found to outperform the conventional Z-R relationships. One goal of this thesis study will be to discover if this holds true at longer ranges from the radar, up to even 50 km. The synthetic radar equation using a composite of reflectivity, differential reflectivity, and specific differential phase was found to be the optimal algorithm to use with respect to calibration errors, DSD variations, drop-shape uncertainty, and possible hail presence as it reduced the root mean square error (RMSE) by factors of.7 for point (gauge) measurements and.7 for areal (radar) estimations, respectively (Ryzhkov et al. 2005). Another study by Bringi et al. (20) considered a variety of estimators including the conventional Z-R relationship with and without attenuation correction, a composite estimator based on R(Z), R(Z, ZDR), and R(KDP), and

22 R(KDP) alone. This research found the composite estimator that uses a combination of rainfall equations to be the most accurate at all thresholds, but especially at rainrates higher than mm hr -. For lower rainrates (i.e., less than mm hr - ), it was concluded that R(Z) after attenuation correction produced nearly the same results as the composite estimator, but it had higher mean absolute error and bias as rainfall intensity increased. The Z-R relationships without correction for attenuation produced the worst results of all the algorithms, as they underestimated the true rainfall amounts by 25-40%. It can therefore be seen that correction for rainfall attenuation is necessary (Bringi et al. 200). Even though it is widely accepted that because dual-pol algorithms give more information about the raindrop size and shape they have advantages over the conventional reflectivity factor alone, it is still necessary to answer the important question of what is the best algorithm and when should it be used (Cifelli et al. 200). Does R(Z, ZDR, KDP) always outperform R(Z)? When should KDP be used over Z or ZDR? The study by Cifelli et al. (200) also proposed the use of a new rainfall algorithm that utilizes hydrometeor identification (HID) to indicate which rainfall algorithm should be used in certain situations. The classifications of hydrometeor analyzed include all rain, mixed precipitation, and all ice. Once the hydrometeor classification was assessed, the rainfall estimates by the subsequently selected radar algorithm were calculated. Furthermore, it was concluded that dual-polarization observations could be used in all three steps of the QPE process in order to improve rainfall estimation. 2

23 These three steps include preprocessing quality control and data enhancement, classification of hydrometeor types, and the final quantification of rainfall estimation. A variety of studies examining the error structure of dual-pol parameters used for QPE show that their efficiency varies with regard to rainfall intensity. In light rain, for example, it was shown that R(Z, ZDR) was no better than R(Z) (Chandrasekar and Bringi, 988). Studies by Sachidananda and Zrnic (987) and Chandrasekar et al. (990; 99) concluded that KDP is noisy when R is small, but it still performs better than R(Z) and R(ZDR) when R is large. It has also been found that composite estimators such as R(KDP, ZDR) outperform R(Z), R(Z, ZDR), and R(KDP) at moderate to heavy rainrates, but it is often necessary to smooth out, or quality control, the KDP and ZDR parameters (Jameson, 99; Ryzhkov and Zrnic, 995). Further evidence to the improvement in quantitative precipitation estimation via use of dual-pol radar data comes from Silvestro et al. (2009). Their proposed Radar Intensity Multiparameter Estimator, or RIME, showed improved R estimation when using a combination of Z, ZDR, and KDP as algorithm input. This study found agreement with the statement that it is important to analyze precipitation type, climatic conditions, and the topographical situation of a given environment in order to choose the appropriate radar algorithm. It is often necessary to use ZDR jointly with Z and KDP to allow for the improvement in QPE because differential reflectivity is key in differentiating between types of rain and associated drop size distributions, as it is insensitive to total drop

24 concentration (Schuur et al. 200). Simply stated, not all QPE algorithms are created equally Quality Control The term quality control, as mentioned previously, refers to when Z and ZDR are corrected for attenuation using heavily filtered ɸDP and simple relations, for example, ΔZ (db) = 0.04 ɸDP (deg) and ΔZDR (db) = ɸDP (deg) (Ryzhkov and Zrnic, 995). Quality control, or QC, also refers to when ground clutter, biological echoes, et cetera are removed from the radar data. QC can be carried out by a variety of methods. In a validation study by Lakshmanan et al. (2007), a number of quality control methods were analyzed for a variety of radar data scenarios. Reflectivity data with quality control is necessary in order to accurately forecast precipitation, especially when its main objective is to be used as input for algorithms that require clean data. Clean data refers to radar data that contains hydrometeors only, without the presence of ground clutter, anomalous propagation (anaprop), biological targets, or electronic signals. The quality control methods are designed to eliminate most of the non-precipitation echoes while retaining the useful precipitation signatures. These quality control methods thus help to reduce error in quantitative precipitation estimation (Fulton et al. 998; Kessinger et al. 200). Before the availability of dual-polarization radar data, the following studies were done on different quality control methods. The work done by Lakshmanan et al. (2007) focuses on the validation of a number of QC methods, 4

25 namely the Radar Echo Classifier (REC) which is used by the National Weather Service (Kessinger et al. 2002) and the Quality Control Neural Network (QCNN) (Lakshmanan et al. 200), utilized by the Warning Decision Support System Integrated Information (WDSS-II) (Hondl, 2002). The purpose of the REC is to improve radar-derived QPE by removing contaminants, especially anaprop and ground clutter. To do this, REC determines which echoes are meteorological based on a fuzzy logic scheme and subsequently removes those targets that are non-weather related. This methodology relies on reflectivity, radial velocity, and spectrum width to classify the different radar echoes and quality control the radar data out to 20 kilometers. The QCNN also uses the variables of reflectivity, radial velocity, and spectrum width, as well as the horizontal reflectivity structure, SPIN and SIGN (Kessinger et al. 200), and echo size to determine which targets are and are not precipitation. SPIN and SIGN are two additional reflectivity features added to the REC and indicate the number of inflection points within the reflectivity difference field and the mean sign of the reflectivity parameter, respectively. The QCNN variables are then used to assign a confidence rating from 0-, with one being the most confident, to signify that the target is indeed precipitation. If the assigned confidence rating is less than 0.4, the target is considered non-meteorological and is removed from the data field. The results of this study found the quality control method utilized by WDSS-II, QCNN, to outperform the REC at a threshold of 0 dbz and above. While it struggles to remove all biological targets, it does well with retaining all 5

26 necessary meteorological targets. QCNN also proved to be better at correctly assessing high reflectivity values. The REC, on the other hand, struggled to correctly identify high Z returns, except when this large Z value was caused by anaprop. This is to be expected, as REC was specifically designed to remove anaprop and ground clutter, not biological or electronic returns. The study concludes that some method of quality control is necessary to yield the most accurate results, especially when the radar data requires clean input values. The QCNN showed to be the best performer for two reasons: it removed the nonprecipitation echoes, but also retained the relevant hydrometeorological data; just as the optimal QC method should behave. Once dual-pol data were made available, the RREC scheme became another viable quality control method for enhancing radar data. RREC, or radar rainrate echo classification, uses a decision tree method to determine the best rainfall algorithm to use based on precipitation type. The precipitation type is determined by echo classification using various combinations of Z, ZDR, RHOHV, and KDP products, as well as texture parameters DSMZ, DSDZ, and DSDP. Once the precipitation type is determined by the decision tree, then a suitable R(Z), R(KDP), R(Z, ZDR), or R(ZDR, KDP) algorithm can be assigned to the event. Thus, if the hydrometeor classification is not accurate, the RREC decision tree method will produce largely imprecise results for the rainfall accumulation. This is especially true when estimating rainfall at great distances from the radar site. 6

27 2.5 Comparison Techniques In order to correct for the presence of bias, miscalibration errors, or any of the other sources of uncertainty surrounding QPE radar data (see Section 2.6 for an in-depth discussion), the data must be compared against other sources that examine rainfall accumulation. This study focuses on the use of rain gauge tipping bucket measurements, as is common when attempting to evaluate the performance of a radar in regards to its use as an operational tool for QPE (Silvestro et al. 2009). It is usually assumed that rain gauge data are the ground truth for the actual amount of rain that fell in an area (Bringi et al. 20). However, there are still errors associated with gauge representation. Because rain gauges represent rainfall accumulation for a specific point, and radar algorithms look at an areal measurement, there will be inherent differences between the two measurements (Kitchen and Blackall, 992). This difference between the gauge point-measurement and volume-averaged rainfall calculated by the radar needs to be acknowledged so that all of the errors are not pinned solely on the radar s rainfall estimation. To correct for the differences in point versus areal measurements, two solutions are proposed. The first, given by Wood et al. (2000), suggests using a super-dense gauge network in the space of a typical radar pixel. This allows for the direct estimation of the difference between the rainfall measured by the gauge and that calculated by the radar. The second solution would be to utilize a spatial correlation function given by Eqn. 2, where Rg represents the rainfall of 7

28 the point gauge measurement, Ra is the true area-averaged rainfall, and var is the variance. VRF = var(r g R a ) var(r g ) (2) The study conducted by Bringi et al. (20) used a combination of the two suggestions and found that there is a higher variance between the rainfall measurements between the rain gauge and radar for higher intensity rain rates. In general, it has been noted that improved understanding of the radar-rainfall uncertainties and the rainfall process itself is largely dependent on the availability of dense and high quality rain gauge networks (Villarini and Krajewski, 2009). The denser the network, the fewer gaps in data there will be. A denser gauge network aids in filling gaps among the gauge sites themselves, and also serves to provide data where no radar measurements are available. 2.6 Sources of Uncertainties It is largely accepted that as rainfall intensity increases, the accuracy of dual-pol radar rainfall products increases as well. This is because the signatures of ZDR and ɸDP are more pronounced, which allows the rainrate algorithms to self-adjust due to DSD variability, which further enables a variety of storm types and extreme events to be captured (Bringi and Chandrasekar, 200). In the study conducted by Bringi et al. (20), it was found that more error, around 70%, exists in lower rainrates (i.e., less than mm hr - ) as compared to only 0% error in higher intensity events (i.e., greater than mm hr - ). However, it is 8

29 also well acknowledged that there are many sources of uncertainties associated with radar-based estimates of rainfall. These uncertainties stem from error sources such as parameter estimation, the observational system and measurement principles, and not fully understood physical processes (Villarini and Krajewski, 2009). Furthermore, if these uncertain radar data are used as input for hydrometeorological models or initial conditions in weather forecasting models, the associated errors also propagate into the results. Thus, the uncertainties need to be well understood in order to sufficiently analyze the results. The discussion on uncertainties can begin with the plethora of error sources surrounding single polarization radars, some of which extend to polarimetric radars as well. These uncertainties stem from radar miscalibration, attenuation, ground clutter and anomalous propagation, beam blockage, variability of Z-R relationship, range degradation, vertical variability of the precipitation system, vertical air motion and precipitation drift, and temporal sampling errors (Villarini and Krajewski, 2009). Some of these uncertainty sources, however, do not apply to polarimetric radar data parameters. The dualpol variables investigated in this study are largely unaffected by miscalibration, precipitation attenuation, and partial beam blockage (Zrnic and Ryzhkov, 996). However, in the case of the Z-R relationships investigated here, it is still very important to correct for rainfall attenuation and beam blockage (Bringi et al. 200a; Keenan et al. 998). A more in-depth discussion on these sources of error as outlined by Villarini and Krajewski (2009) follows. 9

30 2.6. Radar Miscalibration While KDP is essentially immune to miscalibration of the radar, this uncertainty source can still play a part in rainfall estimation. For a wellcalibrated system, the radar constant is known with an accuracy of db (Austin 987). In Eqn., P r is the average power received from a volume of rain-filled atmosphere at the range, r; C is the radar constant known within db accuracy; k is the fractional reduction in the signal attenuation along the path of propagation; and Z is the radar reflectivity factor. P r = CkZ r 2 () Miscalibration is associated with a change in this constant C over time via the deterioration of components of the physical radar, such as the transmitter, receiver, and antenna. Thermal effects also play a role in radar system miscalibration. These inaccurate C values are difficult to detect, can vary due to thermal effects or drift, and will not self-correct; they persist until the system is recalibrated. This C-inaccuracy can be detected either by examining the same storm from a number of overlapping radars, or by comparing the radar rainfall estimates against measurements from another device (e.g., disdrometer data). In short, accuracy of calibration is essential for all functions of effective radar use (Ulbrich and Lee, 999) Precipitation Attenuation Precipitation attenuation is also a source of error, but it is relatively unimportant for KDP until the signal is completely unattenuated, which only 20

31 happens at the S-band for very large hail. However, this reduction in power as the electromagnetic radiation passes through a precipitation field is still important for the non-polarimetric portion of this study. Gases in the atmosphere, clouds, and precipitation can cause attenuation (Rinehart 2004). In this case, the focus will be on attenuation due to rainfall Ground Clutter & Anomalous Propagation The ability to differentiate between meteorological and nonmeteorological targets is key for operational radar applications such as flood forecasting. Ground clutter is caused by scattering in the radar scan volume due to trees, buildings, or other fixed objects that obstruct the beam (Meischner 200). Anomalous propagation, or anaprop, is present when there is a decrease in refractivity with height near the surface, which in turn causes the radar beam to come in contact with the earth s surface at distances far from the radar and return non-meteorological echoes on the radar imagery (Bean and Dutton, 968). These qualities can be identified and removed on radar imagery by finding near-zero Doppler velocity, lack of vertical discontinuity, and great spatial reflectivity variability. However, removing the ground clutter and anaprop discontinuities may result in the underestimation of rainfall by up to 20% (Serafin and Wilson, 2000). 2

32 2.6.4 Beam Blockage Radar beam blockage and partial beam blockage occurs due to obstructions on the beam s path, some of which are unavoidable (i.e., mountains) (Germann and Joss, 2004). These mountainous blockages are especially problematic in the limitation of QPE in the western portion of the United States where radar coverage is already very limited due to topographical conditions (Maddox et al. 2002). Beam blockage is more common at the lower elevation scan angles that, unfortunately, are the most useful for estimating rainfall via radar data. One solution to this dilemma could be to place the radar site on top of the problematic mountain and use negative elevation angles (Brown et al. 2002; Wood et al. 200). The selection of tilt angle to be used by WSR-88D radars is based upon two conditions first, the bottom of the selected angle must be at least 50 meters from the ground and second, the terrain blockage must be less than 50% of the radar beam (Fulton et al. 998) Variability of the Z-R Relationship Both reflectivity (Z) and rainfall rate (R) depend on the drop size distribution, as demonstrated by Eqn. (definition of Z) from Section 2.2 and Eqn. 4 shown here: R = π 6 N(D) D V t (D)dD (4) 22

33 In Eqn. 4 for measuring the rainfall rate, N represents the number of drops, D the diameter, and Vt is the drop terminal velocity in cms-. Eqn. 5 shows the definition for another common unit of reflectivity measurement, dbz. Z = 0log0( Z[mm6 /m ] [mm 6 /m ] ) (5) A power-law relation, more commonly known as the Z-R equation (Eqn. 6), can then compare Z and R. Z = ar b (6) As described in Section 2.2, the drop size distribution depends on the prevailing rainfall process and varies geographically, seasonally, from one storm to another, and even within the same storm. As such, Eqn. 6 can adapt the constants a and b to reflect these variations in the drop size distribution dependencies. It has been widely shown that this DSD variability unavoidably affects the rainfall estimates by radar in a significant manner (Villarini and Krajewski 2009). While polarimetric radar variables are not as sensitive to DSD variation as single polarization reflectivity, this source of uncertainty does still impact the QPE from the radar algorithms. It is necessary to have a large data sample available for comparative purposes in order to produce significant results from Z-R relation analysis Range Degradation Range degradation is one of the biggest problems in the estimation of radar rainfall estimation, and is a problem for single- and dual-polarization radar 2

34 data alike. It is especially problematic in the underestimation of rainfall at a large range from the radar (Kitchen and Jackson, 99; Neyman 996; Smith et al. 996; Meischner et al. 997). As the range of the beam from the radar site increases, both the beam height and width increases. This beam broadening causes overshooting of low clouds and thus low altitude storm systems, and also partial beam filling. Partial beam filling is a problem due to radar beam geometry, i.e., the increase in beam volume with range. Because of this volume increase, small but intense features of the rain system are averaged out over a larger volume and there is a subsequent bias in the measured reflectivity (Tees and Austin, 99; Sanchez-Diezma et al. 2000). This larger beam volume is also more likely to include a number of types of hydrometeors instead of rain alone, which also affects the reflectivity estimation. The issue of overshooting of the radar beam is due to the beam s increase in elevation at greater ranges and is especially problematic in the cold season. Winter storms often have shallow development and are dominated by low altitude precipitation, which are often overshot by the radar beam at a far range. Chumchean et al. (200; 2004) suggested the application of a scale transformation function as a possible solution to correct for partial beam filling and increasing sample volume with range Vertical Variability of the Precipitation System Radar measures the reflectivity factor at a certain height above the ground, but not at the surface itself. For a certain elevation angle of the radar 24

35 beam, this height increases with increasing range. Therefore, partial beam filling and overshooting of low clouds are not the only issues with beam broadening there also needs to be consideration of the non-homogeneity of the vertical structure of the reflectivity signature (Villarini and Krajewski, 2009). This is mainly dependent upon the dielectric constant, which varies based on hydrometeor type. For example, this constant has a value of 0.97 for water and 0.97 for ice and therefore must be accounted for to get the most accurate reflectivity returns (Rinehart 2004). There is also variability in the vertical storm structure due to evaporation, collision, coalescence, and break-up of raindrops aloft. This is also the point at which the bright band becomes important. The bright band occurs just below the 0 C isotherm where ice melts and causes an increase in reflectivity return on the radar imagery as compared to the ice above and rain below this point (Austin and Bemis, 950). It is therefore important to acknowledge where this region occurs in order to correct for it in data analysis. Like drop size distribution, the height, thickness, shape, and strength of the bright band varies from storm to storm and even within a singular storm Vertical Air Motion & Precipitation Drift Vertical motion of the air is highly variable both temporally and spatially and is also independent of range from the radar. The updraft velocity has a decreasing effect on the fall speed of a raindrop, while a downdraft would increase the drop s fall speed as compared to what the same raindrop would 25

36 experience in still air. Furthermore, DSD can also be affected by the presence of an updraft as this upward vertical motion would keep the smaller raindrops aloft and thus also impact the Z-R relation (Smith, 990; Battan, 976). Another source of uncertainty comes from horizontal advections that impact the fall path of raindrops. These sideways motions cause the drops to fall in a non-straight vertical line, and while this is not particularly important at hourly time scales, it becomes increasing significant at smaller time scales. In terms of rainfall rate estimation, precipitation drift reduces the frequency of the occurrence of small raindrops, which also has a direct impact on drop size distribution (Seed et al. 996). Studies by Collier (999) and Lack and Fox (2005; 2007) demonstrated that the impact of precipitation drift is especially noteworthy at high spatial resolution Temporal Sampling Errors The error resulting from temporal gaps in the rainfall observations is more significant for satellite-based QPE, but is still applicable for radar estimated rainfall accumulations. Continuous sampling has lower associated error than samples taken every 0 minutes (Wilson and Brandes, 979). Furthermore, the temporal sampling errors are larger for scattered rainfall events rather than more uniform, widespread ones (Jordan et al. 2000). 26

37 2.6.0 Gauge Sampling Errors It is not only radar errors that have an impact on this study, however. Gauge uncertainties also play a part in the error of comparison data. Gauge errors stem from malfunctions of the gauge itself, site blockages, and timing errors of the tipping bucket to name a few (Bringi et al. 20). Uncertainties in the gauge network also stem from an underwhelming number of gauge sites over a large area. In fact, Villarini and Krajewski (2009) state that an improved understanding of the radar estimated rainfall uncertainties is unavoidably dependent on the availability of dense and high quality rain gauge networks. Therefore, in order to more accurately perform radar and rain gauge measurement comparisons of QPE, better temporal and spatial resolution for both radar coverage and the rain gauge network is necessary Summary of Uncertainties Although various sources and types of error have been discussed thus far, the use of polarimetric radar data is expected to reduce a majority of the uncertainties. However, the range effects such as variations in vertical profiles, beam broadening, beam filling, and mixed precipitation types in the radar sample volume remain an issue in the collection and processing of radar data. The variations in drop size distribution also continue to affect various dualpolarization radar parameters. Therefore, testing dual-pol data with adjustments to DSD variations at multiple ranges from the radar site, especially at long ranges, makes this work novel and innovative. 27

38 Chapter : Methodology The primary tools for the research behind this thesis include the Warning Decisions Support System Integrated Information (WDSS-II) data processing, MatLab coding, and Microsoft Excel computations. These technologies and their processes are explained in the following sections, as is the reasoning behind the chosen radar and gauge site locations and rainfall event selections.. Radar Sites The three radar sites chosen for this research are those that give the best areal coverage of the state of Missouri. The radars used are part of the National Weather Service WSR-88D system, and are S-Band polarimetric instruments that operate on a wavelength of 0.7 cm. The radars are located in St. Louis, MO (KLSX), Kansas City, MO (KEAX), and Springfield, MO (KSGF). In this study, the raw polar radar data are transformed to a limited regular Cartesian grid around the radar. The original area coverage of the St. Louis and Kansas City radars was not sufficient to capture their associated rain gauge sites, so the coordinates of latitude and longitude for the data collection via WDSS-II were shifted. The original coordinates and their respective shifts are shown in Table. below. 28

39 Table. Coordinates of the radar locations. All measurements are given in the format of latitude, longitude and have the unit of degrees. Original Shifted KLSX Top Left , , Bottom Right , , KEAX Top Left , , Bottom Right , , KSGF Top Left , Bottom Right , These coordinates form a grid box of 256 x 256 kilometers around the radar site. By shifting the top left and bottom right corners of the grid box by the same factor, the square is able to maintain its shape while accomplishing the task of encompassing all relevant rain gauge sites..2 Rain Gauge Sites For each NWS radar site, there are a number of associated rain gauge locations that fall in the range of the radar scan volume. While the Missouri Mesonet supplying these gauge sites is relatively extensive, not every site provided the temporal resolution of data necessary to fulfill this body of research. Therefore, a more limited number of sites was selected and analyzed. The gauge locations are grouped by their relation to the nearest radar site. 29

40 Table.2 Coordinates of the rain gauge sites. All latitude and longitude values are measured in units of degrees. The range of the rain gauge site from the nearest radar location is measured in kilometers. KLSX KEAX KSGF Gauge Location Latitude Longitude Range (km) Sanborn Field Capen Park Jefferson Farm Bradford Farm Vandalia Williamsburg Versailles Brunswick Green Ridge Mountain Grove Lamar Mount Vernon Of the gauges selected in the scope of the KLSX radar, four were located in Boone County and therefore have close proximity to one another. All four were included in this study for the sake of demonstrating that a significant range of daily rainfall accumulations can occur even over a small, localized area. See Figure. below for a map of the chosen gauge locations across the state as well as the radar range rings of the associated radar sites. The range rings indicate distance of the rain gauge stations from the nearest radar site. It can be seen that they vary in distance from km. Therefore, it will be necessary to see if the range from the radar has an effect on the equation type that best estimates rainfall at a given location. 0

41 Figure. Map of the chosen rain gauge sites, NWS radar locations, radar range rings, and the domain used in the radar analysis. The inset map in the upper right corner gives a closer look at the four Boone County gauge sites.. Event Selection The rainfall events selected for this body of work were chosen due to their heavy localized precipitation accumulations. All events produced significant rainfall across the state of Missouri in the months of April-October in the year 204. For each of the three radar sites, there are a number of associated rain gauge locations. A majority of the events selected produced daily rainfall accumulations of one inch (25.4 millimeters) or greater for a majority of the sites

42 in question. For example, there are six gauge sites located in the range of the St. Louis radar. To choose a rain event that would produce a good sampling of data, at least four of the six sites needed to measure rainfall totals of an inch (25.4 mm) or greater for that given day. Similarly, the Kansas City and Springfield radar sites have three associated rain gauges per radar location, and thus would need to experience significant daily rainfall totals at two of the three sites. However, there were a limited number of dates where these specifications were met per each radar site. In order to get a sufficient sample size, some events were included in which only one site had accumulations greater than one inch, but the rest measured near half an inch or greater (e.g., KLSX 8/6/204; KEAX 4//204, 4/24/204). The twenty-two intense precipitation dates are divided by radar site and listed in Table. below. Table.a Table of the chosen event dates for the St. Louis radar, duration of events [given in local time], and amount of rainfall recorded at each site [given in mm and (inches)]. Red font indicates events in which rainfall accumulation of less than an inch was recorded. St. Louis- KLSX Duration (LT) 4// /7/ /6/ // /2/ /9/ Total 5 hours San. Jeff. Will. Van. Brad. Cap (2.59) 56.6 (2.2) 4.5 (0.57) 0.8 (5.5) 0. (4.06) 4.4. (.7) 44.2 (6.) 62.5 (2.46) 55.9 (2.20) 2.4 (0.49) 45. (5.72) 2. (4.42) 7.6 (.48) (6.77) 5.8 (2.2) 5.25 (0.60) 9.8 (0.78) 0.7 (4.6) 77.5 (.05) 42.4 (.67) 9.5 (2.58) 57.7 (2.27) 4.2 (.62) 2.8 (.62) 45.5 (.79) 7.4 (2.89) 2.0 (.26) 282. (.) 76.5 (.0) 4. (.62) 0.7 (0.42) 47. (5.79) 0.6 (4.08) 47.2 (.86) (6.78) 70. (2.76) 52.8 (2.08) 2.9 (0.5) 27.8 (5.0) 96.5 (.80) 8.9 (.5) 99.0 (5.7) 2

43 Table.b Table of the chosen event dates for the Kansas City radar, duration of events, and amount of rainfall recorded at each site. Kansas City- Duration KEAX (LT) Green Ridge Versailles Brunswick 4/2/ (.86) 4. (.5) 0.7 (0.42) 4// (0.9) 47.2 (.86) 24.6 (0.97) 4// (.0) 8. (0.2) 28.7 (.) 4/24/ (0.59) 7.9 (0.) 25.9 (.02) 6/9/ (.95) 0.2 (.9) 5.6 (0.22) 8/7/ (.2) 40.4 (.59) 2.0 (.26) 0/2/ (.68) 48.8 (.92).7 (0.54) 0/9/ (.24) 56.6 (2.2) 54.4 (2.4) 0// (.22) 6. (.42) 4.9 (.65) 2/5/ (.27) 47.8 (.88).5 (0.5) Total 70 hours (.07) (4.07) (9.88) Table.c Table of the chosen event dates for the Springfield radar, duration of events, and amount of rainfall recorded at each site. Springfield- Duration Mount Mountain Lamar KSGF (LT) Vernon Grove 5/8/ (0.4) 26.4 (.04) 27.9 (.0) 6/5/ (2.20) 86.9 (.42) 42.7 (.68) 8/7/ (.0) 8. (0.72) 5. (.9) 9// (.08) 26.9 (.06).0 (0.5) 0/2/ (0.78) 20. (0.79) 50.5 (.99) 0/0/ (.57) 8. (.50) 40.6 (.60) Totals 57 hours (6.78) (8.5) (8.27)

44 .. Time Conversion The individual event dates selected were subsequently narrowed down to a select number of hours per day, during which the heaviest rainfall of that date occurred. Because the rain gauge data were given in local time (LT), and the radar data in coordinated universal time (UTC), there was also a conversion from LT to UTC based on Daylight Saving Time for the given event date. In order to make this change, five hours were added to the LT start and end times in order to convert to the correct UTC hours (Table.4). By adding five hours to all events taking place between the dates March 9 November 2, 204, Daylight Saving Time was accounted for; six hours were added to each event outside this time frame (e.g., KEAX 2/5/204). Because the addition of five hours to the LT made a number of the daily events bridge over to a multi-day event, which did not fit well into the MatLab programming materials, the radar day was instead terminated at 259 UTC and thus had a loss of up to five hours of data in these instances. Table.4a Duration of each St. Louis event date given in local time (LT) and coordinated universal time (UTC). St. Louis- KLSX LT UTC 4// /7/ /6/ // /2/ /9/

45 Table.4b Duration of each Kansas City event date given in local time (LT) and coordinated universal time (UTC). Kansas City- KEAX LT UTC 4/2/ // // /24/ /9/ /7/ /2/ /9/ // /5/ Table.4c Duration of each Springfield event date given in local time (LT) and coordinated universal time (UTC). Springfield- KSGF LT UTC 5/8/ /5/ /7/ // /2/ /0/ Data Downloading and Processing Once the radar sites, gauge locations, and event dates and times were selected, the data were collected from the National Climatic Data Center (NCDC), which is a division of the National Oceanic and Atmospheric Administration (NOAA). The data were then untarred, unzipped, and run through a script to extract all relevant data via WDSS-II. The WDSS-II system is described in detail in Section.4.. The ten parameters retained from this process include reflectivity, specific differential phase, differential reflectivity, and RREC, as well as their quality controlled counterparts and rainrate relationships. While the 5

46 data includes radar scans from all elevation angles, only the output from the lowest altitude scans was retained. These data were then used as input for a series of MatLab programs that produced Microsoft Excel files for the pertinent radar and rain gauge measurements. The archived rain gauge data used as input for these programs are available via the Missouri Historical Agricultural Weather Database, which is supplied by weather data from the Commercial Agriculture Automated Weather Station Network. From this source, five-minute rainfall data were requested in order to sum the data into hourly rainfall accumulation periods. From this data, more specific intervals for the various event dates were selected in order to minimize the dataset that needed to be processed. The accumulation times selected for each gauge site were determined by whether or not there was significant rainfall occurring during that string of six to twenty-four hours. By doing this, it was ensured that rain was falling for a majority of the analyzed time and the corresponding radar location would also be measuring rainfall for relatively the same time interval..4. WDSS-II and WDSS-II Processing Overview Warning Decision Support System Integrated Information (WDSS-II) is the second generation of a system of tools for the analysis, diagnosis, and visualization of remotely sensed weather data (Lakshmanan et al. 2007). Unlike the original WDSS program, WDSS-II utilizes data from multiple radars in order to provide the automated algorithms with information, which thus improves the 6

47 temporal and spatial resolution of the program. The automated algorithms in WDSS-II provide real-time products, which are useful for severe weather nowcasting. By using the products in real-world forecasting and decisionmaking contexts, the usefulness of the products can be tested. Since the products are delivered in an integrated manner, the transition from archived weather data to a real-time database is seamless. As this system aids in the forecasting of severe weather, it provides tools for analyzing and diagnosing rotation, hail, windspeed, lightning, and precipitation accumulation and intensity. WDSS-II can also quality control the data, as it can use a network of sensors rather than data from a single radar. The precipitation product is what is used most in this particular study, as the main objective is to compare the accumulated rainfall estimates from the radar sites versus the rain gauge network measurements. Another use of WDSS-II is that the output data are also written in file formats (i.e., NetCDF) that can be easily imported into and used by other research tools, such as the MatLab programs utilized in this body of work, and can therefore be used to analyze individual storm components, and more specifically the precipitation totals. WDSS-II also has the ability to quality control the radar data via the w2dp function. This is the general QC method based on a number of dualpolarization products. These methods include removing the data from nonmeteorological echoes through the use of echo classification and correlationcoefficient; data field smoothing based on the texture parameters DSDZ, DSMZ, DSDP; the production of KDP data, as this can only be created via w2dp and is 7

48 thus automatically quality controlled; and attenuation correction to Z and ZDR based on KDP and texture fields. Another useful component of WDSS-II is the w2merger function. Lakshmanan et al. (2006) describes this technique that is used to merge the base radar data of reflectivity and radial velocity and the subsequent derived parameters from a number of radars into a real-time, rapidly refreshed - dimensional grid. An estimation of the merged radar products comes from an algorithm in the merger routine that takes into account how the geometry of the radar s beam varies with range, the vertical gaps between scans, the disparity of scan times between radars, storm movement, variation of beam resolution between radar type, beam blockage, different radar calibration, and time stamp discrepancies on radar data. By combining data from multiple radars, there is greater accuracy in radar measurements as well as an alleviation of the aforementioned beam geometry problems. By using a -dimensional grid of rapidly updating data, there is, theoretically, real-time data resolution for creating merged products on demand (Lakshmanan et al. 2006). This helps to assess quantified precipitation estimates while also providing better vertical resolution of the data. However, in this particular study, only data from single radar locations were used at a given time, so the merging function was instead used to convert the polar radar coordinates into Cartesian map coordinates. 8

49 .5 Statistical Testing The statistical tests used in this study are summarized in Table.5. The primary aims of the statistics are to assess whether the radar is underestimating or overestimating the rainfall rates and, if it exists, to measure how big the estimation discrepancy is. The over- or underestimation of the rainfall accumulation calculated by the radar is given by the bias. The error will tell the magnitude of the disagreement between the radar and rain gauge measurements. To perform the statistical tests comparing the rain gauge and radar rainfall rates, the data were separated by rain gauge sites and calculated for each event date. The data were then averaged over all event dates for each individual site. The six tests are performed for the data measured by each of the eighteen non-quality controlled equations, and thirty-one quality controlled equations. By separating the data in this way, it is easily shown which type of equation is best at reducing error and bias at each of the rain gauge sites at varying ranges from the radar sites, regardless of storm type and event date. 9

50 Table.5 Statistical tests performed on the data where the variables represent the following: TR = total radar estimated rainfall, TG = total gauge estimated rainfall, where TG and TR are the sum of all gauge and radar measurements. Values surrounded by < > show that summed quantity is averaged over the number of all observed values, N. Statistical Test Bias Mean Absolute Error Fractional Bias Fractional Root Mean Square Error Fractional Standard Deviation Fractional Absolute Difference Equation Σ(R i G i ) N Σ R i G i N < T R T G >/N < T G > < (T R T G ) 2 > < T G > ( < T 2 R T G >/N ) ( < (T R T G ) 2 2 > ) < T G > T G < T R T G > < T G > The primary goal of this body of work is to determine which quantitative precipitation estimation algorithm performs the best, and if the additional quality control of the data helps or harms the rainfall estimation. The use of the term best denotes the equation type that produces error and bias results closest to zero. In order to assess what the definition of best algorithm truly means, statistical tests are necessary for the analysis of the output results. There are two main parts in this assessment the evaluation of bias, and the evaluation of error. The bias shows the systematic over- or underestimation of the data by the radar algorithm and should theoretically produce a pattern in the data from one site to the next. Error, on the other hand, indicates the amount of scatter of 40

51 the QPE values about the true rainfall measurement. In short, the bias will give an assessment of whether or not the true rainfall amount has been overestimated, as indicated by a positive value, or underestimated, as indicated by a negative value. The error indicates the magnitude of radar data inaccuracy, but it will not give insight to whether the data were over- or underestimated. Therefore, both bias and error need to be analyzed together to determine if there exists a truly best algorithm for the radar data. In order to compare the radar rainfall estimations to the rain gauge totals, fractional statistical tests were used. The idea behind using these tools is to even out the impacts of large outlying rain events in a given sample period. For example, in a twenty-four hour period, if there was one hour that produced 400 mm of rain and the rest of the day experienced only mmhr -, that one hour would produce a large bias for the day as a whole. To counteract this, dividing the sample size by the total gauge measured rainfall would even out the large outlying hour and not allow it to dominate the event s statistical results on such a large scale. In short, the use of fractional statistics allows for an even weighting of individual rainfall events on a given date..5. Statistical Testing Definitions In order to accurately assess the analysis produced by the six statistical tests used in this study, there must first be an understanding of what the tools determine. The equations can be found in Table.5 in Section.5, as well as listed in each section below. The desired results from the parameters would be 4

52 those indicating respective values near zero. The parameters of bias and mean absolute errors are expressed in millimeters, whereas the fractional tool values are given in percentages..5.. Bias Σ(R i G i ) N (7) The bias shows the average difference of all of the estimated rainfall accumulations, both those from the radar calculations and the rain gauge measurements. In Eqn. (7) listed above, Ri is the rainfall calculated by the radar at event i where i=-n and Gi is the rainfall measured by the rain gauge for the i th observation, and N is the total number of individual observations. This parameter can be either positive or negative in value and gives a measurement of the over- or underestimation of the radar rainfall measurement. A positive value is indicative of an overestimation, and a negative value indicates an underestimation. This study prefers a positive value of bias. A negative bias would lead to an underestimation of the rainfall amount and could therefore miss the opportunity to issue a flashflood forecast (Ryzhkov et al. 2005) Mean Absolute Error Σ R i G i N (8) The mean absolute error (MAE) is a quantity used to measure how close the radar estimation is to the ground truth rain gauge measurement. This 42

53 parameter is an average of the absolute errors, as the name suggests. MAE is useful for comparing the calculated rainfall from the radar estimation to what was actually measured by the rain gauge in order to assess the radar inaccuracy quantitatively..5.. Fractional Bias <T R T G >/N <T G > (9) Like bias, the fractional bias can produce values that are either negative or positive and thus indicate an under- or overestimation of the radar estimated precipitation. The chevron brackets surrounding the values of TR and TG indicate that these variables have been averaged over the entire set of values for that respective variable. This parameter is normalized by the average of the observed rain gauge values so as to not allow one outlying event to dominate the statistical results Fractional Absolute Difference < T R T G > <T G > (0) Fractional absolute difference gives the absolute value of the difference between the total radar estimated rainfall and the total gauge measured rainfall, normalized by the average of the observed rain gauge values. Here, the term difference indicates the need for subtraction between the radar calculations and rain gauge measurements. Because this parameter measures the absolute 4

54 value, the results can only be positive, just like the values from the mean absolute error Fractional Root Mean Square Error <(T R T G ) 2 > <T G > () The fractional root mean square error, or FRMSE, is the square root of the average of the square of all of the error, normalized by the average of the observed rain gauge values. It is similar to the MAE in that it quantitatively identifies the inaccuracy of the radar data from the gauge measurement, but the FRMSE further enhances the effects of the large outlying events on the outcome of the statistical data. This value represents the sample standard deviation of the differences between the calculated radar rainfall accumulations and the actual measured values of the rain gauges (Hyndman and Koehler, 2006). The FRMSE and the MAE can be used in conjunction to diagnose the variation in the errors between the radar and gauge measurements. The larger the difference is between the two parameters, the greater the variance is in the individual errors in the sample. 44

55 .5..6 Fractional Standard Deviation ( <T R T G >/N <T G > ) 2 ( <(T R T G ) 2 > <T G > 2 ) (2) In general, the standard deviation is a measurement that is used to quantify the amount of variation in a set of data. The closer the standard deviation is to zero, the less spread there is in the dataset. In this study, the fractional standard deviation (FSD) is given by the square root of the difference between the squared FRMSE and squared fractional bias. This parameter describes the spread in data normalized by the average of the observed rain gauge values (Bland and Altman, 996)..5.2 Statistical Analysis Methodology There are two primary statistical tests that can be used to assess the best algorithm for accurately estimating rainfall accumulation by radar: bias and error. Finding the equation that produces values for these metrics closest to zero is the first step in determining the optimal type of radar equation for QPE. It should also be noted that this study prefers a positive bias to a negative bias. A negative bias is indicative of rainfall underestimation by the radar. If the true amount of rain falling at a gauge site is underestimated, this could lead to an undiagnosed flash flood and a significant deal of damage to life and property could ensue. The overestimation of rainfall at a given site on the other hand could lead to forecasting a flash flood that does not happen, but a false alarm is 45

56 preferable to unforeseen damage. The ultimate goal is to find one optimal algorithm that performs best, or produces bias and error values closest to zero, under all circumstances. However this is unlikely due to differences in range, event type, sample size, and available data in general. Instead, the type of equation, as outlined in Table.7, that produces the most accurate rainfall results will be analyzed for a number of scenarios. The optimal equation type is analyzed per radar location, per rain gauge site, and as a function of range of gauge site from the nearest radar location. The effect of quality controlled versus non-quality controlled equations is also examined, as well as the type of QC methods implemented for the various equations. In order to determine the best algorithm per scenario, the equation that produced a result closest to zero for each of the statistical tests was selected. Since FRMSE and FSD produce extremely similar results, due to the small magnitude of the FB value, these two tests are counted as one entity. Thus, for example, for a total of twelve rain gauge sites across the state of Missouri, there is a sample of sixty different optimal equations based on the five different statistical test results for each of the twelve gauge sites. The St. Louis radar (KLSX) has six associated gauges, so thirty shares of the total sample population of sixty measurements, while Kansas City (KEAX) and Springfield (KSGF) split the remaining thirty results with fifteen samples each corresponding to their three gauge sites, respectively. As mentioned previously, a positive bias is preferred to a negative bias. Therefore, if the two smallest values of bias for a 46

57 given site are -0.0 and +0.5, the equation producing the positive value will be counted as the best algorithm in this scenario..6 Radar Algorithm Definitions The set of polarimetric radar equations used in this body of work are taken from Ryzhkov et al. (2005) and summarized in Table.6 below. This set of fourteen equations is expanded into thirty-one equations once a variety of conventional reflectivity equations and quality control regimes are added, as summarized in Table.7. The radar algorithms are subsequently split into categories based on the parameters used in the equations. These parameters include both quality controlled and non-quality controlled versions of reflectivity, differential reflectivity, and specific differential phase. There are five main groupings of the equations, and up to four subcategories within each main category. 47

58 Table.6 Equations separated by type. From Ryzhkov et al. (2005). R(Z) = ar b Precipitation Type a b Stratiform Convective 00.4 Tropical R(KDP) = a KDP b sign(kdp) Algorithm Number a b Source Bringi and Chandrasekar Brandes et al Illingworth and Blackman NSSL NSSL NSSL R(Z, ZDR) = a Z b ZDR c Algorithm Number a b c Source x Bringi and Chandrasekar x Brandes et al x NSSL.59 x NSSL 2.44 x NSSL R(KDP,ZDR) = a KDP b ZDR c sign(kdp) Algorithm Number a b c Source Bringi and Chandrasekar Brandes et al NSSL 48

59 Table.7 Groupings of equations and their various quality control schemes. Eqn. Group Non-QC QC QC-a QC-b RREC RREC R(Z) Strat, Conv, Trop R(Z) R(DSMZ) R(KDP) Eqn. -6 R(KDP) R(KDP) R(Z, ZDR) Eqn. 7-2 R(Z, ZDR) R(DSMZ, R(DSMZ, DZDR) ZDR) R(Z, DZDR) R(KDP, R(KDP, Eqn. -5 R(KDP, ZDR) R(KDP, ZDR) -- ZDR) DZDR) These five different groups correspond to the specific parameters used in each equation type, and are further categorized by the use of quality controlled or non-quality controlled variables within the equation type. As there are a number of equations within each category, the category as a whole is used in the following statistical analysis in order to give a more comprehensive view of which equation type produces the optimal results. There are three separate categories of quality control methods, all of which are listed in Table.7. The polarimetric quality control schemes used in this study correct the data for attenuation, which leads to a general increase in Z and ZDR values. This is especially true as range increases. The QC-a scheme used in the R(Z, ZDR) type equations applied attenuation correction to the Z parameter, and should therefore result in an increase in reflectivity values. Therefore, this should theoretically show that the bias results for these equations become less negative. The QC-b scheme used in both the R(Z, ZDR) and R(KDP, ZDR) categories typically results in an increase in the differential reflectivity values due to attenuation correction performed on that parameter. 49

60 Therefore, as these equations respond to the negative powers of ZDR, this will result in an increase in estimated rainfall rate, and biases should become more negative. The general QC scheme applies quality control to both the Z and ZDR parameters, which should offset the bias results and ultimately exclude extreme over- and underestimation from the data. 50

61 Chapter 4: Statistical Results and Discussion This section will attempt to determine whether an algorithm exists that produces the best QPE with the lowest amount of error and bias. The effect of quality control on the data will also be analyzed to evaluate if this step helps or harms the accuracy of the precipitation estimation. Even though there are not quite enough data points to conclusively assess the effect of rain gauge site range from the radar location, this will also be addressed in the following statistical analysis. In summary, the purpose of this section is to determine if there is one dominant equation that may be used in all scenarios based on radar, gauge site location, and range. 4. Statistical Analysis: Rain Gauge Site Because the rain gauge sites are investigated individually, their results are also expressed individually in the following subsections, For each of the twelve gauge locations, there are five statistical tests used to compare the results of the various types of QPE algorithms. The method of examining the rain gauge site results is the same for each location, and is how the results in Sections 4.2 and 4. are analyzed as well. For each statistical test, the equation that produces results closest to zero is tallied as the best algorithm, as noted by the bolded red font in the data tables. Because FB is such a small value, and this is the only difference between FRMSE and FSD, the fractional root mean square error and fractional standard deviation are tallied as 5

62 one test since the difference between them is so small. From the data tables, bar graphs were made for each of the twelve gauge sites for all six of the statistical tests. Only the bias and mean absolute error are shown in the main text below; the remaining four graphs per site can be found in the appendix. Finally, there are three pie charts produced per gauge site. The first shows the number of times each different group of radar equation produced the best result for the five statistical tests tallied per gauge site. The second pie chart indicates the number of times the various RREC, non-quality control (NON-QC), and quality control (QC, QC-a, and QC-b) methods output the optimal values for the same five statistical tools. The final chart shows the benefits of quality control versus nonquality control and RREC with all QC methods as one category for a more complete idea of whether or not quality controlling the data enhances the results. 52

63 4.. Williamsburg Table 4. Comparison of the values of the six statistical tests for each of the thirty-one equations for the Williamsburg site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

64 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

65 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b.5 Williamsburg Bias Non-QC Quality Controlled Figure 4.a Bias of the radar estimated rainfall at the Williamsburg gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm Williamsburg Mean Absolute Error Non-QC Quality Controlled Figure 4.b Mean absolute error of the radar estimated rainfall at the Williamsburg gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 55

66 WILLIAMSBURG 40% 60% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.c Best equation by type for the Williamsburg site data. WILLIAMSBURG 20% 60% 20% NON-QC QC RREC QC-a QC-b Figure 4.d Best quality control scheme for the Williamsburg site data. WILLIAMSBURG 20% 80% NON-QC QC RREC Figure 4.e Benefits of quality control over non-quality control for the Williamsburg site data. 56

67 The initial analysis of the data table appears to indicate a wide range of best algorithm selections for the given rain gauge site. This is not unique to the Williamsburg station. From the data table alone, it is difficult to discern a pattern in the diagnosis of which algorithm produces the optimal results. For example, in Table 4., the best equation type for bias, MAE, and FB all come from different categories and QC regimes. Table 4. also shows that FRMSE and FSD both produce the best results when using Eqn. b, but as mentioned before, the only difference between these two values is the FB, which is a very small number. For a majority of the cases analyzed in this study, FRMSE and FSD perform the best under the same algorithm and are thus tallied as a single entity. In the bar graph in Figure 4.b., there is a pattern of mainly positive bias for the R(KDP) algorithms, negative for the R(Z, ZDR), and a mixture of positive and negative for the R(KDP, ZDR) category depending on the use of the QC-b scheme. As noted in Section.5, whether the true rainfall amount is being overor underestimated by the radar is given by the bias, and the magnitude of that over- or underestimation is indicated by the value of this error. Thus for Williamsburg there is a pattern of underestimation of the rainfall given by the R(Z, ZDR) equations, and this grouping also produces the least amount of error. Finally, an analysis of the pie chart data is made to determine the best algorithm type. Figure 4.c does indeed indicate that the R(Z, ZDR) equations produce the best results 60% of the time, and that the QC-b scheme gives optimal results for the same number of instances (Figure 4.d). 80% of the time, some type of QC method enhances the data results at the Williamsburg station. 57

68 4..2 Vandalia Table 4.2 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Vandalia site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

69 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

70 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b Non-QC Vandalia Bias Quality Controlled Figure 4.2a Bias of the radar estimated rainfall at the Vandalia gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm Vandalia Mean Absolute Error Non-QC Quality Controlled Figure 4.2b Mean absolute error of the radar estimated rainfall at the Vandalia gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 60

71 VANDALIA 00% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.2c Best equation by type for the Vandalia site data. VANDALIA 60% 40% NON-QC QC RREC QC-a QC-b Figure 4.2d Best quality control scheme for the Williamsburg site data. VANDALIA 60% 40% NON-QC QC RREC Figure 4.2e Benefits of quality control over non-quality control for the Vandalia site data. 6

72 For the Vandalia rain gauge site, there is more agreement amongst the statistical tests about which type of equation produces the best results. In fact, for MAE, FAD, FRMSE, and FSD, all four indicate Eqn. 8b is the optimal algorithm. According to the bar graphs, the R(Z, ZDR) category has a trend of a negative bias, or underestimating the rainfall, for all of the equations except when the general QC scheme was used, in which case a positive bias occurs. For the same category, there are the lowest mean absolute errors of all the equations. The pie chart depicting which type of equation produces the best results indicates that the R(Z, ZDR) category gives the optimal outputs 00% of the time, or for each of the five counted statistical tests. However, there is not as conclusive an agreement for whether or not quality controlling the data is beneficial. For 40% of the tests, the non-quality control scheme produced the best results, and the QC-b regime was best the other 60% of the time, which is indeed a majority, but only slightly especially for such a small sample size. 62

73 4.. Bradford Farm Table 4. Comparison of the values of the six statistical tests for each of the thirty-one equations for the Bradford Farm site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

74 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

75 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b.5 Bradford Farm Bias Non-QC Quality Controlled Figure 4.a Bias of the radar estimated rainfall at the Bradford Farm gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm Bradford Farm Mean Absolute Error Non-QC Quality Controlled Figure 4.b Mean absolute error of the radar estimated rainfall at the Bradford Farm gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 65

76 BRADFORD FARM 20% 80% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.c Best equation by type for the Bradford Farm site data. BRADFORD FARM 40% 60% NON-QC QC RREC QC-a QC-b Figure 4.d Best quality control scheme for the Bradford Farm site data. BRADFORD FARM 40% 60% NON-QC QC RREC Figure 4.e Benefits of quality control over non-quality control for the Bradford Farm site data. 66

77 The Bradford Farm rain gauge site, one of the four sites located in Boone County in central Missouri, has fairly good agreement of which type of radar algorithm produces the most accurate rainfall estimation. Four out of the five statistical tests give the best results when some sort of R(Z, ZDR) equation is used, but the equation itself varies from test to test. Bias is the only test in which an equation from the R(KDP, ZDR) category performs optimally. MAE indicates the NON-QC Eqn. 8 is best, FB tallies for NON-QC Eqn., and the remaining FAD, FRMSE, and FSD produce the best results when the QC-b form of Eqn. is used. Again, FRMSE and FSD produce the same results nearly every time, so these two tests are counted as a single result. Bar graph analysis indicates that a majority of the equations produce a negative bias for the radar data, which is an underestimation of the actual rainfall amount. The conventional R(Z) and R(Z, ZDR) categories produce the least amount of error of the five groupings. However, there are again rather inconclusive results in terms of whether or not the use of a quality control regime enhances the data output. The NON-QC scheme produced optimal results 60% of the time, and the QC-b method enhanced the data for the other 40%. The best algorithm type for Bradford Farm, similarly to the rest of the sites previously investigated, is the R(Z, ZDR) category. 67

78 4..4 Sanborn Field Table 4.4 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Sanborn Field site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

79 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

80 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b Sanborn Field Bias Non-QC Quality Controlled Figure 4.4a Bias of the radar estimated rainfall at the Sanborn Field gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 6 Sanborn Field Mean Absolute Error Non-QC Quality Controlled Figure 4.4b Mean absolute error of the radar estimated rainfall at the Sanborn Field gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 70

81 SANBORN FIELD 40% 60% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.4c Best equation by type for the Sanborn Field site data. SANBORN FIELD 40% 60% NON-QC QC RREC QC-a QC-b Figure 4.4d Best quality control scheme for the Sanborn Field site data. SANBORN FIELD 40% 60% NON-QC QC RREC Figure 4.4e Benefits of quality control over non-quality control for the Sanborn Field site data. 7

82 The rain gauge site at Sanborn Field is another of the four located in Boone County. While the bar graphs appear similar to those created from the data at the Bradford Farm site, the data table and pie charts indicate that 40% of the best results for this site happen when a conventional R(Z) type relationship is used. Both bias and mean absolute error produce values closest to zero out of their respective categories when the R(Z)-Tropical and R(Z)-Convective algorithms are used, respectively. However, the remaining statistical tools output the optimal results when an equation from the R(Z, ZDR) category is used, which amounts to 60% of the time. With regards to the benefits of data quality control, the results from this analysis indicate that the NON-QC regime produces the best results for a majority (60%) of the time. The remaining two statistical tests, FAD and FRMSE, produce the best outputs when the QC-b scheme is used for the data. 72

83 4..5 Capen Park Table 4.5 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Capen Park site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

84 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

85 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b Capen Park Bias Non-QC Quality Control Figure 4.5a Bias of the radar estimated rainfall at the Capen Park gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm Capen Park Mean Absolute Error Non-QC Quality Control Figure 4.5b Mean absolute error of the radar estimated rainfall at the Capen Park gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 75

86 CAPEN PARK 20% 60% 20% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.5c Best equation by type for the Capen Park site data. CAPEN PARK 40% 20% 20% 20% NON-QC QC RREC QC-a QC-b Figure 4.5d Best quality control scheme for the Capen Park site data. CAPEN PARK 20% 20% 60% NON-QC QC RREC Figure 4.5e Benefits of quality control over non-quality control for the Capen Park site data. 76

87 Capen Park s rain gauge site, the third of the four Boone County sites, has rather varied results compared to what has been analyzed thus far. The bias closest to zero occurs when the QC R(Z)-Convective algorithm is implemented, the lowest mean absolute error results under the NON-QC Eqn. 2, the lowest fractional bias happens with the RREC decision tree type algorithm, and the lowest fractional absolute difference, fractional root mean square error, and fractional standard deviation occur with the use of the QC-b Eqn.. Analysis of the bar graphs indicate a mix of positive and negative bias for the R(Z) category, largely positive bias results for the R(KDP) grouping, a negative bias trend for the R(Z, ZDR) equations, and a mix of positive and negative bias results for the R(KDP, ZDR) group depending on the type of quality control scheme implemented. The MAE also produces varying results within the individual equation groupings as well as amongst QC types. The pie chart indicating the superior equation type (Figure 4.5c) shows that again, the best type of algorithm is the R(Z, ZDR) category for 60% of the statistical tests. RREC and R(Z) each produced the best result 20% of the time. The topic of which type of QC scheme is best, as indicated in Figure 4.5d, is also rather inconclusive for the Capen Park gauge site. The QC-b method produces the best results 40% of the time, while QC, NON-QC, and RREC each are optimal for 20% of the statistical tests. A better figure to examine for determining whether or not the use of QC enhances data results is the third pie chart, Figure 4.5e. This figure shows that some type of QC regime produces the best results for 60% of the tests, while NON-QC and RREC are optimal for only 20% each. 77

88 4..6 Jefferson Farm Table 4.6 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Jefferson Farm site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

89 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

90 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b.5 Jefferson Farm Bias Non-QC Quality Controlled Figure 4.6a Bias of the radar estimated rainfall at the Jefferson Farm site for the thirty-one radar equations, separated by quality control method. Measured in mm Jefferson Farm Mean Absolute Error Non-QC Quality Controlled Figure 4.6b Mean absolute error of the radar estimated rainfall at the Jefferson Farm gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 80

91 JEFFERSON FARM 20% 80% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.6c Best equation by type for the Jefferson Farm site data. JEFFERSON FARM 40% 20% 40% NON-QC QC RREC QC-a QC-b Figure 4.6d Best quality control scheme for the Jefferson Farm site data. JEFFERSON FARM 20% 80% NON-QC QC RREC Figure 4.6e Benefits of quality control over non-quality control for the Jefferson Farm site data. 8

92 The rain gauge site at Jefferson Farm is the final of the four Boone County gauge locations. The data table results show widely varying results in terms of which specific algorithm is superior. The lowest bias result comes from the use of Eqn. 4, the lowest MAE from Eqn. 8, the lowest FAD from Eqn. 2, Eqn. 2b produces the lowest FAD and FRMSE, and Eqn. 0b gives the lowest FSD. So, in this case, not even FRMSE and FSD agree upon which equation produces the best results. The bias bar chart indicates very large negative values associated with the R(Z, ZDR) type equations, while the MAE values are relatively even across equation types. The pie chart analysis depicts the result of best equation type in a more accessible format. The R(Z, ZDR) type equations produce the optimal outputs for 80% of the statistical tools used to analyze the Jefferson Farm radar data. However, there is some variance in the results of the QC analysis. The unaltered, NON-QC data perform best for only one statistical test out of the five, or 20% of the time, but there is no clear advantage for the use of the QC scheme over QC-b, as they both produce the best results 40% of the time. In general, however, the use of some sort of quality control regime does enhance the results at this site 80% of the time. At these long ranges from the KLSX radar, it appears that the attenuation correction done by the QC scheme does a fair job of reducing the underestimation bias, and in this case, most of the KLSX gauge sites have a largely negative associated bias. 82

93 4..7 Green Ridge Table 4.7 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Green Ridge site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

94 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

95 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b Green Ridge Bias Non-QC Quality Controlled Figure 4.7a Bias of the radar estimated rainfall at the Green Ridge gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm Green Ridge Mean Absolute Error Non-QC Quality Controlled Figure 4.7b Mean absolute error of the radar estimated rainfall at the Sanborn Field gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 85

96 GREEN RIDGE 20% 80% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.7c Best equation by type for the Green Ridge site data. GREEN RIDGE 20% 80% NON-QC QC RREC QC-a QC-b Figure 4.7d Best quality control scheme for the Green Ridge site data. GREEN RIDGE 20% 80% NON-QC QC RREC Figure 4.7e Benefits of quality control over non-quality control for the Green Ridge site data. 86

97 The Green Ridge rain gauge site has more conclusive results than most other sites with regards to which specific equation produces the best overall results for the statistical tools. For four out of the six tests, the NON-QC Eqn. 2 produced the best output. The use of quality controlled Eqn. 2a resulted in the lowest bias, and NON-QC Eqn. 5 gave the lowest fractional bias. An analysis of the bar graphs shows bias values relatively close to zero for many of the R(Z, ZDR) type equations. This same algorithm group also produced relatively low mean absolute error values in comparison to the rest of the equation types. The pie charts produced from the Green Ridge data are fairly conclusive across all three figures. Figure 4.7c shows that the R(Z, ZDR) type equations produce the best results for 80% of the statistical tests, while R(KDP, ZDR) is superior for the remaining 20%. The original non-quality controlled data at this site are better than any quality control scheme for 80% of the data, while the QC-a scheme produces better results for the other 20%. 87

98 4..8 Brunswick Table 4.8 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Brunswick site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

99 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

100 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b Brunswick Bias Non-QC Quality Controlled Figure 4.8a Bias of the radar estimated rainfall at the Brunswick gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 6 Brunswick Mean Absolute Error Non-QC Quality Controlled Figure 4.8b Mean absolute error of the radar estimated rainfall at the Brunswick gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 90

101 BRUNSWICK 20% 80% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.8c Best equation by type for the Brunswick site data. BRUNSWICK 20% 80% NON-QC QC RREC QC-a QC-b Figure 4.8d Best quality control scheme for the Brunswick site data. BRUNSWICK 20% 80% NON-QC QC RREC Figure 4.8e Benefits of quality control over non-quality control for the Brunswick site data. 9

102 The Brunswick rain gauge site data produce similar pie chart percentage results to those from Green Ridge, but the data table is rather different, as there is more variation amongst the specific equations that produce the best results for the data. The bias is lowest when the Z(R)-Stratiform equation is used, while the mean absolute error, fractional absolute difference, and fractional root mean square error are all best when using Eqn. 2. At this site, unlike a majority of the data from the other gauge sites, the FRMSE and FSD do not produce the best results for the same specific equation. Fractional standard deviation has an optimal output for Eqn. 8. Like Green Ridge, the bar graphs for Brunswick bias also show values close to zero for the R(Z, ZDR) type equations, and relatively high positive bias values for the equations involving specific differential phase [e.g., R(KDP), R(KDP, ZDR)]. There is also a substantial decrease in the amount of error for the algorithms using the parameters of reflectivity and differential reflectivity [e.g., R(Z), R(Z, ZDR)]. With regards to the pie charts, Figure 4.8c appears similar to Figure 4.7c from the Green Ridge data. The best results are produced with the use of an R(Z, ZDR) type equation 80% of the time, while R(Z) gives optimal results for the remaining 20%. The NON-QC data are best for 80% of the time as well, while QC enhances the data for the other 20%. 92

103 4..9 Versailles Table 4.9 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Versailles site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

104 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

105 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b 2 Versailles Bias Non-QC Quality Controlled Figure 4.9a Bias of the radar estimated rainfall at the Versailles gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm Versailles Mean Absolute Error Non-QC Quality Controlled Figure 4.9b Mean absolute error of the radar estimated rainfall at the Versailles gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 95

106 VERSAILLES 20% 80% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.9c Best equation by type for the Versailles site data. VERSAILLES 20% 80% NON-QC QC RREC QC-a QC-b Figure 4.9d Best quality control scheme for the Versailles site data. VERSAILLES 20% 80% NON-QC QC RREC Figure 4.9e Benefits of quality control over non-quality control for the Versailles site data. 96

107 The Versailles rain gauge site produced pie chart results similar to those from the other gauge sites associated with the Kansas City radar (KEAX), especially the Brunswick site. Like Brunswick, the bias closest to zero comes from an R(Z) type relation, specifically R(Z)-Tropical. Eqn. produces the best results for MAE and FAD, and Eqn. 2 is best for FRMSE and FSD. The bias bar chart shows a general underestimation trend for the Versailles radar data, except for when the R(KDP) type equations are used. As at the Brunswick site, the lowest mean absolute error occurs for the R(Z) and R(Z, ZDR) type equations at this location as well. The pie chart results for Versailles and Brunswick are identical. Both produced the best results 80% of the time when using an equation from the R(Z, ZDR) category, and the R(Z) category for the other 20% of the statistical tests. Furthermore, the NON-QC regime also produced the best results at this location for 80% of the tests, while the general QC scheme was superior for the remaining 20%. 97

108 4..0 Mount Vernon Table 4.0 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Mount Vernon site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

109 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

110 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b 4 Mount Vernon Bias Non-QC Quality Controlled Figure 4.0a Bias of the radar estimated rainfall at the Mount Vernon gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 6 Mount Vernon Mean Absolute Error Non-QC Quality Controlled Figure 4.0b Mean absolute error of the radar estimated rainfall at the Mount Vernon gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 00

111 MOUNT VERNON 20% 80% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.0c Best equation by type for the Mount Vernon site data. MOUNT VERNON 20% 60% 20% NON-QC QC RREC QC-a QC-b Figure 4.0d Best quality control scheme for the Mount Vernon site data. MOUNT VERNON 20% 80% NON-QC QC RREC Figure 4.0e Benefits of quality control over non-quality control for the Mount Vernon site data. 0

112 The data table results for the rain gauge site in Mount Vernon are fairly scattered in regards to whether a specific equation is best for every statistical test. The best bias result comes from the use of Eqn. 4, the lowest mean absolute error occurs with Eqn. 0a, Eqn. 0b is best for producing a fractional bias closest to zero, Eqn. 0a for fractional absolute difference closest to zero, Eqn. 2a for the lowest fractional root mean square error, and Eqn. 7b for the lowest fractional standard deviation. Therefore, there is no single equation that produces the best results across the board for the Mount Vernon data. Analysis of the bar graphs also shows greatly varying trends in the bias. For the RREC and R(Z) type equations, there is a pattern of negative bias. For the R(KDP), R(Z, ZDR), and R(KDP, ZDR) categories, there is a mix of positive and negative bias within each grouping. With the exception of Eqn. 7, the lowest mean absolute errors occur for the R(Z, ZDR) type equations. Even though there is no specific equation that produces the best results for all the statistical tests performed on the Mount Vernon data, Figure 4.0c shows that the R(Z, ZDR) type equations work best for this site 80% of the time. The QC-a scheme also produces the best results for 60% of the statistical tests, while NON-QC and QC are even at 20% each. Overall, the use of some sort of quality control scheme enhances the data results 80% of the time at the Mount Vernon gauge site. 02

113 4.. Lamar Table 4. Comparison of the values of the six statistical tests for each of the thirty-one equations for the Lamar site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

114 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

115 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b 4 Lamar Bias Non-QC Quality Controlled Figure 4.a Bias of the radar estimated rainfall at the Lamar gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm Lamar Mean Absolute Error Non-QC Quality Controlled Figure 4.b Mean absolute error of the radar estimated rainfall at the Lamar gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 05

116 LAMAR 40% 20% 20% 20% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.c Best equation by type for the Lamar site data. LAMAR 20% 20% 60% NON-QC QC RREC QC-a QC-b Figure 4.d Best quality control scheme for the Lamar site data. LAMAR 20% 80% NON-QC QC RREC Figure 4.e Benefits of quality control over non-quality control for the Lamar site data. 06

117 Of all of the rain gauge sites, the Lamar site produced the most widely varying results with regards to which type of radar equation is best. An R(KDP) type equation produced the bias value closest to zero, and R(Z, ZDR) algorithm was best for both MAE and FRMSE, R(Z)-Tropical gave the lowest FB, and RREC had the best results for FAD. The bar graphs for Lamar have similar patterns as those for Mount Vernon. The RREC and Z(R) equations produce a negative radar bias, while the R(KDP), R(Z, ZDR), and R(KDP, ZDR) groups have a mix of positive and negative bias results. Like Mount Vernon, the lowest MAE also occurs for the R(Z, ZDR) equations with the exception of Eqn. 7. The pie chart results show that an R(Z, ZDR) type equation is best 40% of the time, while RREC, R(Z), and R(KDP) are each best for 20% of the statistical tests. Figure 4.e has the most conclusive results for this location, as it shows that some sort of quality control regime enhances the results for the Lamar gauge site 80% of the time. Of this 80%, the general QC scheme is best for 60% and QC-b is best for the other 20%. 07

118 4..2 Mountain Grove Table 4.2 Comparison of the values of the six statistical tests for each of the thirty-one equations for the Mountain Grove site s data; the lowest value per test is indicated in bold red. Bias and MAE are measured in mm, fractional tests in %. NON-QC Bias QC Bias NON-QC MAE QC MAE NON-QC FB QC FB RREC Strat Conv Trop a b a b a b a b a b b b b

119 NON-QC FAD QC FAD NON-QC FRMSE QC FRMSE NON-QC FSD QC FSD RREC Strat Conv Trop a b a b a b a b a b b b b

120 RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b RREC Conv 5 7 7b 8a 0 0b a 2 2b b 4b 5b Mountain Grove Bias Non-QC Quality Controlled Figure 4.2a Bias of the radar estimated rainfall at the Mountain Grove gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm Mountain Grove Mean Absolute Error Non-QC Quality Controlled Figure 4.2b Mean absolute error of the radar estimated rainfall at the Mountain Grove gauge site for the thirty-one radar equations, separated by quality control method. Measured in mm. 0

121 MOUNTAIN GROVE 40% 20% 40% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.2c Best equation by type for the Mountain Grove site data. MOUNTAIN GROVE 20% 40% 40% NON-QC QC RREC QC-a QC-b Figure 4.2d Best quality control scheme for the Mountain Grove site data. MOUNTAIN GROVE 40% 60% NON-QC QC RREC Figure 4.2e Benefits of quality control over non-quality control for the Mountain Grove site data.

122 Mountain Grove s rain gauge site, like the others associated with the Springfield radar (KSGF), has widely varying data and thus does not have a single equation that is best for all the statistical tests used. The Z(R)-Convective algorithm produces the lowest bias, Eqn. a gives the lowest mean absolute error and fractional absolute difference, Eqn. gives the lowest fractional bias, and Eqn. 4b the lowest fractional root mean square error and fractional standard deviation. The bias bar graph results have an interesting pattern within the equation groupings. The R(Z) type algorithms all produce a low positive bias, the R(KDP) type a large positive bias, but the R(Z, ZDR) and R(KDP, ZDR) categories are not quite so straightforward. The resulting negative or positive bias varies based on what type of quality control method was used, as does the mean absolute error for these categories. The pie charts show that in addition to there not being best single equation, there is not a best type of equation either. Both R(Z, ZDR) and R(KDP, ZDR) produce the best results for 40% of the data, while R(Z) is best for the remaining 20%. Similarly inconclusive are the results for the best quality control scheme in Figure 4.2d. The NON-QC and QC-a schemes are best 40% of the time each, and QC-b produces the best results for the other 20% of the time. Therefore, the overall results of whether or not quality control enhances the results of the data concludes that some sort of QC scheme is beneficial 60% of the time, while the unaltered NON-QC is best the other 40%. 2

123 4.2 Statistical Analysis: Radar Location Results for the best type of equation based on radar location are deconstructed as follows:. Best type per radar a. Quality control versus non-quality control per radar b. Best type for all radar sites 2. Quality control versus non-quality control per equation type a. Type of quality control per equation type b. Quality control versus non-quality control for all radar sites c. Type of quality control for all radar sites Graphical representations of the statistical analyses for these subcategories are examined below Best Equation Type The KLSX radar site has six associated rain gauge sites. For each gauge site, the best equation type was selected for each of the five statistical tests. Therefore, KLSX produces results for a sample size of thirty instances of best equation type. Similarly, KEAX and KSGF both have three associated rain gauge sites, so have sample sizes of fifteen measurements apiece. The results for each radar location are first broken down into what was the best equation type for each of the five categories: RREC, R(Z), R(KDP), R(Z, ZDR), and R(KDP, ZDR). In addition to determining the best radar algorithm for a series of rainfall events, another aim of this study is to determine if quality controlling the data is

124 constructive or deconstructive for the accuracy of the rainfall estimation. This analysis is also performed for each radar site. KLSX Totals by Type 4% 0% 0% % 7% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.a Best equation by type for the rain gauge sites associated with the St. Louis radar location. Of the thirty measurements for best equation type for KLSX, R(Z, ZDR) was optimal for 7% of the sample, which is a significant majority. R(Z) and R(KDP, ZDR) were selected as the optimal algorithm type three times each, and RREC and R(KDP) one time each. Thus, for the St. Louis radar, radar rainfall equations utilizing the parameters of reflectivity and differential reflectivity produce the best QPE results. Of the sample, the use of quality control on the data produced superior results 60% of the time. 4

125 KEAX Totals by Type 0% 7% % 0% 80% Figure 4.b Best equation by type for the rain gauge sites associated with the Kansas City radar location. KEAX, with a sample size of 5 measurements, also produced a significant majority of optimal results (80%) when R(Z, ZDR) equations were used. RREC and R(KDP) were the optimal algorithms 0% of the time. However, this site produced different results from KLSX in terms of the usefulness of quality controlling the data. The use of QC produced enhanced output data only 20% of the time, which is relatively underwhelming compared to the 60% improvement for KLSX data. RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) 5

126 KSGF Totals by Type 20% 7% % 7% 5% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.c Best equation by type for the rain gauge sites associated with the Springfield radar location. The data from the Springfield radar produced the widest range of results in terms of optimal algorithm type prominence. Like KLSX and KEAX, equations based on reflectivity and differential reflectivity still performed best out of the five categories, but only 5% of the time. Furthermore, QC algorithms produced the best results for 7% of the data, which is significantly more than for KLSX (60%), and especially KEAX (20%). 6

127 Totals by Type % 2% 2% % 70% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.d Best equation by type for all twelve rain gauge sites. Finally, the analysis of best equation type is made for all three radar locations as a whole. The sample size in this instance is sixty. As expected, based on the results from the individual radar data, R(Z, ZDR) produces optimal outputs for a significant majority of the dataset. Of the five equation types, rainfall rate estimation based on reflectivity and differential reflectivity is the most accurate 70% of the time Quality Control vs. Non-Quality Control Each equation type, with the exception of RREC, has a scheme of quality controlled parameters applied to the variables involved. These are outlined in detail in Table.7 in Section.5.2. First, it is determined whether or not the application of QC methods produced enhanced results over the original non- 7

128 quality controlled outputs, per equation type. Then, the assorted types of QC method for that algorithm category were examined to further determine which scheme of quality control performed best. Finally, quality controlled versus nonquality controlled data are examined for the sum of the dataset from all radar locations and the various QC schemes are analyzed for all sites. R(KDP) 0% 00% NON-QC QC Figure 4.4a Benefits of quality control over non-quality control for the R(KDP) type equations. The relationship of rainfall and specific differential phase, R(KDP), was selected as the best algorithm type only twice out of the total sample. In both instances, the QC data produced better results than NON-QC. Although the parameter of KDP is not often considered capable of being quality controlled, the QC R(KDP) algorithm in this case has been run through the w2dp scheme in WDSS-II and thus does indeed fall under the category of quality control. It should be noted that QC produced better results by a very small margin, and is therefore not necessarily significant for this parameter. 8

129 R(Z) 29% 7% NON-QC QC Figure 4.4b Benefits of quality control over non-quality control for the R(Z) type equations. The conventional reflectivity-rainfall relation performed best seven times out of the total sample of sixty measurements. Of these seven instances, the original non-quality controlled (NON-QC) data produced the best results 7% of the time, and quality controlled (QC) only 29%. R(KDP, ZDR) 29% 7% NON-QC QC Figure 4.4c Benefits of quality control over non-quality control for the R(KDP, ZDR) type equations. 9

130 R(KDP, ZDR) QC 0% 00% QC QC-b Figure 4.4d Best quality control scheme for the R(KDP, ZDR) type equations. Like R(Z), the category of R(KDP, ZDR) algorithms produced the best results only seven times out of the total sample of sixty. Unlike R(Z), however, R(KDP, ZDR) has an additional subcategory in the QC scheme. Of the seven occurrences of this algorithm type as the best estimator, NON-QC produces better results than QC 7% of the time. The two times QC was superior was due to the QC-b scheme, in which the parameter ZDR was corrected for attenuation. 20

131 R(Z, ZDR) 8% 62% NON-QC QC Figure 4.4e Benefits of quality control over non-quality control for the R(Z, ZDR) type equations. R(Z, ZDR) QC 9% 54% 27% QC QC-a QC-b Figure 4.4f Best quality control scheme for the R(Z, ZDR) type equations. The QPE algorithm type that produced the majority of optimal results (42 out of 60, or 70%) is the category of rainfall relation to a combination of reflectivity and differential reflectivity parameters. For this category, quality controlled data produced improved results over non-quality controlled data 62% of the time. This QC category is further dissected into two subgroups of QC- 2

132 a and QC-b. The general QC scheme applies to occurrences in which only the Z parameter of R(Z, ZDR) is subject to data quality control. In QC-a, both Z and ZDR are controlled. QC-b has QC applied only to the ZDR parameter. Of these subcategories, the QC-b scheme produced the best results 54% of the time. Therefore, since R(Z, ZDR) is the optimal equation type of all five types, and QC-b produces the best results of this category, it can be argued that an R(Z, ZDR) equation under the QC-b scheme is the optimal QPE algorithm for all radar sites. Total % 54% 4% NON-QC QC RREC Figure 4.4g Benefits of quality control over non-quality control for all equation types. 22

133 Total 27% 4% 2% % 5% NON-QC QC RREC QC-a QC-b Figure 4.4h Best quality control scheme for all equation types. Finally, in examining all radar locations together for the whole dataset of sixty measurements, it is found that QC data outperform NON-QC data 5% of the time. This is not necessarily significant, as QC performs best only slightly over the halfway point of 50%. Therefore, each algorithm type must be examined individually to determine the effects of quality control on the data. 4. Statistical Analysis: Function of Range By using NOAA s Latitude/Longitude Distance Calculator, as adapted from the Great Circle Calculator ( the distance between rain gauge sites and the nearest radar location was determined. From these data, the sites were then sorted into the close range category, or any site less than 00 km from the nearest radar, and the far range category, any site greater than 00 km from the nearest radar (Table 4.). 2

134 Table 4. Separation of the rain gauge sites by close range (any site less than 00km from the nearest radar location) and far range (any site more than 00km from the nearest radar location). The best bias and MAE values per site are also given. Gauge Station Distance (km) Best Bias Best MAE Close Range Mount Vernon Green Ridge Lamar Williamsburg Vandalia Far Range Mtn. Grove Brunswick Versailles Bradford Farm Jefferson Farm Capen Park Sanborn Field Just as was done for the statistical analysis per radar location and per gauge site, the analysis as a function of range also counts the best equation type for each of the five statistical tests and determines the optimal algorithm category by the highest score. There are five close range gauge sites, and five statistical tests per site, so the total sample size for this range is twenty-five. The seven far range gauge sites also have five statistical tests per site, and thus a sample size of thirty-five. 24

135 Close Range 4% 4% 4% 6% 72% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.5a Best equation by type for the rain gauge sites at a close range. Close Range 28% 2% 6% 20% 4% NON-QC QC RREC QC-a QC-b Figure 4.5b Best quality control scheme for the rain gauge sites at a close range. 25

136 For the close range sample size of twenty-five, an R(Z, ZDR) equation produced the optimal results for eighteen statistical tests, or 72% of the time. Four of the twenty-five best results came from R(KDP, ZDR), and the remaining best results are split evenly to one test apiece for the RREC, R(Z), and R(KDP) type equations. The results showing the benefits of data quality control are much more varied, and there is no clear frontrunner for which QC scheme is best. While 64% of the results showed best results when a QC scheme was applied, and only 2% indicate that the unaltered NON-QC mode is superior, the deconstruction of which type of QC method is best is relatively inconclusive. The QC-b scheme shows slightly better results with 28% of the best results coming from this mode. QC and QC-a both have similar percentages of optimal results as well, however. QC accounts for 20% of the best statistical results, and QC-a accounts for the remaining 6% of the quality control scheme best data. 26

137 Far Range % 9% 7% % 68% RREC R(Z) R(KDP) R(Z, ZDR) R(KDP, ZDR) Figure 4.5c Best equation by type for the rain gauge sites at a far range. Far Range 26% 6% 5% % 4% NON-QC QC RREC QC-a QC-b Figure 4.5d Best quality control scheme for the rain gauge sites at a far range. 27