Validating NEXRAD MPE and Stage III precipitation products for uniform rainfall on the Upper Guadalupe River Basin of the Texas Hill Country

Transcription

1 Journal of Hydrology (2008) 348, available at journal homepage: Validating NEXRAD MPE and Stage III precipitation products for uniform rainfall on the Upper Guadalupe River Basin of the Texas Hill Country Xianwei Wang a, Hongjie Xie a, *, Hatim Sharif b, Jon Zeitler c a Laboratory for Remote Sensing and Geoinformatics, University of Texas at San Antonio, One UTSA Circle, San Antonio, TX 78249, United States b Department of Civil Engineering, University of Texas at San Antonio, One UTSA Circle, San Antonio, TX 78249, United States c NOAA/NWS, Austin/San Antonio Weather Forecast Office, New Braunfels, TX, United States Received 5 January 2007; received in revised form 7 September 2007; accepted 18 September 2007 KEYWORDS NEXRAD; Stage III; MPE; Rain gauge; Precipitation evaluation Summary This study examines the performance of the Next Generation Weather Radar (NEXRAD) Multisensor Precipitation Estimator (MPE) and Stage III precipitation products, using a high-density rain gauge network located on the Upper Guadalupe River Basin of the Texas Hill Country. As point-area representativeness error of gauge rainfall is a major concern in assessment of radar rainfall estimation, this study develops a new method to automatically select uniform rainfall events based on coefficient of variation criterion of 3 by 3 radar cells. Only gauge observations of those uniform rainfall events are used as ground truth to evaluate radar rainfall estimation. This study proposes a new parameter probability of rain detection (POD) instead of the conditional probability of rain detection (CPOD) commonly used in previous studies to assess the capability that a radar or gauge detects rainfall. Results suggest that: (1) gauge observations of uniform rainfall better represent ground truth of a 4 4km 2 radar cell than non-uniform rainfall; (2) the MPE has higher capability of rain detection than either gauge-only or Stage III; (3) the MPE has much higher linear correlation and lower mean relative difference with gauge measurements than the Stage III does; (4) the Stage III tends to overestimate precipitation (20%), but the MPE tends to underestimate (7%). ª 2007 Elsevier B.V. All rights reserved. * Corresponding author. Tel.: ; fax: address: Hongjie.Xie@utsa.edu (H. Xie) /$ - see front matter ª 2007 Elsevier B.V. All rights reserved. doi: /j.jhydrol

2 74 X. Wang et al. Introduction Precipitation, characterized by high spatial and temporal variation, is one of critical inputs for hydrological modeling. It is also an important factor influencing agriculture, water resources and ecosystems. Accurate measurements of precipitation are very important for all rainfall-related applications. Traditionally, rain gauges physically measure rainfall accumulation at a point (324 cm 2 ) and generally provide good quality data for a small area. These point measurements of precipitation have been used in all kinds of hydrological models (Jayakrishnan et al., 2004). Problems with gauge rainfall measurements were documented in several studies (e.g., Legates and DeLiberty, 1993). Another problem with rain gauge networks is that they are subject to degraded levels of accuracy with increased rainfall intensities such as those associated with flood producing storms. In general, rain gauge networks are not capable of detecting precipitation at the resolution and extent necessary for most hydrometeorology application. Errors caused by inadequate gauge representation of precipitation fields are typically amplified in runoff predictions (e.g., Finnerty et al., 1997). Weather radar measurements of precipitation, which provide precipitation data with much higher spatial resolution compared to rain gauges, have served meteorology for over 40 years and hydrology around 20 years (Krajewski and Smith, 2002). The installation of the Next Generation Weather Radar (NEXRAD) system across the United States in the early to mid 1990s by National Weather Service (NWS), which includes more than 160 WSR-88D (Weather Surveillance Radar-1988 Doppler) radars, has revolutionized the NWS forecast and warning programs through improved detection of severe wind, rainfall, hail, and tornadoes (Fulton, 2002). NEXRAD rainfall products in the river forecast centers have four stages (I IV) according to the sequentially increasing amount of preprocessing, calibration, and quality control performed (Fulton, 2002; Jayakrishnan et al., 2004; Xie et al., 2006). The radar Stage III precipitation products are hourly accumulation and have a spatial resolution of 4 km by 4 km with Hydrologic Rainfall Analysis Project (HRAP) after being calibrated with gauge observation and combined individual radar observations to cover an entire river forecast center. The Multisensor Precipitation Estimator (MPE), developed by the NWS Office of Hydrology in March 2000, is a product that merges rainfall measurements from rain gauges, and rainfall estimates from NEXRAD and Geostationary Operational Environmental Satellite (GOES) products. The NWS West Gulf River Forecast Center (WGRFC) switched from Stage III to MPE as the preferred precipitation estimation program in October 2003, and ended Stage III in December, Thus, since January 1, 2005, only MPE has been produced and distributed by the WGRFC (Greg Story, personal communication, April 2005). In spite of better spatial representation of rainfall variability by radar compared with rain gauge networks, there are limitations of radar estimates due to data contamination and uncertainty issues (Smith et al., 1996; Legates, 2000; Xie et al., 2006). In particular, radar rainfall overestimation is caused by the presence of hail, large raindrops, or melting; underestimation occurs due to small raindrops, dry ice, attenuation, truncation error, and beam blockage. Minimizing these errors has been one of the major tasks in radar meteorology for decades (Schmid and Wuest, 2005). Many approaches have been developed to validate NEX- RAD rainfall using rain gauge data (Xie et al., 2006; Habib et al., 2004; Jayakrishnan et al., 2004; Ciach et al., 2003; Habib and Krajewski, 2002; Ciach and Krajewski, 1999a). In the course of validation for different rainfall measurements, time-space ambiguities and sampling volume difference are major causes for the difference between different instruments (Ciach and Krajewsk, 1999b). The impact of the rain gauge representativeness error on the radar gauge difference can be as large as 50 80% for instantaneous and hourly rainfall at a grid size of 3 km by 3 km (Kitchen and Blackall, 1992). Thus, a critical issue is how to get a measurement representing areal rainfall within a radar cell. The most accurate method is to use a very high-density rain gauge network (4 10 gauges) within each radar cell (Habib et al., 2004; Ciach et al., 2003; Habib and Krajewski, 2002; Ciach and Krajewski, 1999a). However, this can be costly in terms of equipment and labor to maintain and implement. As suggested by Gage et al. (1999) that uniform rainfall, like light widespread stratiform rain, would minimize the time-space ambiguities and sampling volume difference between different instruments, thus is preferred for calibration purpose. In this study, the assumption is that hourly point rain gauge measurement could represent areal rainfall within a radar cell only for spatially uniform rainfall events, and that the point-area error can be best mitigated in uniform rainfall events. For rainfall events with high spatial variability, point rain gauge measurements cannot be directly assumed to be representative of conditions across a radar grid cell. However, they can be used to some extent to assess the spatial uncertainty (i.e., as representing a single realization from a grid cell probability density function) and the difference of probability of rain detection between radar and gauge. To test the assumption, this study develops algorithms to (1) select spatially uniform rainfall events based on hourly radar rainfall measurements and (2) use geospatial statistics to quantify the rainfall difference between gauge network and radar for those uniform events. To the best of our knowledge, this is the first time that the MPE product is quantitatively evaluated for its performance in a period of a full year (2004). The same rain gauge network is also used for validating the Stage III product for The quality of MPE and Stage III can thus be indirectly compared. Study area The selected study area was determined by the availability of high-density rain gauge data. The area lies in the Hill Country of central Texas, covering the Upper Guadalupe River Basin and partly intersecting the recharge zone of the Edwards Aquifer (Fig. 1). Central Texas has a high occurrence of flash flooding, especially in the rapidly growing and urbanizing corridor from San Antonio to Austin, where enhanced orographic lift along the Balcones Escarpment is co-located with the greatest elevation gradient (thus concentrating runoff and flood potential) from the Texas Coastal Plains to the Hill Country. The Edwards Aquifer is

3 Validating NEXRAD MPE and Stage III precipitation products 75 Figure 1 Study area: the Upper Guadalupe River Basin in Texas Hill Country. the major water source for over 1.7 million people along this corridor. Precipitation is the only recharge water source for the aquifer. Accurate measurements of rainfall over these areas are critical model inputs not only for water source budget but also for flood forecasting. Thus, knowing the quality of those continuous and spatially distributed NEXRAD rainfall products and improving it if possible are very important. Data Rain gauge data provided by Guadalupe-Blanco River Authority (GBRA) cover four counties: Kerr, Kendall, Comal, and Guadalupe (Fig. 1). These gauges have not being incorporated into the NEXRAD Stage III or MPE data products, so they should provide an independent evaluation of the quality of the NEXRAD products. According to the progress of Stage III improvements, especially the correction of truncation error started in 2002 (Fulton, 2002), and the development of MPE product, two years of data were selected for evaluation. The Stage III of year 2001 was just before the correction of truncation error and 2004 was the first available year for MPE data. There were 32 gauges available in 2001 (in Kerr, Kendall, and Comal counties) and 50 in 2004 (18 more gages were added to the network in the rest of the Comal County and in Guadalupe County). Some gauges have missing data for one or several months. All rain gauge data have a 6-min accumulation time-step. We develop several Visual Basic for Applications (VBA) scripts to aggregate the 6-min gauge data into hourly accumulations and convert their local time (Central Standard Time) to Coordinated Universal Time (UTC) used in the NEXRAD products for further analysis. The naming convention of MPE is slightly different from Stage III. An automatic approach that was originally developed by Xie et al. (2005) for processing Stage III rainfall data is revised to process the MPE data. These steps include untarring monthly files into hourly files, uncompressing the hourly files into XMRG format and then converting the XMRG files first into ASCII text format and then into GIS grid files, and defining and converting the HRAP projection into the Texas Center Mapping System. Finally, the hourly MPE rainfall data collocated with 50 rain gauges were extracted and transferred into ASCII text format file for further comparison and analysis. A detailed description on how to retrieve the Stage III data was documented in Xie et al. (2005). Methodolgy The NEXRAD MPE and Stage III precipitation data were evaluated using gauge data from uniform rainfall events. The definition of uniform rainfall is closely associated with time and space scales. It is not an absolute physical phenomenon. For example, physically, a fast-moving storm could not be assumed as a uniform rainfall in that short period (couple of minutes) in meteorological processes. However, during a longer period, like an hour or a day, the total rainfall amount at a particular area could be very similar from one place to another, thus this rainfall could be treated as a uniform rainfall event in that area. In this paper, time and space scales for a uniform rainfall event are hourly and 3 by 3 radar cells. A longer time and/or smaller spatial scale could get more uniform rainfall events. Ideally, independent ground truth datasets should be used to identify these uniform rainfall events. However, such datasets of very high-density (like 4 8 gauges/radar cell) are unavailable at a watershed scale. To define uniform rainfall events, radar data is the only available data source at present with high spatial and temporal resolution over a watershed. We

4 76 X. Wang et al. realize that the error of the radar data itself may cause some uncertainties on the determination of uniform events. But the chance of these uncertainties to change an originally non-uniform event to a uniform event or a uniform event to a non-uniform event is very low. Thus, to identify uniform rainfall events, we used radar data itself in this paper. An automated method was first developed to calculate the rainfall coefficient of variation (CV) of radar cells. The method is based on a moving window of 3 by 3 radar cells, through calculating the CV of nine cells. A threshold value of CV used for uniform rainfall event is defined totally based on the frequency distribution of CV values and the correlation coefficients between radar and gauge rainfall. For those uniform rainfall events (CV less than or equal to the threshold value), the gauge collocated at the center radar cell (in the 3 by 3 radar cells) is then used as the ground truth to validate the radar rainfall of that cell. Below are the definitions of various statistic parameters used in the study: (a) Coefficient of variation (CV) measures the relative scattering in data with respect to their mean (Xie et al., 2006). It is defined here as the ratio of standard deviation of hourly radar precipitation to the mean hourly precipitation in a 3 by 3 radar cell window. It is used to scale the spatial variability of precipitation. (b) Pearson correlation coefficient (R) is between the hourly radar rainfall and rain gauge rainfall (paired and non-zero rainfall value) (Xie et al., 2006). (c) Radar probability of rain detection (POD), which directly expresses the probability that a rainfall can be detected within a radar cell by the radar or gauge, is proposed and described in Eq. (1). It is modified from the CPOD used by McCollum et al. (2002) and Xie et al. (2006). CPOD was defined as the conditional probability of rain detection that radar can detect a rainfall if a collocated rain gauge detects as Eq. (1 0 ). Since neither can detect all rainfall events that occur within a radar cell, the CPOD only tells the relative probability that radar or a gauge can detect a rainfall occurring within a radar cell. In contrast, POD represents the absolute probability of radar or gauge detects a rainfall within a radar cell, assuming that all rainfall events occurring within a radar cell can be detected together by either the rain gauge or radar in combination, although neither of them can detect all events individually: P r ¼ PðR r > 0jR r > 0orR g > 0Þ R Count ¼ ð1þ R Count þ G Count Pairs R G P rg ¼ PðR r > 0jR g > 0Þ ¼ Pairs R G ð1 0 Þ G Count where P r is the radar s POD, R_Count (G_Count) is the total hours that radar (gauge) detects rainfall, Pairs_R_G is rainfall hours being concurrently detected by both radar and rain gauge, and P rg is radar s CPOD. (d) Similar to radar POD, the gauge POD directly expresses the probability that a gauge can detect the rainfall detected by radar or the gauge: P g ¼ PðR g > 0jR r > 0orR g > 0Þ G Count ¼ R Count þ G Count Pairs R G : ð2þ (e) Estimation bias (EB) is the normalized difference of total rainfall amount between the concurrent (or all) radar estimation and gauge measurement evaluated over a long period (one or more years) (Jayakrishnan et al., 2004), with an assumption of gauge rainfall as the true rainfall: EB ¼ Total R r Total R g ð3þ Total R g where Total_R r and Total_R g are the total rainfall concurrently or individually detected by radar and gauges. (f) Mean absolute difference (MAD) between concurrent radar rainfall (R r ) and gauge rainfall (R g ) (non-zero rainfall value) in all rainfall events (Willmott, 1982): P n i¼1 MAD r g ¼ PðR r > 0 & R g > 0Þ ¼ jr rðiþ R g ðiþj : ð4þ n (g) Mean relative difference (Rd g ) is defined as the ratio of MAD to the conditional mean of rain gauges. It is used for comparison of MAD for different scale dataset: Rd g ¼ MAD 100% ð5þ CM g (h) Conditional mean (CM) precipitation is the average of hourly rainfall accumulation over non-zero rainfall hours (Smith et al., 1996; Xie et al., 2006). Results The effects of CV on correlation (R) In a 3 by 3 radar cell window, where a rain gauge is collocated in the central cell, CV between the nine cells was used to examine the spatial variability of rainfall. The definition of a CV threshold to determine uniform or non-uniform is very challenging and there is no universal means to judge one number is better than the other. As it is seen in Fig. 2, the correlation (R) values between radar and gauges are relative to the CV values, and R values of MPE are always higher than those of Stage III under the same CV value. Using one same CV value as a threshold value for both Stage III and MPE to define uniform and non-uniform events allows comparing the agreement of radar data with rain gauge data. In Fig. 2a, overall, R value decreases as the CV value increases. For the high uniform rainfall events (CV < 0.1), accounting for 5% of concurrent rainfall pairs (hours), R value is 0.96 (see Fig. 3a). In contrast, R is 0.54 for the high non-uniform rainfall events (CV > 1), accounting for 11% of all rainfall hours. For CV < 0.5, R varies linearly with CV; for CV of , R only has little change, then dramatically decreases when CV > 0.8. Based on the frequency distribution of CV and the correlation, we take CV = 0.8 as a

5 Validating NEXRAD MPE and Stage III precipitation products 77 Figure 2 The effects of spatial variability of rainfall pattern on R between rain gauge and radar MPE precipitation data in 2004 (a) and in 2001 (b). X-axis is the spatial coefficient of variation (CV) of rainfall in a 3 by 3 radar cell window. Primary Y-axis is the R between gauge and MPE or Stage III. Secondary Y-axis is the relative frequency distribution of CV. Figure 3 Scatter plot of hourly radar MPE and rain gauge precipitation for high uniform ((a): CV < 0.1, 5% of all pairs) and high nonuniform rainfall ((b): CV > 1, 11% of all pairs) in 2004 at the Upper Guadalupe River Basin. threshold for separating uniform from non-uniform rainfall events, i.e., uniform events when CV 6 0.8, otherwise non-uniform events when CV > 0.8. In this case, uniform rainfall events account for about 68% of total MPE rainfall hours and 93% of total rainfall amount for the year 2004, while non-uniform events accounts for only 32% of total hours and 7% of total amount. For those uniform events, the collocated gauge observations of rainfall are used as the ground truth to evaluate the radar precipitation. According to this classification, in all concurrent rainfall,

6 78 X. Wang et al. which are rainfall events that concurrently detected by a gauge and radar within a radar cell, 82% of pairs are uniform rainfall events, only18% of pairs are non-uniform rainfall events (see Table 1). As for non-uniform events, the gauge data cannot be used for validation. For example, for the high non-uniform rainfall events of Fig. 3b, MPE and gauge values have the largest discrepancy, and MPE is much less than gauge rainfall (EB = 39%, R d = 78%); in comparison with 10% (EB) and 24% (R d ) for high uniform rainfall events (Fig. 3a). In 2001 (Fig. 2b), in a general trend, R decreases as CV increases. Only 2% of rainfall events are high uniform as CV < 0.1, and R equals to 0.88; in contrast, 27% of rainfall events are high non-uniform as CV > 1.0, and R only equals to Although R values fluctuate from 0.62 to 0.76 when 0.1 < CV 6 0.8, similar with in 2004, R dramatically decrease as CV > 0.8, especially as CV > 1.0. Thus, for comparison purpose, we also take CV = 0.8 as the threshold for separating uniform rainfall events from non-uniform rainfall events. Applying this threshold, the uniform rainfall events account for about 52% of total Stage III rainfall hours and 85% of total rainfall amount for the year 2001, while nonuniform events account for 48% of total hours and 15% of total rainfall amount. The proportion of non-uniform events in 2001 is higher than in To be noticed, the CV threshold value of 0.8 for determining uniform or non-uniform rainfall events for this study is totally based on the frequency distribution of CV values and the correlation coefficients between radar and gauge Table 1 Statistic comparison between uniform and nonuniform rainfall events concurrently detected by gauges and radar All GBRA data in 2001 All GBRA data in 2004 Gauge Stage III Gauge MPE Uniform pairs Uniform pairs ,844 Count % of all pairs 62% 82% Total rainfall (mm) 11,054 13,207 45,742 42,469 Amount % of all pairs 81% 87% 94% 95% Mean (mm h 1 ) MAD (mm h 1 ) R d 63% 39% Estimation bias 20% 7% R Non-uniform pairs Non-uniform pairs Count % of all pairs 38% 18% Total rainfall (mm) Amount % of all pairs 19% 13% 6% 5% Mean (mm h 1 ) MAD (mm h 1 ) R d 80% 76% Estimation bias 24% 23% R Note: Pairs are these rainfall events that are concurrently detected by gauges and radar. rainfall, i.e., the selection of the CV threshold is based on the statistical results. Thus, this CV threshold value is acceptable, although it has limitation and may not be transferable to other area or other period of datasets. Concurrent gauge and MPE rainfall in 2004 Fig. 4 shows the scatter plots of collocated and concurrent radar MPE and gauge precipitation data for uniform (82%) (a) and non-uniform (18%) (b) events in In the 12,844 pairs of uniform rainfall hours, total gauge rainfall amount is 94% of the total concurrent gauge rainfall amount; total MPE rainfall amount is 95% of total concurrent MPE rainfall amount; the statistic results are R of 0.88 (R 2 = 0.77), MAD of 1.4 mm h 1, gauge CM of 3.6, MPE CM of 3.3 mm h 1, and R d of 39%. In the 2893 pairs of non-uniform rainfall, gauge s rainfall amount is 6% of the total concurrent gauge s; MPE s rainfall amount is 5% of total concurrent MPE s; R is 0.62 (R 2 = 0.38), MAD is 0.8 mm h 1, gauge CM is 1.0 mm h 1, MPE CM is 0.8 mm h 1, and R d is 76% (see Table 1). It is clear that MPE and gauge precipitation data match much better for uniform rainfall than for non-uniform rainfall. This supports our assumption that point rain gauge measurement of rainfall could represent areal rainfall within a radar cell only for spatially uniform rainfall events, while for non-uniform events a single gauge measurement should not be used as ground truth to evaluate the radar rainfall. Concurrent gauge and Stage III rainfall in 2001 In the all concurrent rainfall events (Fig. 2b), 62% (2325 h) of them are uniform rainfall, and 38% as non-uniform rainfall. In the 2325 uniform pairs, gauge rainfall amount is 81% of total concurrent gauge rainfall amount; radar rainfall amount is 87% of total concurrent radar rainfall amount. Fig. 5 displays the scatter plots of Stage III and gauge precipitation data for uniform (a) and non-uniform (b) rainfall in For uniform rainfall, Stage III and gauge precipitation data have a linear correlation of 0.71 (R 2 = 0.50), MAD of 3.0 mm h 1, gauge CM of 4.8 mm h 1, Stage III CM of 5.7 mm h 1, and R d of 63%; for non-uniform rainfall, the R is 0.47 (R 2 = 0.22), MAD is 1.4 mm h 1, gauge CM is 1.7 mm h 1, Stage III CM is 1.3 mm h 1, and R d is 80% (see Table 1). Basic statistics of Stage III (2001) and MPE (2004) MPE in 2004 The basic statistics of Stage III, MPE, and gauge precipitation data are shown in Table 2. In all 50 gauges/radar cells of 2004, there are total 34,352 h detected by gauges or radar MPE and 15,737 pairs/h (46% of total hours) concurrently detected by radar and gauge; gauges detected 19,564 h with POD of 57%, total rainfall of 50,730 mm, and CM of 2.6 mm h 1 with maximum of 88.9 mm h 1 ; radar detected 30,525 h with POD of 87%, total rainfall of 52,989 mm, and CM of 1.7 mm h 1 with maximum of 64.7 mm h 1. Radar has a higher POD and detects more rainfall hours per radar cell than the corresponding gauge;

7 Validating NEXRAD MPE and Stage III precipitation products 79 Figure 4 Scatter plot of hourly radar MPE and rain gauge precipitation for uniform ((a): CV 6 0.8, 82% of total pairs) and nonuniform rainfall ((b): CV > 0.8, 18% of total pairs) in 2004 at the Upper Guadalupe River Basin. Figure 5 Scatter plot of hourly radar Stage III and rain gauge precipitation for uniform ((a): CV 6 0.8, 62% of pairs) and nonuniform rainfall ((b): CV > 0.8, 38% of total pairs) in 2001 at the Upper Guadalupe River Basin. radar also estimates a higher total rainfall amount with a positive EB of 5%, but the CM of radar (1.7 mm h 1 ) is less than the CM of gauge (2.6 mm h 1 ). In the 15,737 rainfall pairs, the total rainfall of MPE and gauges are 44,803 and 48,753 mm, respectively, and CM of radar (2.8 mm h 1 )is also less than CM of gauge (3.1 mm h 1 ). The radar detects less rainfall with an EB of 8% than gauges. The MAD and R d are, respectively, 1.3 mm h 1 and 42% with a linear correlation of R = 0.79 (including uniform and non-uniform pairs). As the concurrent radar and gauge rainfall pairs are only 46% of the total rainfall hours, 80% of total gauge hours (96% of total gauge rainfall amount), or 52% of total MPE hours (85% of total MPE rainfall amount), how can we compare the rest of rainfall detected at different times by gauge or radar (i.e., the non-concurrent rainfall events)? Fig. 6 displays the monthly and monthly accumulations of all (a) and concurrent (b) gauge and radar rainfall amount in Both cases show major rainfall accumulations were in April, June, October, and November. In the all rainfall plot (a): the monthly radar rainfall is higher than gauge, except in August, September, and October; radar accumulation is always higher than the gauge value (annual EB of radar is 5%). In the concurrent rainfall case (b): the monthly radar rainfall is lower than gauge rainfall except in May, and the radar accumulation is always lower than the gauge s (annual EB of 8%). From Table 2, non-concurrent gauge detected rainfall hours only account for 11% of total rainfall hours detected by radar or gauge, or 4% of gauge detected rainfall accumulation for the year (with CM of 0.5 mm h 1 ), while non-concurrent radar detected rainfall hours account for 43% of total rainfall hours, or 15% of radar rainfall accumulation for the year (with CM of 0.6 mm h 1 ). For non-concurrent cases, due to non-uniform and small rainfall rate, radar performance is better: radar detects rainfall of 8185 mm (or 14,788 h) compared with 1977 mm (or 3827 h) from gauges. Overall, the results suggest that (1) for concurrent cases, the gauge detected more rainfall amount than radar (radar underestimates), and (2) for non-concurrent cases, gauges missed more rainfall events than radar. Three reasons for the poor gauge performances in non-concurrent cases are: (1) point gauge observation and spatial heterogeneity of rainfall, (2) higher sensitivity of radar than gauge for those small rainfall events, and (3) evaporation

8 80 X. Wang et al. Table 2 Statistic comparison between Stage III, MPE and gauge precipitation data in 2001 and 2004 All GBRA data in 2001 All GBRA data in 2004 Gauge Stage III Gauge MPE All Rainfall count (h) ,564 30,525 Total count (h) 11,810 34,352 Total rainfall (mm) 17,507 20,266 50,730 52,989 Estimation bias 16% 5% Mean (mm h 1 ) Maximum (mm h 1 ) POD CPOD Non-concurrent Rainfall count (h) ,788 Count % of all 43% 25% 11% 43% Total rainfall (mm) Amount % of gauge or radar 23% 25% 4% 15% Mean (mm h 1 ) Estimation bias 30% 314% Gauge_Radar pairs Gauge_Radar pairs ,737 Count % of all 32% 46% Total rainfall (mm) 13,565 15,124 48,753 44,803 Amount % of gauge or radar 77% 75% 96% 85% Mean (mm h 1 ) MAD (mm h 1 ) R d 67% 42% Estimation bias 12% 8% R Notes: All means all the rainfall individually detected by rain gauges or radar in an entire year in the collocated radar cells and includes concurrent Gauge_Radar pairs and non-concurrent rainfall. Gauge_Radar pairs refers the rainfall concurrently detected by gauge and radar in the collocated radar cells in a year. Non-concurrent refers rainfall separately detected by radar but not by gauges, or detected by gauges but not by radar. Figure 6 Time series plots of monthly and monthly accumulation of 50 collocated radar and gauge rainfall in 2004: all rainfall being individually detected by gauge or radar (a) and rainfall concurrently detected by gauge and radar (b). Acc in the legend is the abbreviation of accumulation. effects of less than one tip in the gauge due to high temperature and/or low relative humidity. It is also possible that some small rainfall maybe never reach the ground (to be recorded by rain gauge) due to evaporation below radar beam. The radar underestimates of the concurrent case (Fig. 6b) can be further separated into concurrently uniform events (Fig. 7a) and concurrently non-uniform events (Fig. 7b). It is found that the three figures have similar trend

9 Validating NEXRAD MPE and Stage III precipitation products 81 Figure 7 Time series plots of monthly and monthly accumulation of 50 collocated radar and gauge rainfall in 2004: concurrently uniform rainfall (a) and concurrently non-uniform rainfall (b). of distribution. But their relative difference between gauge and radar is different. For example, radar EB is 8% for all concurrent events, 7% for concurrent uniform events, and 23% for concurrent non-uniform events. The 7% of concurrent uniform events is what we called the radar underestimates, since, only in this case, that gauge rainfall can be used as a ground truth to validate the radar estimates. Stage III in 2001 In all 32 gauges/radar cells of year 2001, there were total 11,810 h detected by gauges or radar Stage III and 3771 h (32% of total hours) concurrently detected by radar and gauge (see Table 2); gauges detect 8812 h with POD of 75%, total rainfall of 17,507 mm, CM of 2.0 mm h 1, and maximum of mm h 1 ; Stage III has 6769 h with POD of 57%, total rainfall of 20,266 mm, CM of 3.0 mm h 1, and maximum of 67.5 mm h 1. Stage III has lower POD and detects less rainfall events per radar cell than a gauge while having higher total rainfall amount than a gauge, with an EB of 16%. In the 3771 concurrent rainfall pairs, total gauges rainfall is 13,565 mm with CM of 3.6 mm h 1 ; total Stage III rainfall is 15,124 mm with CM of 4.0 mm h 1. Stage III has more rainfall with an EB by 12% than gauges. Their MAD is 2.4 mm h 1, and mean relative difference is 67% with a linear correlation of R = 0.68 (including uniform and nonuniform pairs). The concurrent Stage III and gauge rainfall is only 32% of the total rainfall hours, 43% of total gauge hours and 77% of total gauge rainfall amount, or 56% of total Stage III hours and 75% of total Stage III rainfall amount (see Table 2). Fig. 8 shows the monthly and monthly accumulation rainfall of all (a) and concurrent (b) gauge rainfall and Stage III rainfall in In the all rainfall case (a): the monthly rainfall of Stage III is higher than gauge rainfall except in January, February and October; and monthly accumulation of Stage III is lower than gauge before July, then higher than the gauge rainfall thereafter (the annual EB of radar is 16%). In the concurrent rainfall case (b): monthly rainfall of Stage III is higher than gauge rainfall except in October and November; and monthly accumulation of Stage III is always higher than gauge for the entire year (annual EB of Stage III is 12%). However, in non-concurrent case, the total gauge rainfall hours (5041 h) is much more than those of Stage III (2998 h), though the total rainfall amount from gauges (3942 mm) is less than that from Stage III (5142 mm). This means that (1) in concurrent case, Stage III detected more rainfall amount than gauge and (2) in non-concurrent case, gauge could miss many rainfall events as observed in 2004 Figure 8 Time series plots of monthly and monthly accumulation of 32 collocated radar and rain gauge rainfall in 2001: all rainfall being individually detected by gauge or radar (a) and concurrent rainfall detected by gauge and radar (b).

10 82 X. Wang et al. while Stage III missed even more, mainly due to the truncation error of the Stage III algorithm (Xie et al., 2006). Fig. 9 displays the monthly accumulations of concurrently uniform (a) and non-uniform (b) gauge and Stage III rainfall amounts in In Fig. 9a, the concurrently uniform rainfall has similar trend as the all concurrent rainfall in Fig. 8b, but having larger R d of 20% (see Table 1). In Fig. 9b of the non-uniform concurrent rainfall, Stage III monthly rainfall is less than gauge monthly rainfall except in April, June, and July; and monthly accumulation of Stage III is always less than gauge with EB of 24%. It suggests that the Stage III s underestimation of non-uniform rainfall (mainly due to truncation error) cancels out in somehow its overestimation of uniform rainfall (EB = 20%), which results in an overall effect of EB by 12% for all concurrent rainfall pairs. Regionally, for all concurrent radar and gauge rainfall pairs, the correlation (0.79 for MPE and 0.68 for Stage III) between radar and gauge is much higher than that of Stage III for the Sevilleta National Wildlife Refuge (NWR), New Mexico, where the best R in non-monsoon seasons is only 0.41 and even lower in monsoon seasons (Xie et al., 2006). It suggests that NEXRAD Stage III itself has better quality in the central Texas than in the semiarid New Mexico mountain area. Frequency distribution of rainfall intensity Figs. 10 and 11 show, respectively, the absolute and relative frequency distributions of all rainfall count (h) and rainfall amount (mm) of radar (MPE or Stage III) and gauge versus rainfall intensities (rate) in 2001 and 2004, in terms of all collocated gauges and radar cells. The rainfall intensity is classified into six categories: <0.5, , , , , >20.0 mm h 1. In 2004 (see Fig. 10b), MPE detects more rainfall count (total 30,525 h) than gauges (total 19,564 h), particularly for small rainfall events (<0.5 mm h 1 ). Among which MPE detects 16,908 h and total 2330 mm, gauges only detect 8239 h and 1749 mm. In the rest of categories, MPE also detects more rainfall hours than gauges when rainfall intensity is less than 10 mm h 1 ; and almost similar rainfall hours as rainfall intensity is larger than 10 mm h 1 ; while the gauge rainfall amount becomes much larger than radar s when rainfall rate is larger than 20 mm h 1. Unlike 2004, in 2001 (see Fig. 10a), gauges (total 8812 h) detect more rainfall count than Stage III (total 6769 h), especially when rainfall intensities less than 2.0 mm h 1. For example, when rainfall intensities less than 0.5 mm h 1 Figure 9 Time series plots of monthly and monthly accumulation of 32 collocated radar and rain gauge rainfall in 2001: concurrently uniform rainfall (a) and concurrently non-uniform rainfall (b). Figure 10 Absolute frequency distribution of all rainfall count (h) and rainfall amount (mm) of Stage III and gauge in 2001 (a) and MPE and gauge in 2004 (b) versus rainfall rate.

11 Validating NEXRAD MPE and Stage III precipitation products 83 Figure 11 Relative frequency (%) distribution of all rainfall count (rainfall hours) and rainfall amount (mm) of gauges (a) and radar (b) versus rainfall rate in 2001 and and between 0.5 and 2.0 mm h 1 gauges detect rainfall count of 3776 and 3055 h and total rainfall amount of 843 and 2870 mm, while Stage III only detects rainfall count of 2349 and 2077 h and total rainfall amount of 459 and mm, respectively. The main reason is the truncation error of Stage III algorithm for small rainfall events (Xie et al., 2006). For rainfall intensity >2 mm h 1, the rainfall amount is 79% of total rainfall amount for gauge and 87% of total rainfall amount for Stage III, indicating that Stage III detects more rainfall hours and rainfall amount than gauges in larger rainfall intensities. Overall, Fig. 11a shows that the rain gauge data of 2001 and 2004 have similar frequency distribution of rainfall count and rainfall amount, though gauge data in 2004 has little bit lower frequencies of rainfall count and amount for small rainfall events but higher frequencies for larger rainfall events. This is mainly because year 2004 was wetter and had more storm rainfalls than 2001 in the study area. When rainfall intensity is less than 0.5 mm h 1, the rainfall count is 41% and 43% of the total rainfall count in 2001 and 2004, but the rainfall amount is only 5% and 3% of the total rainfall amount during the entire year, respectively. When rainfall intensity is larger than 5 mm h 1, the rainfall count (the sum of last three) is 9% and 13% of the total rainfall count, but their rainfall amount is 58% and 67% of the total rainfall amount in 2001 and 2004, respectively. It suggests that low-intensity rainfall events play a minor role in the total rainfall amount and water resource management, and the high-intensity rainfall events play a critical role in the total rainfall amount, water resource and flood management. Overall, similar performance of rain gauge data in 2001 and 2004 indicates the quality of rain gauge networks is good and consistent between 2001 and 2004, which projects that the comparison between gauge and Stage III (2001) and between gauge and MPE (2004) should be reasonable and should eventually provide an effectively and efficiently relative quality assessment of Stage III and MPE. MPE data in 2004 have much higher frequency for small rainfall events (<0.5 mm h 1 ) than Stage III (see Fig. 11b) due to truncation error of the Stage III data (Xie et al., 2006). The MPE algorithm fixed the truncation error and greatly improves the capability of detecting small rainfall events. The radar frequency of rainfall amounts has similar distribution as well as gauge observation. The rainfall amounts in low-intensity (<0.5 mm h 1 ) events is only 2% (2001) and 4% (2004) of the total rainfall although the rainfall count is 35% (2001) and 55% (2004) of total rainfall count for Stage III (2001) and MPE (2004), respectively. Meanwhile, the rainfall amount of high-intensity (>5.0 mm h 1 ) rainfall events is 67% and 60% of the total rainfall although their rainfall count is only 16% and 9% of total count for Stage III in 2001 and MPE in 2004, respectively. Discussion and conclusions Point-area measurement error Given the spatial variation of rainfall, the gauge recorded rainfall amount cannot represent the ground truth of areal mean rainfall for high heterogeneous or non-uniform rainfall events within a radar cell (4 km by 4 km) as observed. Table 3 lists the 10 pairs of gauge and radar rainfall values with top maximum absolute differences between gauge and MPE. Those are extreme cases that have maximum radar gauge difference with MAD by 35.6 mm h 1. Among the 10 gauge radar pairs, there are seven pairs that gauge rainfall has larger value than MPE, while there are three pairs that MPE has larger values than gauge. As pointed out by Xie et al. (2006), because of the great spatial variation of storm rainfall, a large rainfall rate observed by the gauge may not represent the areal average rainfall within a radar cell, resulting in a great difference between the gauge and radar observations. Particularly at 1800 UTC on 9 June, 2004, the hourly gauge rainfall at the gauge kr2 was 42.9 mm, while the collocated areal (average) radar rainfall was only 2.3 mm. In contrast, radar MPE may detect large precipitation for a cell, but a gauge only detects small precipitation because of its heterogeneity. For instance, at 0400 UTC on 14 May, 2004, rainfall occurred in cells kr19, kr20, and kr22. MPE detected 64.4, 24.4, and 12.8 mm, respectively, but the collocated gauges only reported 23.9, 0.8, and 0.3 mm, respectively. CV values at these three MPE cells are, respectively, 0.86, 1.22, and 0.91, all larger than the threshold value of 0.8. Those are typically non-uniform rainfall events, and the corresponding single point gauge measurements cannot represent the ground truth to evaluate the radar rainfall of these cells. In addition, one possible explanation for these extreme large discrepancies is that the funnels of some

12 84 X. Wang et al. Table 3 Comparison of gauge and radar rainfall among the top 10 pairs with maximum absolute difference in all 50 gauges/ radar cells in 2004 Date Time (UTC) Gauge ID MPE (mm h 1 ) Gauge (mm h 1 ) Absolute difference 10/2/ gp /9/ gp /23/ gp /2/ ck /24/ kr /5/ gp /30/ gp /25/ kr /14/ kr /9/ kr CM (mm h 1 ) or MAD (mm h 1 ) rain gauges may have been temporarily clogged by grass, bird debris or others. This could delay the onset of precipitation, causing the event to be delayed and underestimated, and could cause erroneous large rainfall later if the clog breaks or drains. Point-area representativeness error is a major problem in radar rainfall estimation. This study showed that the spatial variability of rainfall is a major component of the difference between gauge rainfall and radar rainfall. Single gaugebased calibration to Stage II, prior to the Stage III and MPE products, may be good for uniform rainfall events, but could bring more uncertainty to non-uniform rainfall events. Therefore, non-uniform rainfall events will require more complex analysis schemes for calibration and evaluation. This study proposed a new method by using the radar rainfall to examine the spatial variability (CV) of rainfall in a 3 by 3 radar cell window, and then uniform rainfall can be selected totally based on CV. The collocated gauge observation of uniform rainfall is then used as the ground truth to evaluate the radar rainfall. Thus, we can accurately evaluate the radar rainfall estimates even without a high-density gauge network. However, the comparison holds only for spatially uniform rainfall events and not across the entire spectrum of precipitation events. Uniform versus non-uniform rainfall events Fig. 12 compares the relative frequency distribution of rainfall count versus rainfall rate for uniform rainfall and nonuniform rainfall for Stage III in 2001 (a) and for MPE in 2004 (b). Since the Stage III data has a significant truncation error of light rainfall events, so we also discuss the MPE data, while the Stage III data is only shown as reference. If rainfall rate of <0.5 mm h 1 is assumed as light rainfall event, about 55% of MPE rainfall events (Fig. 11b) are actually light rainfall events. Among these, 52% are uniform rainfall, and the rest 48% are non-uniform rainfall (Table 4). Of all non-uniform events, 83% are actually light rainfall, which differs from what usually means that light rainfall events as uniform rainfall. For all the uniform rainfall events, 43% are light rainfall events and another 30% are relatively light rainfall events ( mm h 1 ). This suggests that most of the uniform rainfall events are light or relatively light rainfall events, but light rainfall events are not necessary uniform rainfall events (in this time and spatial scales). Only 5% of uniform rainfall events are heavy rainfall (>10 mm h 1 )(Fig. 12b) due to the fact of limited heavy rainfall events. However, as shown in Table 4 (again, the stage III data is shown as reference), most (>90 %) of the Figure 12 Relative frequency (%) distribution of rainfall count (rainfall hours) versus rainfall rate for uniform and non-uniform rainfall events for Stage III (a) in 2001 and MPE (b) in 2004.

13 Validating NEXRAD MPE and Stage III precipitation products 85 Table 4 Distribution of non-uniform and uniform rainfall count (h) at different rainfall intensities Rainfall intensities (mm h 1 ) Stage III in 2001 MPE in 2004 Total count Non-uniform (%) Uniform (%) Total count Non-uniform (%) Uniform (%) > , < , > heavy rainfall events indicated by MPE are uniform rainfall (in this time and spatial scales). This is different from what usually means that most heavy rainfall events are non-uniform rainfall events. Probability of rain detection (POD) and CPOD Both POD and CPOD can examine the relative capability that radar or gauge detects the rainfall events within a radar cell (see Table 2). As described in Eqs. (1) and (1 0 ) in the Section Methodology, CPOD is a relative probability, i.e., radar CPOD is the probability that radar detect a rainfall given a collocated gauge detects it; but POD is an absolute probability, i.e., radar POD is the probability that radar can detect a rainfall that occurs within a radar cell, assuming that gauge and radar together can detect all rainfall events occuring in that radar cell, and gauge POD is the probability that gauge can detect a rainfall that occurs within a radar cell. For example, among all rainfall data of 50 gauges/cells in 2004, gauge POD equals to 0.57, meaning that gauges have probability of 57% to detect rainfall events occuring within a radar cell; while radar (POD 0.89) has the probability of 89% to detect rainfall in (a radar cell) (see Table 2). It is reasonable that radar has higher probability to detect rainfall since radar can almost see any rainfall event within a cell (16 km 2 ) while a gauge can only see rainfall occuring within 1 m 2 of the gauge location. Most missed rainfall events that cannot be detected by gauges are small rainfalls (<2 mm h 1 ). For example, in the 2004 case (not shown), among the gauge missed rainfall events, 78% of them is less than 0.5 mm h 1 and 95% of them is less than 2 mm h 1. In addition, radar may not see all rainfall events because of radar beam overshooting in stratiform rainfall during cold season, beam (by terrain) blockage, and radar sensitivity (Xie et al., 2006; McCollum et al., 2002). Another possible reason that radar may miss rainfall events is that the WSR- 88D precipitation mode, radar volume scan frequency, is 1 scan/4 6 min. So fast-moving storms or rapid development/decay of storms could lead to significant underestimates of rainfall. Ideally, gauges and radar together are able to detect almost all rainfall events. Therefore, POD could represent the absolute probability that radar or gauge detects a rainfall. In all cases of Table 2, it is found that CPOD is always smaller than the POD, meaning that CPOD underestimates the actual probability of rain detection of gauge and radar. So POD proposed in this paper should be a better scale than the CPOD. The much lower POD of Stage III (2001) than MPE (2004) and gauge again illustrates the existence of serious truncation error in the Stage III precipitation processing system. Overestimation and underestimation In the concurrent uniform rainfall, the monthly Stage III is higher than gauge rainfall except for October and November, and the annual accumulation of Stage III is higher than gauge s by 20% in 2001, which is different from the study of Young et al. (2000) and Jayakrishnan et al. (2004), but consistent with Xie et al. (2006) whose results shows that Stage III in overestimates gauge s by 33% in monsoon season and 55% in non-monsoon season. In all cases, the CMs of Stage III are always larger than those of gauges, which is also consistent with the results from Xie et al. (2006). While for MPE, the monthly MPE is always lower than gauge rainfall except for May, and the annual accumulation of MPE rainfall is lower than gauge s by 7% in 2004; CMs of MPE are always less than those of gauges. For all rainfall individually detected by gauge or radar, MPE has more rainfall count (h) by 56.0% and a little bit more rainfall amount (5%) than gauges in In 2001, however, Stage III has less rainfall count and amount in January, February, and October, and less rainfall count and more rainfall amount in the rest of the year. Even the total rainfall count of Stage III is 23% less than gauge s, its total rainfall amount is still 16% higher than the gauges. This indicates that Stage III truncates the small rainfall events but much overestimates larger rainfall events. Improvements of MPE over Stage III The improvements of MPE reflected in two aspects: (1) the truncation error of Stage III algorithm is fixed in the MPE product, i.e., the higher POD of MPE than gauge and Stage III, and the lower POD of Stage III than gauge and MPE; the much more higher rainfall counts/ratio of MPE for small rainfall (<0.5 mm) than gauge and Stage III, and the much lower rainfall counts/ratio of Stage III than gauge and MPE; (2) the MPE has higher R, lower MAD and R d with gauge observation than Stage III does, which means that MPE has better agreement with gauge observations.