Comparing NEXRAD and Gauge Rainfall Data Near San Antonio, TX

Transcription

1 N. Johnson 1 Comparing NEXRAD and Gauge Rainfall Data Near San Antonio, TX Surface Water Hydrology Final Project Nate Johnson 5/5/05

2 N. Johnson 2 Table of Contents Table of Contents Introduction Hydrologic Modeling and NEXRAD Timeseries, Arc Hydro, and GIS Motivation and Objectives Background Time Series File Structure Radar Rain Gage Comparisons Methodology NEXRAD to WDM through Arc Hydro Timeseries Procedure of Application NEXRAD Gauge Comparisons Procedure of Analysis Results Cumulative Difference Results Storm-Bias Results Conclusions / Future Work References Appendix A... 30

3 N. Johnson 3 1. Introduction 1.1. Hydrologic Modeling and NEXRAD Traditional hydrologic modeling applications have relied upon rain gauges for rainfall estimates, however, even dense gauge networks rarely capture the immense spatial variability of rainfall. While differences between rain gauge point measurements and radar measurements have been amply documented (Wilson, Jayakrishnan, Neary), an increased understanding of the factors that affect biases and errors in the data has resulted in better calibration techniques and more accurate data in recent years. (Smith and Krajewski 1991, Kitchen and Blackall 1992) Radar estimates for rainfall provide a nearly spatially continuous estimates of rainfall, which is appealing to hydrologic modeling applications. Though data has been used experimentally since the early 1960 s, confusion and misunderstandings about the accuracy and reliability of the data have prevented its widespread use for operational applications in hydrologic engineering. (Wilson 1979, Jayakrishnan et. al. 2004) Recently, researchers and practitioners have shown a growing interest in using radar estimates for rainfall in hydrologic modeling. (Neary et. al. 2004) 1.2. Timeseries, Arc Hydro, and GIS NEXRAD data is collected, stored, and distributed by members of the weather and atmospheric sciences community. It is widely used by researchers in this area partly because the format of the data is a common one for them and easy to access. Though the data is available free of charge through a web-based distribution system, accessing the data and preparing it for use in hydrologic models is not a trivial task. Radar rainfall data is stored and distributed in the form of spatially continuous data sets describing the rainfall over a single time interval. Hydrologic modeling applications typically require rainfall data in the form of a timeseries of values at a specific spatial location, or a timeseries of spatially averaged values over an area. The vast difference in these two data structures is not easy to overcome and makes the radar data inaccessible to many hydrologic modelers. Another technology that has been extremely influential on the field of hydrologic modeling over the past 25 years is Geographic Information Systems (GIS). GIS

4 N. Johnson 4 technology has provided tools for spatial analysis of environmental data and a structure for organizing and managing hydrologic data. ESRI is the most widely used GIS software company, and a customization of their ArcGIS software for water resources applications called Arc Hydro has been developed in conjunction with the Center for Research and Water Resources at the University of Texas at Austin. (Whiteaker 01) In addition to tools helpful for preparing raw data for hydrologic modeling applications, the Arc Hydro tools contain a structure for organizing data in a spatial and temporal domain. The Arc Hydro Timeseries structure has been used by the South Florida Water Management District to store and distribute NEXRAD data and to perform spatial analysis on the data using ESRI s GIS software. The Arc Hydro timeseries structure is different from both the gridded NEXRAD structure and traditional hydrologic modeling timeseries. It uses a relational database to allow queries accessing data across space or time domains Motivation and Objectives The motivation for this project is the desire to use radar rainfall information in hydrologic modeling applications. The USGS in San Antonio, TX uses the Hydrologic Simulation Program in Fortran (HSPF) to do much of their hydrologic and water quality modeling. They currently use a relatively sparse gauge network and would like to investigate the use of NEXRAD data to provide greater spatial coverage. The objectives of the project presented here fit within this context and are twofold. The first objective is to obtain the NEXRAD data and prepare it for use in HSPF models. The second objective is to compare the NEXRAD data and the gauge data currently used for hydrologic modeling in order to begin to understand the differences between the two. Additional analysis will be necessary quantify the implications of the two data sources on the results of HSPF modeling applications. The objective of the project presented here is only to compare the rainfall estimates from the two sources.

5 N. Johnson 5 2. Background 2.1. Time Series File Structure Many spatially continuous environmental data sets are stored in a gridded format. The NetCDF (UniData 2005), grib (BADC 2005), and xmrg (NOAA 2005) are all examples of gridded formats developed by the atmospheric and whether science communities to store observed and modeled data sets that represent a continuous spatial field. A truly spatially continuous data set would have values defined at every infinitesimally small finite portion in space. Spatially continuous data sets are typically stored on a grid format, where the values at the centers or nodes of the grid represent a spatially averaged value. Usually these data sets, while considered continuous in space, represent only a single point in time. Temporally, the value could represent any type data, an average or cumulative value over the previous or subsequent timestep, or an instantaneous measurement at that exact time. A collection of these gridded files spaced at regular time intervals make up a continuous spatial and temporal description of conditions in the environment. The NetCDF file structure can store a collection of temporally consecutive, spatially continuous data sets in a single file, while individual grib and xmrg files are necessary for each timestep. Whatever the details of the overall file structure, gridded data sets are used most often to represent spatially continuous data at a single point in time. One benefit of a gridded data structure is that because values are stored at regular spatial intervals, it is not necessary to store spatial Figure 1 xmrg File Extracted to ASCII text format. information with every value. Metadata at the beginning of gridded data sets provides all the necessary spatial information to locate a value in the data set for any point in space.

6 N. Johnson 6 For instance, Figure 1 shows the header information and a few data values from an xmrg file (here extracted to ascii format) used by the National Weather Service s (NWS) West Gulf River Forecast Center (WGRFC) to store NEXRAD rainfall data. The data set values describe a spatial average (over the grid cell) of the cumulative amount of rainfall that fell on August 1, 2001 between 12:00 AM and 1:00 AM at each of about 164,000 HRAP grid cells covering the southeastern US. Metadata at the beginning of the file give the coordinates (in HRAP X and Y grid cell numbers) of the lower left corner of the data set, and the number of rows and columns contained in the data set. After the metadata, a simple list of about 164,000 numbers, delimited by a single space, provide the values for each of the grid cells described in the metadata. While grid data structures are used widely to store spatially continuous data, such as NEXRAD rainfall and the results of atmospheric models, most hydrologic models require input data sets to be structured as a series of values at regular intervals describing an individual location. The.wdm and.dss formats are examples of this type of timeseries data structure where metadata including spatial location, units, and type of data are followed by a list of values or date-time value pairs. Each data set in this structure represents a timeseries of values at a single spatial location. Figure 2 shows an example of a.wdm data set (here extracted to ASCII format) consisting of a list of metadata Figure 2 wdm File extracted to ASCII text format followed by time value pairs for rainfall at a gauge near San Antonio. This structure is distinct from the grid data structure discussed earlier, where a data set represents a single point in time at all spatial locations. Figure 3 illustrates the differences between the two types of timeseries structures using the space time variable domains.

7 N. Johnson 7 In both the gridded and timeseries cases, the data sets represents a series of values that all have the same variable type, but are continuous in either space or time. Figure 3 a) Timeseries structure representing all time at a single location. b) grid structure representing all locations for a single time. If data from a space-centered (gridded) data structure is used to develop input data sets for modeling applications, which require a time-centered (timeseries) structure, the enormous difference in the file structures must be contended with. For most local-scale hydrologic modeling applications only a small area of the entire spatial domain contained in large NEXRAD data sets is required. However, to access even a few cells in a large gridded file, it is necessary to open the entire data set and search through to find the few required values. When building a temporally continuous timeseries from a collection of spatially continuous data sets it may be necessary to open thousands of gridded files and pick out only a few values from each. If the data is to be stored in the time-centered (timeseires) structure described above, each value retrieved from a space-centered (gridded) data set must be placed in its own time-centered data set corresponding to its spatial location. The possibility of manually extracting data from large gridded data sets to build modeling timseries is daunting. Even for an automated process, the computational and memory costs of accessing many large data sets and writing to many timeseries datasets are extremely expensive. The Arc Hydro data model is housed within the structure of an ESRI s geodatabase, and consequently has a structure different from either of those mentioned above. A geodatabase has all the standard features of a relational database (such as Microsoft Access) including table objects and relationship classes, but is also capable of storing geospatial information.

8 N. Johnson 8 This makes a geodatabase a powerful tool for storing, accessing, and analyzing timeseries data associated with spatial objects. In the Arc Hydro format, a single table contains all the timeseries values in the database. Spatial information is present on each record in the form of a 'FeatureID' corresponding to a spatial feature in the geodatabase. In addition to having spatial data on each timeseries record, the Arc Hydro format also includes metadata on each record in t he form of a 'TSTypeID' corresponding to a record in the TSType table. Figures 4 and 5 are excerpts from Dr. Maidment s Arc Hydro book (Maidment 2002) and demonstrate how timeseries are linked to geospatial features in a geodatabase. Figure 4 Arc Hydro Timeseries related tables. From Maidment 2002.

9 N. Johnson 9 Figure 5 Relationships in Arc Hydro timeseries structure. From Maidment 2002 Because the Arc Hydro timeseries format contains both temporal and spatial information on every record, an Arc Hydro timeseries table can be used as either a space-centered data set or a time-centered data set depending on the query used to retrieve data. The Arc Hydro timeseries structure is built upon a relational database, and different types of data sets can be retrieved directly from a single table using queries on space, time, and variable attributes. Though gridded and timeseries formats are efficient in terms of disk space, they may not be efficient for storing very sparse datasets. Gridded data structures avoid storing spatial data with each value by using the file structure to implicitly store spatial information. However, a value for each grid cell is required, regardless of whether a value was actually measured or is necessary at that location. Likewise, timeseries structures require a value for every timestep at a location weather or not the value is meaningful. A quick analysis of rainfall data at two different time intervals revealed that gridded data structures may not be the most efficient method of storing data for sparse data sets. At a daily time step, less than 25% of cells contained non-zero measurements for a 10,000 square mile watershed near San Antonio Texas over the relatively rainy year of At an hourly time step, less than 4% of

10 N. Johnson10 grid cells contained non-zero values for two rainy months in Table 1 and Figure 6 illustrate the findings of this analysis. Table 1 Summary of the Number of Cells used to store rainfall data. Figure 6 a) San Antonio area (2 months of hourly data) b) San Marcos area (1 year of daily data) After the two months of hourly NEXRAD data from August and September of 2001 for the WGRFC s entire area was uncompressed to the GRIB file format, it occupied approximately 485 MB of hard disk space. The area for which the hourly data was extracted to an Arc Hydro geodatabase represented approximately 1% of the total WGRFC s area. After zero values were removed and two months of data was converted to Arc Hydro format for the area shown in Figure 6a, the database size was a little over 4 MB. Assuming that this area is representative of the entire area, had the entire area (~100 times) been extracted, the database would have occupied approximately 400 MB of disk space, still less than the grib files themselves. For sparse rainfall data sets, the benefits gained from storing limited amounts of spatial information are balanced by the losses associated with storing many zero or meaningless values. The value of the Arc Hydro timeseries structure for sparse data sets is that it stores

11 N. Johnson11 spatial information with every value in a data set. Because each record contains both spatial and temporal information, the value can be located in the space-time domain without having to store continuous spatial or temporal data sets that often contain many unnecessary values Radar Rain Gage Comparisons Radar estimates of rainfall are made by measuring reflectance off raindrops in the area surrounding a radar tower. While many factors are used in the conversion of this reflectance to rainfall, the algorithms are based on the simple notion that the volume of water in a given volume of air is related to the backscatter of radar off raindrops. The basic relationship used to convert the reflectance (Z) to rainfall (R) is: b R = az Where a and b are empirical constants, R is the rainfall in [mm 6 /m 3 ], and Z is the reflectance in dbz. A better understanding of the factors that affect reflectance have resulted in many improvements to the specifics of the algorithm over the past 20+ years. Complicated algorithms for removing anomalies in the data from ground interference, hail effects, and distance biases are used to ensure that the rainfall estimates predicted from the Z-R relationship are as accurate as possible. (Steiner et. al. 1999) In addition to these adjustments, rainfall data are calibrated to observed rain gage data and combined with overlapping radar coverages. The current process of converting reflectance to estimates of rainfall is often called the WSR-88D (Weather Surveillance Radar 1988 Dopplar) Algorithm in reference to the radar hardware used to collect the reflectance data. Radar estimates of rainfall are made for the entire lower 48 states as a part of the NEXRAD program (Fulton et. al. 1998) and available on the web from the National Weather Service s websites. (NWS 2005a) Radar data from the NEXRAD program go through many steps during the calibration process from raw reflectivity data to rainfall estimates. A thorough discussion of the processing to obtain the final product, called Stage III data, is available on the NWS website. (NWS 2005b) The Stage III data is available on as frequent as a 15 minute timestep, but is typically distributed at an hourly timestep and represents the best estimate of rainfall from the NWS. Comparisons between NEXRAD precipitation estimates and gauge rainfall have been extremely influential in identifying and quantifying biases in the Z-R relationship and are currently used in real-time to calibrate Stage III NEXRAD data. They have led to vast

12 N. Johnson12 improvements in the algorithms used to develop the Stage III products. The most common criteria for comparison is the mean accumulation at a rain gauge, and the radar cell immediately above it, however other comparisons are based on matching long-term exceedence percentiles, or optimizing other radar performance criteria. (Steiner et. al. 1999) All rainfall estimates contain some sort of temporal averaging or accumulation, and comparisons are made on an hourly basis, storm basis, or long term averages. In addition to temporal accumulation of rainfall rates, spatial variability of rainfall within a single radar cell also introduces uncertainty into radar gauge comparisons. Some work has been done to characterize this uncertainty (Steiner et. al. 1999, Kitchen and Blackall 1992), but most studies have not explicitly accounted for spatial variability of rainfall within a single radar cell. The spatial variability of rainfall becomes more important when making comparisons at shorter timesteps. (Koren et. al. 1999) Kitchen and Blackall (1992) attempted to characterize how much of the scatter in rainfall gauge comparisons could be attributed to what they termed representativeness errors, or differences in sample volumes. They point out that most calibration schemes are designed to minimize the differences between gauge and radar estimates even though the two estimates characterize fundamentally different quantities. Gauges provide data on how much rain fell at an individual point in space while radar estimates characterize the spatial average over some grid cell domain. They attempt to resolve differences due to spatial variation in rainfall by using a dense gauge network with spacing on the order of 1 km 2. Many hydrologic modeling applications rely on areal averages of rainfall, and do not require accurate rainfall information at a specific point. Johnson et. al. (1999) also examined the differences between mean areal rainfall estimate using radar and gauge data. Their study focused only on characterizing the differences, however, and did not make conclusions about which was more accurate. In addition to comparing areal mean rainfall Johnson et. al. also examined the results of a continuous hydrologic model using both gauge rainfall data and radar-derived rainfall. Johnson et. al. did consider the method of calibration or gage adjustments used by the NWS in detail, but worked with the Stage III NEXRAD rainfall data.

13 N. Johnson13 3. Methodology The objectives of this project include two tasks. The first involves accessing NEXRAD data and converting it into a form that could be used for hydrologic modeling. A methodology and associated tools are developed to efficiently extract data from gridded NEXRAD files and write them to Arc Hydro timeseries format. The Arc Hydro timeseries format is used as an intermediary between the NEXRAD grib files and the.wdm timeseries files used for hydrologic modeling with HSPF. In the second phase of the project, NEXRAD data is compared with rain gauge data for an area near San Antonio. Comparisons included cumulative differences over a two month period and storm-by-storm comparisons. Only point comparisons are included and consideration was not given to how NEXRAD and rain gauges perform in estimating areal average precipitation NEXRAD to WDM through Arc Hydro Timeseries As mentioned in the Introduction, this project is part of a larger effort to use NEXRAD data in hydrologic modeling. There will be three distinct steps in the larger process of preparing NEXRAD data for use in HSPF applications. The first step will involve extracting the gridded xmrg NEXRAD rainfall data to Arc Hydro timeseries format. The second step will be to perform spatial analysis in ArcGIS to estimate average areal precipitation for input to HSPF models. The final step will write the resulting catchment-average precipitation timeseries to.wdm format for use in HSPF. This project will foucs on the first of these steps and previous work and available tools from the Center for Research and Water Resources (CRWR) will be utilized to accomplish the remaining tasks. The motivation for extracting NEXRAD data to the Arc Hydro timeseries format is to make use of this previous work. From the Arc Hydro timeseries format, GIS tools for spatial averaging, interpolation, and aggregation can be used to efficiently and accurately estimate average drainage area precipitation. In addition to the benefits provided by ArcGIS tools for spatial analysis, much effort at CRWR has gone into developing tools that convert Arc Hydro timeseries data to other formats. Tools have already been developed to create NetCDF,.dss, and.wdm files from Arc Hydro timeseries. Once drainage area precipitation estimates are made, the timeseries

14 N. Johnson14 can be easily written to.wdm files for HSPF applications, or any of the other supported file formats Procedure of Application For this project, hourly Stage III NEXRAD data were obtained for two months of 2001 over the entire Southeastern US. This data is available from the NWS and is distributed in the xmrg compressed, binary format. (NWS 2005a) Data was extracted from the binary xmrg file format to the ASCII grid format shown in Figure 7 using a slight modification of tools available from the NWS. (NWS 2005c) Each row in the text file contained the HRAPX and HRAPY coordinates for the cell it describes. This format is slightly different than the traditional gridded data structure in that it has spatial information stored on every record. This is not the way data is stored in the xmrg file, but the spatial information was added in the process of extracting the binary data to ASCII format. The HRAP grid is a projection used to combine data for estimates of rainfall from different radar stations in the Stage III development process. (NWS 2005b) For August and September of 2001, 24 hours * (30+31 days) = 1464 files are used to describe a spatially and temporally continuous field of rainfall. An algorithm was developed in ArcMap (using Visual Basic for Applications (VBA)) to download the data for specified NEXRAD cells from a web server. Initially all data (including zero values) were stored, however, it quickly became appearant that major disk space savings could be made by removing zero values. The program takes as inputs: - Min and Max X and Y HRAP coordinates - Start and End DateTime (within the Aug-Sep 2001 period) - Arc Hydro database to store the timeseries data. Figure 7 ASCII grid file structure

15 N. Johnson15 The extracted data was written to a Microsoft Access database in the Arc Hydro timeseries format. Two tables, TSType and Timeseries, are populated in the process. The TSType table has only one record and was assigned appropriate TSType information: - Variable NEXRAD Stage iii Rainfall - Units in - IsRegular False - DataType Incremental - TSInterval Daily - Origin Recorded The Arc Hydro TimeSeries table contained the values for thousands of records extracted from the xmrg files with date, location, and variable information on every record. The DateTime for each value was reconstructed from the filename, which contained the month, year, day, and hour described by the observation. For the two months used in this project the filename convention in the distributed xmrg files are consistent, however there are discrepancies for dates prior to The FeatureID was defined to be the concatenation of the three-digit HRAPX and Y coordinates with added zeros when necessary. The TSValue was the value extracted from the text file. The initial algorithm was designed to simply loop through all the lines in the text files and check if the HRAPX and HRAPY coordinates fell within the specified bounds and was quite inefficient. A later algorithm took advantage of the grid structure of the NEXRAD files to skip directly to the required values and was designed to read directly from the text file format used by ArcGIS to store grid information. This second algorithm parsed the values from a text file into one huge array with a number of elements equal to the number of columns times the number of rows. Commands for text parsing based on a single delimeter are extremely efficient in VB, and parsing the 165,000 values takes less than one second. The array element for a specified HRAP cell X i, Y i, was found using the metadata information at the beginning of the file. If X min, Y min, N cols, and N rows, are the coordinates of the lower left corner and dimensions of the HRAP grid contained in the ASCII file, the required element was found using: Element# = ( X X )*N + ( Y ) i min cols i Y min The value from this element corresponds to the given HRAP cell (X i, Y i ), and the value can be written to Arc Hydro TimeSeries and TSType tables as described previously.

16 N. Johnson16 The file format shown in Figure 1 is used by ArcGIS to import ASCII grid information and build GIS raster datasets. Metadata at the beginning of the files give the coordinates of the lower left corner of the grid in HRAP X and Y coordinates, and a list of values follows. A program was developed at the Texas Advanced Computing Center (TACC) for reading xmrg files distributed from the NWS and writing to the ArcGIS grid ASCII format. This program mimics the process laid out by Xie et. al. (2005) for extracting xmrg files to ASCII format and makes use of the programs available from the NWS website. The algorithm described above was then used to extract the data from the ASCII files and write them to an Arc Hydro database. Efforts are currently underway to automate the entire process to write xmrg files directly to the Arc Hydro format. This work at the TACC will build upon the algorithm developed as a part of this project for writing NEXRAD data to Arc Hydro format. The tools developed at TACC will extract data from the xmrg files to Arc Hydro timeseries and be much more efficient in terms of computation time and disk space than the current system which requires the use of intermediate text files. By the end of the summer of 2005, the TACC hopes to have data for the entire area covered by the WGRFC for the past five years extracted to Arc Hydro format. This database can then be queried to retrieve either grid or timeseries datasets of NEXRAD data. With the NEXRAD Stage III data in Arc Hydro timeseries format, it is available to tools necessary for preparing data for hydrologic modeling. GIS tools can be used to perform spatial analysis and timeseries tools are available to write the data to the.wdm format required for HSPF modeling NEXRAD Gauge Comparisons The USGS near San Antonio is interested in using NEXRAD rainfall data as inputs to hydrologic and water quality models especially in areas with sparse rain gauge coverage. To support this effort, an analysis of how gauge and radar data compare was carried out and presented here. The data used was hourly Stage III NEXRAD data for two very rainy months in the summer of 2001 over the area shown in Figure 6a. Two methods were used in the comparison. The first method simply looked at the cumulative difference over August and September of The second investigated whether or not there were storm-specific biases in the data and how these biases could be eliminated to improve the NEXRAD estimates over the gauges.

17 N. Johnson17 Hourly rainfall data from eight rain gauges near San Antonio are currently used by the USGS for hydrologic modeling. The locations of these gauges, the agency that they belong to, and the NEXRAD cell immediately above each are presented in Figure 8 and Table 2. Figure 8 Gauges used in comparison. Table 2 Rain Gauge, responsible agency, and associated NEXRAD cell Procedure of Analysis The first method used to compare the gauge data with the NEXRAD estimates was simply the cumulative difference over August and September of Some of the cells

18 N. Johnson18 were close to the border of two NEXRAD cells, so the four closest cells were each compared to the respective gagues. A cumulative difference plot was generated using the WDMUtil software distributed with the Environmental Protection Agency s (EPA) BASINS system. (EPA 2005) For each of the 1,464 hours in the two months, the gauge measured rainfall was subtracted from the NEXRAD estimates. A cumulative difference plot was helpful to visualize the differences between the two rainfall measurements over time and eliminated any biases associated with timing differences between the NEXRAD and gauge measurements. The user interface for GenScn is shown in Figure 9 for the comparison of rainfall estimates at the Boerne rain gauge. Figure 9 GenScn program s Cumulative Difference Plot

19 N. Johnson19 The second comparison method followed the approach outlined in Wilson and Brandes (1979). It divided the observed record into rainfall events and compared the radar and gauge estimates at the same locations. Wilson and Brandes analyzed each storm using the average ratio of gauge to radar depths, called the average storm bias, (G/R) Ave, in an attempt to eliminate biases in the NEXRAD estimates. They used about 20 gauges in Oklahoma to analyze 14 storms in 1974 and The average storm bias is defined as the average ratio of gauge to radar rainfall across all the gauges for a single storm: Where: ( G / R) N i = Ave = 1 Gi R i N - (G/R) Ave is the average ratio of gauge and radar depths for the storm - G i is the total gaug precipitation for the storm at gauge i [in] - R i is the total radar precipitation for the storm at gauge i [in] - N is the number of gauges The relative dispersion (or Coefficient of Variation) about (G/R) Ave is used to quantify the amount of variability in G/R across the gauges for each storm: Where: C. OV.. Storm = σ ( Gi / Ri ) ( G / R) Ave - C.O.V. Storm is the coefficient of variation about (G i /R i ) Ave for the storm - σ(g i /R i ) is the standard deviation of the ratio for each gauge for the storm The average difference (error) is calculated as a percent of the gauge rainfall for the event: Diff Ave = N i = 1 Gi Ri G i / N Where: - Diff Ave is the average error across all gauges for the storm

20 N. Johnson20 Wilson and Brandes used the storm-average G/R to remove biases associated with each storm. In their Oklahoma-based study the average difference for all the storms (Average of Diff Ave for each storm) was over 60% while the average scatter about the mean G/R (Average of C.O.V. Strom for each storm) was 30%. By multiplying the radar measurement at each gauge by the mean storm bias, (G/R) Ave, they were able to reduce the average difference between gauge and radar to 24%. The average percent difference with the storm bias removed is: Diff 2 Ave = N i = 1 Gi Ri G ( G / R) Where: - Diff2 Ave is the average error for all gauges after removing the storm bias i Ave / N An alternative to using the simple mean of the gauge ratios to calculate the average storm bias assumes that the variable (G/R) Ave has a lognormal distribution. With a lognormal distribution, the factor in the equation used to remove the storm bias becomes: F Gi log R i log = 10 and the difference after removing the lognormally distributed storm bias is Diff 3 Ave = N i = 1 G F i log Ave ( G / R) G i Ave / N For this project, eleven individual storm events were identified during the months of August and September of 2001 that produced more than 2 mm (~0.08 inches) of rain at three or more gauges. Wilson and Brandes noted that when precipitation is less than 2 mm, the spatial variability of rainfall (observed by radar) is very large and may not be appropriate for comparison to gauges. Radar/Gauge pairs were only included in the analysis if both gauge and radar estimates were greater than 2 mm for the event. For this analysis, only the NEXRAD cells immediately above the gauges were used for comparison. The corresponding NEXRAD cells were found using a spatial query in the ESRI software ArcMap to find the HRAP grid cell that contained each gauge. The cells associated with each gauge are shown in Figure 10.

21 N. Johnson21 Figure 10 Gauges used for comparison and associated NEXRAD cells. The total gauge and NEXRAD rainfall was found manually for each of the 11 storms and summarized in a Microsoft Excel file. The calculations presented above were performed for each of the eleven storms. For two of the storms, only three gauges record greater than 2 mm of rainfall at both the gauge and NEXRAD cell. All other storms had > 2 mm for at least 6 stations. Seven of the eleven storms occurred during the week of August 28 th to September 1 st, and a plot of the gauge precipitation for the week (in inches/hour) is shown in Figure 11.

22 N. Johnson22 Figure 11 Seven storms during the week of August 27 September 3

23 N. Johnson23 4. Results 4.1. Cumulative Difference Results The results of the cumulative difference analysis are presented in the figures in Appendix A. The four plots in each figure correspond to each of the four NEXRAD cells in the area. Rainfall in the San Antonio area was over 11 inches for the two months analyzed at many of the rain gauges. There was no identifiable pattern to the cumulative differences amongst the gauges. The four NEXRAD cells closest to each gauge were analyzed, and NEXRAD estimates were consistently higher for three of the gauges, and consistently lower for three others. Table 3 shows the total rainfall at the gauge, the difference at the cell immediately above the gauge (normalized by the rain gauge total), and the average difference of the four cells surrounding the gauge. Table 3 Cumulative differences for each gauge (as a percentage of gauge total) Cumulative differences in the cells immediately above the gauges ranged from -31% to +5%, and differences between adjacent cells showed similar magnitudes. For four gauges, San Antonio Airport, Selma, Helotes, and Government Canyon the cell immediately above the gauge gave a smaller cumulative difference to the average of the four surrounding cells. For Boerne, Bulverde, and Scenic, the averages of the surrounding cells were smaller than the immediate cell, and there was very little difference for Cedar. In general, the cell immediately above the gauge did not show closer estimates to the gauge than an average of adjacent cells.

24 N. Johnson Storm-Bias Results The results of the second comparison are summarized in Table 4. Statistics related to the difference between rain gauge measurements and the NEXRAD cell immediately above each gauge are shown for the 11 storms. Table 4 Results of storm-to-storm radar/gauge comparisons The average G/R, (G./R) Ave, was greater than one for all but two of the eleven storms. Though these two storms were fairly small (average gauge rainfall ~ 0.5 inches), there was no clear trend between the G/R ratio (storm bias) and the total storm rainfall. The large storm on the 29 th of August showed the largest NEXRAD to gauge ratio. However, similar magnitude storms on the 30 th and September 6 th had G/R ratios near one, and the largest storm on August 27 th did not have a large G/R ratio. The relative dispersion about the average G/R was normalized by the gauge rainfall, and fell between 25% and 50% for most storms. There was a slight trend towards larger dispersion for larger storms as illustrated in Figure 12, however two storms with less than an inch of average precipitation had relative dispersion over 35% (8/28 and 9/1) and two large storms had relative dispersion under 35% (8/27 and 9/6). The overall average dispersion about (G/R) Ave for all storms considered was 35%.

25 N. Johnson25 Figure 12 Coefficient of Variation (relative dispersion) around (G/R) Ave The average difference between gauge and rainfall estimates for all the gauges was variable amongst storms. Four storms had average differences of 20% or less, three more were between 20% and 30%, and three of the remaining four were greater than 40%. The average for all the storms considered was 28%. In this analysis removing the average storm bias did not reduce the differences between gauge and rainfall as it did for the Wilson and Brandes study. Only one storm (on 8/19) showed significant improvement in the average difference with the storm bias removed and for two storms, the average difference increased significantly with the removal of the storm bias. Assuming a normally distributed storm bias resulted in a larger difference for all but three of the storms, and increased the average difference for all storms from 28% to 31%. Treating the (G/R) Ave for each storm as a lognormally distributed variable showed better results, but still did not improve the overall average difference. There are a few reasons why removing the average storm bias did not reduce the differences between radar and gauge measurements for this analysis. First, the original overall average difference in this study is significantly less than it was for the Wilson and Brades study (28% vs. 63%). From a qualitative perspective, the removal of storm biases will only result in better agreement between gauge and radar rainfall estimates if the biases

26 N. Johnson26 (quantified by (G/R) Ave and the average difference) are smaller than the scatter about the average storm bias (quantified by the relative dispersion (C.O.V. Storm ) about (G/R) Ave ). For the Wilson and Brandes study, the original overall average difference across all storms was 63% and the relative dispersion about (G/R) Ave was 30%. For this analysis, the original overall average difference across all storms was 28% and the relative dispersion about (G/R) Ave was 35%. For the two storms that showed larger differences with the removal of the storm bias (on 8/29 and 9/1), the relative dispersion about (G/R) Ave was very large. The relative dispersion cannot be directly compared to the average difference to say definitively weather or not improvements are possible by removing average storm biases because they are quantifying slightly different things. However, the reason that removing the storm bias did not decrease differences in this study is most likely due to the large scatter about the (G/R) Ave and the relatively small original average differences. Another factor that may account for the lack of improvement is the number and quality of gauges used in the analysis. The Wilson and Brandes study used about 20 gauges for many of the storms as opposed to 6-8 for most storms in this study. With a smaller number of gauges in this study, a few outliers could skew the results significantly. In addition, it is not known weather or not the NOAA stations used in this study had already been used in the calibration of the Stage III NEXRAD data. These gauges did not show significantly different results than the USGS gauges, but using gauges for validation that have been previously used in calibration is not a good practice.

27 N. Johnson27 5. Conclusions / Future Work This work demonstrated that even though the task of reformatting and manipulating data is tedious and time consuming, having NEXRAD data into the Arc Hydro timeseries format is extremely helpful for preparing data for HSPF modeling. From this standard format, the data is available to GIS tools for spatial analysis and tools for writing data to the.wdm file format. Analysis of the cumulative differences between radar and gauge estimates of rainfall showed differences between -30% and +5% with an overall average difference (absolute value of the difference) of 12%. Average differences (absolute differences) on a storm-tostorm basis were between 4% and 47%, with an overall average difference of 28%. These differences (12% over two months vs. 28% storm-to-storm) suggest that temporal averaging over the two months reduced the difference between gauge and radar estimates of rainfall. Removing the storm bias did not result in smaller differences between gauge and radar rainfall estimates as it did in the 1979 study by Wilson and Brandes. This is likely due to the improvements made to the calibration of radar rainfall over the past 20 years and the improved quality of the Stage III data and. Compared to the 63% storm-to-storm difference that Wilson and Brandes reported, relatively small differences (on the order of 25-30%) between gauge and radar estimates were found for the NEXRAD Stage III data in this study. Future work will involve the development of additional tools to fully automate the process of preparing NEXRAD data for use in HSPF modeling. These tools will extract required NEXRAD data from a geodatabase, perform the spatial averages over subbasins, write the data to the appropriate places in.wdm files, and update the HSPF model files to read the new timeseries. Further comparisons between NEXRAD and gauge data in the area will also be required before the NEXRAD data can be used confidently. In addition, consideration must be given to the implications of spatial averaging in NEXRAD/gauge comparisons and HSPF modeling applications.

28 N. Johnson28 References BADC BADC Grib Documentation. British Atmospheric Data Centre. EPA BASINS: Better Assessment Science Integrating Point & Nonpoint Sources. United States Environmental Protection Agency BASINS Program. Fulton, R., Breidenbach, J., Seo, D. and Miller, D The WSR-88D Alogirthm. Weather and Meteorology. 13(2) Jayakrishnan, R., Srinivasan, R., and Arnold, J Comparison of raingage and WSR-88D Stage III precipitation data over the Texas Gulf basin. Journal of Hydrology (292) Johnson, D., Smith, M., Koren, V., and Finnerty. B Comparing mean areal precipitation estimates from NEXRAD and gauge networks. Journal of Hydrologic Engineering. 4(2) Kitchen, M. and Blackall, R.M Representativeness errors in comparisons between radar and gauge measurements of rainfall. Journal of Hydrology (134) Koren, V., Finnerty, B., Shaake, J., Smith, M., Seo, and D., Duan Scale dependencies of hydrologic models to spatial variability of precipitation. Journal of Hydrology Maidment, D. R Arc Hydro. GIS for Water Resources. ESRI Press. Redlands, CA. Neary, V., Habib. E., and Fleming, M Hydrologic Modeling with NEXRAD in Middle Tennessee. Journal of Hydrologic Engineering 9(5) NOAA xmrg File Format. National Oceanic and Atmospheric Administration s: NWS Hydrologic Laboratory. NWS. 2005a. Archive of River Forecast Center Operational NEXRAD Data. NOAA Hydrologic Data Systems Group. NWS. 2005b. About the Stage III data. NOAA/NWS Distributed Model Intercomparison Project. NWS. 2005c. NEXRAD Stage III Precipitation Data. (#Codes to Read XMRG Files) NOAA/NWS Distributed Model Intercomparison Project. Smith, J. and Krajewski, W Estimation of the Mean Field Bias of Radar Rainfall Estimates. Journal Applied Meteorology (30) Steiner, M., Smith, J., Burges, S., Alonso, C., and Darden, R Effects of bias adjustments and rain gauge data quality control on radar rainfall estimation. Water Resources Research. 35(8) UniData NetCDF Documentation. University Coorporation for Atmospheric Research: UNIDATA Program.

29 N. Johnson29 Whiteaker, T A Prototype Toolset for the ArcGIS Hydro Data Model. Thesis. The University of Texas at Austin, CRWR Online Report < Wilson, J. and Brandes, E Radar Measurement of Rainfall A Summary. Bulltein of the American Meteorological Society 60(9)

30 N. Johnson 30 Appendix A

31 N. Johnson 31

32 N. Johnson 32

33 N. Johnson 33

34 N. Johnson 34

35 N. Johnson 35

36 N. Johnson 36

37 N. Johnson 37

38 N. Johnson 38