Final Report Rainfall Runoff Prediction Written by George Limpert In association with CARES and Chris Barnett With the mentoring of Dr.

Transcription

1 Final Report Rainfall Runoff Prediction Written by George Limpert In association with CARES and Chris Barnett With the mentoring of Dr. Neil Fox For CS 4970, Senior Capstone Design I

2 Table of Contents 0. Introduction Page 3 1. Problem Definition Page Problems & Needs Page Background Information Page WSR-88D Page GIS Page Literature Survey Page The WSR-88D Rainfall Algorithm Page Automated Bright Band Detection Page Calibrating Radar Precipitation Estimates Page Comparison of Radar Precipitation Estimates Page Effect of Storm Type on Radar Precipitation Estimates Page Goals & Prototypes Page Cost & Feasibility Page Requirements Analysis Page Introduction Page Overall Description Page System Requirements and Constraints Page Operating Environment Page Market Users and Characteristics Page Environmental Constraints Page System Components Page Software Interfaces and Libraries Page Communication Interfaces Page Hardware Interfaces Page Software Maintenance, Life Cycle and Support Page Performance Requirements Page Resource Requirements Page Evaluation Metrics Page Alternative Solutions Page Design Specifications Page Introduction Page Basic System Design Page Data Requirements Page Software Design Page Testing Methods Page Scheduling Page System Implementation Page System Testing Page Technical Report Page Introduction Page Execution Page Prototypes Page Conclusion and Discussion Page Future Work Page Licensing Page References Page 72

3 0. Introduction In this report, I will describe the process by which I attempted to develop a system for decoding radar data for the purpose of predicting the runoff of rainfall. The reader will be presented with a background on the technologies in use and some related work. Following this, the problem will be clearly defined and the requirements for the project will be outlined. The design of the proposed finished product will then be described. This describes the process in which the system was developed. The second half of the report focuses on the current status of the product. It is not completed, but a working prototype has been created. The report describes the prototype, its functionality, and describes the future work on the project. An evaluation of the project is included at this point. It is my belief that while the project did not meet its initial goals by the end of the semester, it is not a failure. Unintended uses of the product and the software within have been found. For example, a completely unrelated forecasting system uses some source code from the software in this project. Also, the radar data format, which is somewhat obscure, has been partially documented. This is beneficial for someone else wishing to decode level III radar data, even if their product serves a completely different purpose. I hope after reading this report that the reader will gain a better understanding and appreciation for the doppler radar technology that the National Weather Service has developed. Furthermore, I hope the reader will gain a better appreciation for the hard work that is involved in forecasting weather. As a meteorologist, it is frustrating to be at the receiving end of jokes about how the forecasting of weather is extremely inaccurate. While there are still many busted forecasts, our understanding of the atmosphere and the processes within is constantly improving. Furthermore, the

4 technology used to detect and predict the weather is constantly being developed and improved. At the time of writing this report, there is work underway to replace the WSR-88D systems with newer radar technology. Despite the great benefits realized by the implementation of these radar systems, there is much better technology that is being developed. Weather forecasting is far more accurate than, for example, flipping a coin, despite the comments of some comedians. It is a hard science which benefits greatly from chemistry, computer science, mathematics, physics, and many other branches of science. Even if you don't find the product useful, hopefully the reader will have a better appreciation for meteorology after reading this report. Special thanks is owed to three people who greatly assisted in the development of this system. First, I want to thank Steve Lack for his efforts in contributing to and improving this project. Without his help in understanding the operation of radar, I would have struggled much more with decoding the level III data. Second, I want to thank Chris Barnett for his help in providing me with information about GIS file formats and for his guidance in developing this project. Without him, I would have developed this project with little sense of direction and without a clear goal in mind. Furthermore, I would have been ignorant to the requirements that a system such as the one I am developing has. Finally, I want to thank Dr. Neil Fox for mentoring me and assisting me in the development of this project. He is as close to an expert on radar as we have at the University of Missouri. His knowledge of the subject was very helpful in decoding this data. Furthermore, he helped me with finding answers to meteorological questions which I did not have an answer to such as the resolution of the digital precipitation array. Also, he helped me in verifying the data I converted. Finally, he guided my project away from my initial plans and directed me towards creating a system which

5 would have practical uses at the university. Without his mentoring, I would not have even started this project. With that being said, I hope the reader finds my project report very useful and informative. I also hope the reader comes away with a better appreciation for the science of meteorology.

6 1. Problem Definition 1.1. Problem & Needs One of the many important aspects of meteorology is hydrology. While it is extremely important to understand what is occurring within the atmosphere and about precipitation falling, it is also important to understand and forecast what happens once the precipitation reaches the ground. Floods kill more people each year than any other meteorological phenomena. Even in areas that don't usually receive severe weather, flooding is often a threat. Hydrology is the branch of meteorology that deals with what happens to precipitation after it reaches the surface. Hydrology also has important uses to agriculture. For example, soil has a limit on how much water it can absorb. Different types of soils can absorb different amounts of water before becoming saturated. Once the soil is saturated, additional rainfall will run off. This can lead to flooding, but can also lead to erosion of topsoil. Chemicals and fertilizers that have been applied may be washed away with the water when run-off occurs. While these chemicals are useful to farmland, they may be harmful in other places, particularly in streams and rivers. Some of these chemicals include pesticides and herbicides which may be harmful to wildlife. It is important to ensure that these chemicals, when applied, to not run-off into areas where they would be harmful. Because of these needs in agriculture and to predict flooding, it is very useful to develop a product which can be used to forecast when run-off will occur. While this depends somewhat on the amount of rainfall received over a certain time, it also depends on the land. Topography and soil type are major factors in determining when run-off will occur. Topographical data is widely available in formats that can be manipulated and displayed by GIS systems. While many methods exist for

7 estimating precipitation, one of the best available is radar estimates of precipitation. Radar data, however, is distributed in a very different format. To create a useful product, it is necessary to merge geographic and precipitation data into a common format so they can be displayed together. The purpose of this project, given these needs, is to create a system for collecting radar estimates of precipitation and converting this data into a format which can be manipulated by GIS software. This data will then be delivered to a GIS system which is capable of merging the radar data with existing geographic data to create useful products. The data will likely be distributed through an existing website to make it available to the agricultural community.

8 1.2. Background Information Background Information: WSR-88D An understanding of Doppler radar and the WSR-88D systems implemented by the National Weather Service is essential for understanding this project. Data from these radars are the basis for the rest of this project. Ordinary radars operate by sending out pulses of energy. A portion of this energy may be reflected back to the radar by objects in the path of the energy pulse. The time it takes for a pulse to return to the radar site can be used to estimate the distance of the object. The amount of energy reflected can also be useful in determining some characteristics of the object which reflected the energy. Doppler radar adds another useful parameter that can be detected. Radial velocity is a measure of an object's motion towards or away from the radar site. This can be determined because of the Doppler Effect. An approaching object, when reflecting the energy, will also cause the waves to be compressed, resulting in a blue shifting. An object moving away, when it reflects the energy, will cause the waves to be stretched, which results in red shifting. By detecting these variations in frequency of the reflected energy pulse, an object's radial velocity can be determined. In meteorology, all of these parameters are extremely useful. Areas in the atmosphere with greater concentrations of particles will reflect more energy than areas of lesser particle concentration. Areas with more particles, if the particles are water droplets, are likely receiving heavier precipitation. Reflectivity is somewhat proportional to the amount of precipitation. Velocity is also important, particularly in the context of severe weather. While velocity information may be used to detect atmospheric boundaries and some other features of the atmosphere, it is very important in circulations. A localized strong circulation in a thunderstorm may suggest a possible tornado. A wider and weaker circulation may indicate the

9 presence a storm structure called a mesocyclone. When combined together in a Doppler radar system, velocity and reflectivity data are very useful in detecting areas where rain, snow, and severe weather are occurring. The WSR-88D system performs an entire scan every five, six, or ten minutes depending on the mode of operation the radar is in. This scan consists of several sweeps and can be pieced together to create an approximate three-dimensional image of the volume of the atmosphere. Within each volume, there are several sweeps, and the exact number varies depending on the mode of operation of the radar. The sweeps within a volume are conducted at different elevations ranging from 0.5 to 19.5 degrees. Within each sweep, a number of pulses or rays are sent. Usually there are 367 rays within a sweep. And within each ray, there are a number of range gates, corresponding to a distance from the radar. In the WSR-88D system, there are usually 460 reflectivity range gates and 920 velocity range gates. Each reflectivity range gate accounts for a 1 km distance from the radar and each velocity range gate accounts for a ¼ km distance from the radar. The hardware collects the data in an analog form. This data is be referred to as level I data. The data is then converted to a digital form, which is referred to as level II data. The level II data is equivalent to level I data but is in a digital form. Often, level II data is referred to as raw data. Level II data can then be analyzed by computer algorithms to produce other data that may be also useful to meteorologists. This analyzed data along with a subset of the level II data is referred to as level III data. While many features of storm structure can be identified by a human forecaster, algorithms can be designed to deliver similar results and with an impartiality that a human is usually incapable of possessing. Furthermore, examining

10 data through computer algorithms is much quicker than relying on a human observer. A human is certainly capable of detecting some circulations, identifying storm cells, and estimating the motion of individual storms. Computer algorithms, however, are also used to determine the height of cloud tops, indicate areas where hail may be occurring, and estimate precipitation amounts. None of these can easily be done by most people but can easily be done by using computers to analyze level II data. Despite the usefulness of the WSR-88D Doppler radar system, there are significant limitations. For example, there are over 100 WSR-88D radars deployed across the United States. Despite this, there are many areas, particularly in the western United States, that are not covered well if at all by radar. Furthermore, there are many limitations in the actual radar systems that do not relate to the implementation of the radars. It must be understood that radar does not indicate what weather is occurring at the surface. Radar indicates what is occurring in the atmosphere above the surface. While it can provide a strong indication of what may be occurring at the surface, there is never a guarantee. It is still necessary for the National Weather Service to rely on spotters to provide ground truth observations. The lowest beam elevation used is 0.5 degrees, which also means that the beam will be farther above ground at distances farther from the radar site. The curvature of the Earth also causes the beam to be higher above the surface of the Earth at distances farther from the radar site. Because of this, at far distances from radar sites, the radar beam may actually be above the clouds. It is possible for no energy to be reflected back to the radar from a given location while precipitation is actually occurring there. This puts a strong limitation on the usefulness of radar to detect what is occurring at distances far from the radar site.

11 There are also many other factors which may reduce the accuracy of radar data. While radar beams can penetrate clouds, terrain around the radar site can absorb the beam. There are also many sources of false echoes. The most common of these occurs near the radar site and is referred to as ground clutter. There are even algorithms for removing ground clutter from radar images; however these may also remove legitimate echoes in the process. Keeping these limitations in mind, radar is still a very useful tool in observing the weather. Weather radar is best viewed as a system with two major parts. One part is the radar dish, which collects the data. The other part is a computer and some algorithms to analyze the data that is collected. The radar system as a whole produces data that is very useful for the observation and forecasting of weather, particularly in the area of severe weather.

12 Background Information: GIS GIS is an acronym for geographic information system. A GIS is a computer system which stores, manipulates, displays, and analyzes data that has a geographical context. Many types of data such as boundaries, geographic features such as rivers and mountains, man-made structures, and aerial photographs are also suitable for analysis and display by GIS systems. Because meteorological data is referenced in the context of an area at a location on or above the surface of the Earth, it can also be manipulated by a GIS system.

13 1.3. Background Literature Background Literature Title: The WSR-88D Rainfall Algorithm One of the algorithms the National Weather Service uses to interpret radar data is used for estimating precipitation amounts. This algorithm has been revised many times and does not rely only on radar data. This article describes the algorithm that is used by the radar system in estimating precipitation. There are several steps involved in estimating precipitation. The first step involved is to detect whether precipitation is occurring within the coverage area around the radar. This step takes radar data which is edited to remove ground clutter as input. The data is examined to determine if certain thresholds are exceeded for the purpose of detecting if significant precipitation is occurring. The next step involved is to collect rain gage data. This only occurs after significant precipitation has been detected. Rain gage data is collected in real time. The following step involves processing actual radar data and converting it into a polar grid in which each box is one degree wide and one kilometer long. The algorithm only processes data for the first 230 kilometers around the radar. It attempts to select data at various elevation angles which is approximately one kilometer above the surface. This is done by taking into account the terrain and the various elevation angles. It also takes into account beam blockage due to terrain. Several elevation angles used to create the entire polar grid. At ranges far from the radar, the higher value from two elevation angles is chosen. Some other adjustment of radar data may also be performed during this step. After these adjustments, the reflectivities are increased at times to account for partial beam blockages. Also, areas of particularly high reflectivity gradients are edited for the purpose of removing ground clutter. Some false echoes are then removed from the data, if they can be

14 detected. One step of this is to check the next elevation angle for a significant decrease in reflectivity. If this is observed, the echo is determined to be false. At this point, when most false echoes and sources of inaccuracy have been removed or accounted for, a simple formula is used to convert from reflectivity into a precipitation estimate. This is done with an exponential formula which has two constants, zra and zrb. These constants vary depending on the location. The next step is to check for hail contamination. Like the constants above, the hail thresholds are determined operationally and vary between radar sites. Reflectivities are capped at a certain value to prevent overestimation of precipitation due to hail. The article emphasizes that this is a very subjective estimate and is not necessarily accurate. Research is still being done in the area of detecting hail to prevent overestimation. The precipitation estimates are compared with the estimate from the previous scan. If the differences are too great, the entire scan is discarded. This is done to prevent estimates from being contaminated with some kinds of interference or false echoes, despite the efforts made to account for these in earlier steps. If the data is not discarded, another adjustment is made at the farther ranges of the scan to account for the beam degrading. This step is performed to prevent underestimation of rainfall at the edges of the coverage area. This calculates rainfall rates and not total accumulation. To create storm total estimates, rainfall estimates are summed. Missing periods are filled in through interpolation, provided the missing periods are not very long. Any outliers are removed and replaced through averaging of neighbors. The data can be adjusted using ground truth data from rain gages. The article describes the technique as experimental due to difficulties in acquiring real time data. The first step involved is to attempt to wisely select which rain gages are used

15 to adjust the radar estimates. Data from rain gages is paired with data from radar, through selecting the closest value from nearby radar estimates to what the gage indicates. A variety of methods are then used to only choose good pairs of data. Some statistical methods are also used for quality control purposes. Once these steps are completed, graphical and digital products can then be created from the rainfall estimates. These products can then be used for forecasting or disseminated as level III data. The article then describes rain gage networks and also some of the other experimental work to overcome limitations of the rainfall estimation algorithm. Some of these include finding better values for some tunable parameters of the algorithm, solving problems involved with overestimates due to melting precipitation, underestimates at far ranges of the scan, better handling of false echoes, and other areas of research. The algorithm is still being improved and the final third of the article is used entirely to discuss future improvements to better estimate rainfall.

16 Background Literature Title: Automated Detection of the Bright Band Using WSR-88D Data One source of overestimation of precipitation is referred to as a bright band. In addition to cumulonimbus clouds, nimbostratus clouds can also produce precipitation. If the precipitation melts on the way to the surface from nimbostratus clouds, there will be a layer of melting precipitation which produces high reflectivities. Because the radar beam is tilted, this layer of high reflectivity is observed as a bright band. This band can stay over the same area for several hours, under some conditions, and can result in significant overestimates of precipitation. This article describes a technique for detecting and automatically accounting for the bright band in precipitation estimates. The first step of this algorithm is to acquire a good sample of data. In areas of significant beam blockage, the data may be discarded completely. Data from all elevations is considered in detecting the bright band. A minimum reflectivity is defined for what constitutes a bright band. The algorithm then checks higher and lower elevations for significant changes in reflectivity. If this is observed, and is in within a defined distance from the radar, it is considered by the rest of the algorithm as a possible bright band. In the case of the bright band being particularly close to the ground, only the top of the bright band may actually be observed. To account for this, the radar also considers data from the RUC-2 model to identify at what height melting would occur. Once a possible bright band is identified, the data is then examined to check if the band extends around the radar site and if the height of the possible bright band is consistent. If these tests are satisfied, then the algorithm identifies the area as a bright band. The article then discusses the accuracy of the estimates. Results from the

17 algorithm were verified through vertical pointed radars, weather balloon observations, and RUC-2 model output. The article seems to suggest that the algorithm performed well, but that some parameters may need to be tuned. Some opportunities to further improve the algorithm are discussed in the article. Unfortunately the article does not seem to mention how the data can be adjusted to achieve more accurate precipitation estimates when a bright band is identified.

18 Background Literature Title: Real-Time Calibration of Radar Precipitation Estimates This article discusses methods of calibrating radar estimates of precipitation using data from rain gages. The article starts by discussing some weaknesses of rain gages. These include mechanical issues leading to underestimation of rainfall and the distances between rain gages. Radar estimates of precipitation are discussed as a means of overcoming these problems. Unfortunately, as the article states, these estimates are not necessarily accurate. Combining data from radar estimates and rain gages is suggested as a means to overcome the limitations of both approaches. Data from radar estimates is used to pinpoint trends in rainfall while data from rain gages is used to provide actual measurements of rainfall. The article also suggests that the method of combining data from the two sources needs to be done automatically and not require a human operator. This study uses level III data because of the difficulty in acquiring level II data in real time. The digital precipitation array product is used because it is the only digital representation of rainfall amounts. All of the other level III products provide only a range of possible precipitation amounts and do not provide actual estimated values. The digital precipitation array is a product with approximately four kilometer square boxes in a grid to represent the data. This study also discusses possibly merging estimates from multiple radars by selecting the highest value over a particular point. The basis for this decision was that most of the overlapping areas will be at the edge of radar coverage areas where precipitation tends to be underestimated. Several techniques are discussed to improve the accuracy of calibrating radar estimates with rain gage data. These include adjusting the calibration to account for

19 distance from the radar site, using logarithms to remove exponential terms, and adjusting for mechanical problems with rain gages. There still is the concern, however, that an area of high reflectivity may not match up with an area of high precipitation at the surface. A situation such as this could occur with strong winds in the atmosphere causing precipitation to not fall directly downwards. This would be a factor in situations where the horizontal reflectivity gradient is very high. If this occurs, it is possible for a rain gage to receive high amounts of rainfall when an area of low reflectivity is observed on radar, or a small amount of rain at the surface being paired with a radar estimate of high rainfall. This can be prevented by setting a maximum on how much radar estimates can be changed due to calibration. The article states, however, that rarely is this situation actually a problem. In this study, each radar was calibrated separately and the calibrations were based on rainfall estimates from prior storms over a period of months. In some cases, the calibration process significantly and unreasonably inflated radar estimates. By changing a multiplier, these issues were accounted for to produce a more reasonable estimate. The results of this study show that some localized areas had their precipitation estimates increased significantly due to calibration while areas that received lesser amounts of precipitation had their estimates decreased. The article states that the calibrated estimates agreed closely with observed amounts of precipitation for the storm. Satellite data may also be incorporated, according to the article, to further increase the accuracy of calibrated estimates.

20 Background Literature Title: A Comparison of Radar Reflectivity Estimates of Rainfall from Collocated Radars This article describes attempts to improve radar estimates of precipitation and testing of a new radar technology by NCAR. The article starts by discussing potential problems with precipitation estimates. It discusses issues with bright bands along with several other possible sources of inaccuracy. One potential source of error highlighted is the limited resolution of radar at distant ranges. Also, at farther ranges where the beam passes through higher altitudes, if water droplets are frozen, reflectivities will be decreased resulting in underestimated precipitation. Another source of potential error is due to differences in raindrop size. The new radar system, referred to as S-Pol was compared against the existing WSR-88D system. The WSR-88D radars are equipped to deal with issues such as beam blocking and ground clutter, which the S-Pol radars were not. Several techniques were incorporated to reduce the effects of these problems that the S-Pol radars would encounter. The article states that calibration of rain gages with radar estimates of precipitation improves estimates. Also, tipping bucket rain gages may significantly underestimate precipitation if strong winds are present. The article states, therefore, that in these scenarios, the calibrated estimates should be the same or greater than the values produced from rain gages. The radars were evaluated around Denver and Wichita. The radars performed better in Kansas than in Colorado and the article offers several explanations for this. These possible explanations include smaller storm size in Colorado, higher cloud bases, more hail in Colorado storms, and the subcloud layer being drier. The article states that the radar estimates of precipitation were in good

21 agreement, usually, even when the estimates varied greatly from what was observed by rain gages. The conclusion drawn from the data is that the main factor in inaccuracies was because of differing raindrop sizes in storms. Well maintained radars should not have significant variations in precipitation estimates and differences in radars are unimportant compared to differences between storms, according to the article.

22 Background Literature Title: A Comparison of NEXRAD WSR-88D Radar Estimates of Rain Accumulation with Gauge Measurements for High- and Low-Reflectivity Horizontal Gradient Precipitation Events This article reports on a study which attempts to evaluate the effectiveness of the WSR-88D algorithm for estimating rainfall. The study attempts to measure the performance of the algorithm over many cases instead of highlighting a single case. The storms studied are divided into two major cases. One case is those with high horizontal reflectivity gradients and the other is with low horizontal reflectivity gradients. Data from several radar sites throughout the United States were used in the study. Cases with high reflectivity gradient tended to involve storms with cores that had high reflectivity values. The cases with low gradients tended to also have lower reflectivity values. One conclusion of this study is that variations between storms have a greater impact on differences in precipitation estimates than differences between radars. This study notes that the distance from the radar also affects the accuracy of the radar estimates of precipitation. Four regions are defined when analyzing these results, those being areas near to the radar, two mid-ranges, and the farthest distances from the radar. In cases of high reflectivity gradients, precipitation was underestimated close to the radar. In mid-ranges, precipitation was overestimated. At the farthest ranges, the estimates were similar to the totals actually observed. With low reflectivity gradients, radar underestimated the precipitation totals in all cases but to the greatest extent in the areas nearest to and farthest from the radar site. The article suggests that close to the radar, some legitimate echoes may be mistaken for ground clutter and incorrectly filtered by the algorithm. When the lowest

23 elevation detects ground clutter, the precipitation estimation algorithm chooses a higher angle of elevation. This can lead to underestimation. Underestimation of rainfall in cases of low reflectivity gradient is attributed somewhat to the likelihood that the precipitation was mostly of a stratiform nature. In this situation, if the radar beam is below or above the clouds, it may lead to poor estimates. The article also suggests that the zra and zrb values used to convert reflectivity values to precipitation estimates are optimized for cases of high reflectivity gradients and will not perform well in situations of low reflectivity gradients. Also cited as possible causes of inaccurate precipitation estimates are bright bands, different raindrop sizes, and radars that were poorly calibrated.

24 1.4. Goals & Prototypes The eventual goal of this project is to create an online product which can be viewed by members of the agricultural community to aid them in deciding whether to apply chemicals and fertilizer. As much of this product as possible should be generated automatically. This should product should integrate easily with existing GIS systems already present at the university. The first prototype, which has already been completed, converts radar estimates of precipitation into raw data which is stored in a text file and into a PNG file for easy viewing. Later prototypes will be used to experiment with different types of radar data which is available and different formats for which to convert data into for integration with GIS systems. Two radar products are being considered for use in this product. One product is referred to as the digital precipitation array. It has 256 levels of precipitation and is in a 131x131 grid. This product estimates the amount of precipitation that has fallen to the ground in the past hour. A second product is a one hour precipitation estimate

25 in a radial format. This product has 16 levels, instead of 256, but has a higher resolution. Each point along the grid is two kilometers long and one degree wide. While this provides much greater resolution than the digital precipitation array, it does not have nearly as good of precision. These products will be evaluated during the prototyping to determine which is more suitable for the end product.

26 1.5. Cost & Feasibility While the main coding project involved in this is to convert radar data, this project actually involves creating an entire system. The level III data used in this product is freely available from National Weather Service servers and can be downloaded in nearly real time. One component of this system is to create a way of automatically retrieving this data. This data must then be converted to a format which can be read by GIS software. The converted data must be then sent to another computer which has the GIS software. This other computer system will then integrate this data with other geographical data to create an end product which is available over the web. There are many servers already at the university and any one of them could be used to convert the level III radar data. It is not a particularly resource intensive task and could probably work alongside other processes. The data used is available at no cost and is small in size so bandwidth should not be a major issue. The GIS systems are already in use at the university, so little if any additional cost will be realized from this product. Overall, there are few costs from adding this product to the products already made available from the university. Because of the easy availability of this data, the low amount of computation required in processing this data, and the systems already in place to display this data after it is processed, this solution is very cost-effective while also providing a good product.

27 2. Requirements Analysis 2.1. Introduction The purpose of this project is to develop a system for decoding and converting precipitation estimates into a format that can be analyzed and displayed by GIS software. The goal of this, upon completion, is to generate a product which may be viewed online and is useful for predicting runoff and flooding. Precipitation estimates will be combined with terrain data such as elevation models and soil types to generate a composite product. This is the requirements analysis for this project and will define the environment for this product to operate in, explicitly what this product is, and how it will be evaluated in the future. During the planning of this project, several alternative possibilities were considered, and those will be presented as well, both for the purpose of explaining the decision process and for suggesting future work.

28 2.2. Overall Description As described previously, this project is to develop a system for the purpose of converting precipitation estimates to a format which is supported by GIS software. The precipitation estimates are provided by National Weather Service (NWS) radars in the form of level III data. When downloaded from NWS servers, the level III data is in a complex format which is partially run-length coded. One component of this system will be a program to automatically download level III data files for the radars which cover the state of Missouri. A major component of this system will be the software which decodes the data files and converts them to a format which GIS software supports. The conversion of level III data to another format will likely be done on a machine which does not have GIS software. Therefore this data will need to be automatically sent to another machine which does have the appropriate software. After this data is sent, the machine with the GIS software will need to incorporate the data to be displayed online. The website in which this data will be displayed already exists. If the site cannot currently be updated automatically to incorporate new data, this capability may need to be added. This is a simple description of the system as a whole and the major components within and how the components relate to each other within the system. The process is linear, will likely be done without any human intervention, and will need to be done quickly enough to provide reasonably current data on a website.

29 2.3. System Requirements and Constraints Operating environment The software produced will likely run on a UNIX machine. One goal of this product is to have minimal human involvement necessary to produce the product. Keeping this in mind, Windows does not have a reputation for being as reliable as UNIX. Keeping these in mind, it is reasonable to expect the software will be run on UNIX. It is, however, a goal of the project to attempt to maintain as much portability as possible. It is feasible in the future that changes will occur which will result in the product running on a different operating system such as Windows. It should be possible to run the product on a new operating system with little difficulty. It is expected that software will need to be recompiled to run in a new environment. This is acceptable and is a result of not writing the software in an interpreted language such as Java. The source code, however, should not require significant modifications to operate in a new environment, provided the environment meets some reasonable requirements. For example, appropriate compilers and environments exist on Windows to run a large portion of UNIX software. An environment with these tools should be able to run the software with little difficulty. The software needed to display the end results of this product in a web interface already exist. The product is to be displayed in a web page which already exists. Therefore, the web server is of little concern to this project. Once this product is delivered to the computers with GIS software, there should be little need for modifications. If the web server were to change, it should not affect the operation of the system. While the data generated may need to be stored within a database, it is not a concern of this project. Much of the GIS data is apparently already stored in a database and it is a concern of the GIS software. While development tools such as source control and integrated development

30 environments could be used in creating this software, it is not particularly useful. The software is not particularly complex and the use of these tools may introduce possibly undesirable complexity. Instead of maintaining a large and complex software package to interpret data, the goal is to develop a large number of modules for import and export of data with a simple core. Each module should be independent of other modules. By keeping the individual modules independent and simple, it should minimize or eliminate the need for complex development tools.

31 Market Users and Characteristics The product which is being provided will be publicly available without charge. Since it will be produced and provided by the University, there is little need to be concerned with competition in an open market. The product need not be profitable and should not be since it is funded by tax dollars through a University with a mission to serve the state of Missouri and specifically in the area of agriculture. Regulatory constraints should not be an issue in the development of this product. Since relatively little processing power is required to decode and convert level III data, it should already run on hardware owned by the University. The level III data files are relatively small files, particularly when compared to level II data, and should not require significant bandwidth to download. The GIS software which will decode this data and the webpage in which it is to be displayed already exist. Therefore, little additional expense will be realized by incorporating this data. Keeping this in mind, there is little economic expense in producing this product. Therefore, it is very feasible to produce this product using resources which are already available. There are two customers for this product that have different requirements. One customer is the Center for Agricultural, Resource and Environment Systems (CARES) which is an active participant in developing this product. They will, however, directly receive the decoded data files and have specified some requirements. The data must be provided on a timely basis and should be provided in a format which can be interpreted by GIS software. While several formats have been suggested, one that seems likely to be used is the shapefile format. These requirements are simple and will be met during the development of the software. The second customer in this is members of the agricultural community who will visit the website for the end product. One would expect that the requirements of this customer have already been met in the design of the website, which already exists.

32 Therefore, these requirements are not of concern to this project.

33 Environmental Constraints While it might seem likely that any system will require some human involvement, that is not a major concern in this project. The system is designed to operate with minimal human interaction. The system operates in a linear fashion with no major decision making. Both ends of the line are other systems, instead of being humans. One end is the NWS servers, from which data is automatically retrieved. The other end is the GIS software, which is also automated. Keeping this in mind, human involvement is not a major concern in this project. One obvious goal of this project is to produce quality data. The major factor in determining the quality of the data is the quality of the data actually collected by the radar. If the radar malfunctions and subsequently collects poor data, then there is no way this product will produce good data. If the radar is producing good data, which hopefully it is, then the quality of the data produced by the software will also be good. One would hope the NWS maintains their radars well, and that it is reasonable to expect quality data as output from this product. Reliability is a major goal of this product. It is important that data be processed in a timely fashion. Furthermore, it is important that this system require little if any maintenance and human involvement. A major threat to reliability is potential changes to the format in which level III data is disseminated. If this is changed, and the NWS does make periodic modifications to the format, it may also break the software which interprets and converts the data. This will adversely affect reliability and cannot be easily accounted for when planning this project except to make the source well commented so it can be modified as necessary with as little difficulty as possible. There are few if any safety issues involved in operating this system. Therefore, it is not a major concern when developing this product.

34 System Components As described previously, there are a few major components to this system which operate in a linear fashion. This means that when one portion of the system completes its task, the data is merely passed along to the next component. There are no major decisions to be made in the operation of the system. The first component of the system will retrieve data from NWS servers as necessary and as is available. This component must retrieve the correct data files from the servers, as many are available at a time. In excess of 200 files are usually available of a level III product for a given radar. That means that many thousands of data files are available from NWS servers at any given time. The correct one must be selected. Also, the downloading of data must not place unnecessary load on the University network connection or on NWS servers. NWS servers are not particularly fast and the component must consume as few resources as possible so the access of others to the data is not negatively impacted. Furthermore, if this component malfunctions, it could lead to the NWS banning the computer's IP from accessing the server. Also, it is entirely possible that radar data may not be available for significant lengths of time due to outages or other reasons. This component must detect this and behave in a sane way which enables the rest of the system to correctly function. Once this step is complete, the data is passed to a decoder. This component must decode the level III data and convert it to a format which can be understood by GIS software. The major requirement for this component is that it must correctly decode the data according to the standard for storing level III data. Unfortunately, the standard for level III data is vague and not particularly well documented. Therefore, some care must be taken and there must be rigorous testing to ensure that data is decoded correctly.

35 The third component of this data is responsible for the transmission of this data to the computer with GIS software. This component must transmit the converted data when it is available and must deliver the data in a timely fashion. This must deliver the correct data and ensure that the computer with GIS software correctly receives the data. Should either computer fail, this component should behave in a reasonable way and not cause harm. The fourth component of this data is the GIS system which takes the data and displays it within a website upon request. This component is already developed and is not within the scope of this project. While there are some obvious requirements of this component, both in the resources required and in the website produced, it is not within the scope of this project to modify this component or specify requirements.

36 Software Interfaces and Libraries There are two major interfaces in this software. One is the interface with the NWS servers and the requirements for this interface have already been specified. The other interface is responsible for delivering the converted data to the GIS system. The requirements for this interface have already been specified. The main requirements for these components is that when one fails, the rest of the system behaves in a reasonable way and does not cause damage. If possible, the rest of the system must continue to function. If it is not possible, the rest of the system must be ready to function as soon as the failing component becomes available once again. It is likely that only one external software library will be used in this system. That library is for creating a shapefile, or possibly a file of another GIS format. These libraries should be free and open source. If a shapefile is the format chosen for the output of data, there is already an existing library called shapefile. This library has an interface from C and is free and open source. This makes it a good choice for this project.

37 Communication Interfaces The likely communication interfaces in this project are the various network links. One interface is an external network interface for downloading level III data from NWS servers. This network link is already available from the University. The other interface will be across the University network for the purpose of sending data from the server responsible for converting the data to the server responsible for displaying the data. Once again, this network link is already available and need not be a concern of this project.

38 Hardware Interfaces The scope of this project does not involve hardware. Therefore, it is not a concern of this project and requirements will not be specified.

39 Software Maintenance, Life Cycle and Support Clearly, this project will need to be maintained as the NWS makes changes to the format in which they distribute radar data. Unfortunately, the NWS does not make these updates on a regular basis and may not make the specifications on the changes easily available. Furthermore, as bugs are found, these will need to be fixed if they are critical. A major part of this maintenance is to make the code easily understandable by programmers in the future who may be responsible for the software. This may be accomplished by properly commenting the code and by organizing the code in a way so that functions and functionality can be easily found by someone examining the source. Also, a goal of this project is to make the code reasonably portable so that it can run on newer hardware when it is time for an upgrade. This will extend the lifecycle of the software beyond current hardware and allow it to be implemented on other machines if necessary. While it is likely that this software will eventually need to be replaced or rewritten, it will be designed with an architecture that will extend its life as long as possible. By using modules to load and export the data, there may be new applications to this software that have not been envisioned at the time of this writing.

40 2.4. Performance Requirements The performance requirements for this system are not particularly strict. Since the precipitation estimates are hourly and new data will only be input every hour, it isn't a major problem if the efficiency isn't particularly great. On the other hand, the software should still run reasonably well because other software may be running on the same machine along with it. Considering that level III data is not particularly large, because of the lack of many elevation angles, the memory requirements of the program should be relatively low. For example, it is acceptable to use a few megabytes, but not more than that.