ILLINOIS STATE WATER SURVEY. at the. University of Illinois Urbana, Illinois A STATISTICAL METHODOLOGY FOR THE PLANNING AND EVALUATION

Transcription

1 ILLINOIS STATE WATER SURVEY at the University of Illinois Urbana, Illinois A STATISTICAL METHODOLOGY FOR THE PLANNING AND EVALUATION OF HAIL SUPPRESSION EXPERIMENTS IN ILLINOIS by Paul T. Schickedanz, Stanley A. Changnon, Jr., and Carl G. Lonnquist PART 2 of FINAL REPORT ON HAIL EVALUATION TECHNIQUES Principal Investigator: Stanley A. Changnon, Jr. National Science Foundation Atmospheric Sciences Section NSF GA-482 April 14, 1969

2 ACKNOWLEDGMENTS This research was supported largely by funds from the Atmospheric Sciences Section of the National Sciences Foundation, NSF GA-482. Additional support and data came from the Crop-Hail Insurance Actuarial Association and the State of Illinois. The work was done under the general supervision of Glenn E. Stout, head of the Atmospheric Sciences Section of the Illinois State Water Survey. Useful suggestions were furnished by Floyd A. Huff and J. Loreena Ivens of the Survey staff, and the drafting of all figures was supervised by John W. Brother, Jr. Mrs. Edna Anderson performed many of the computations throughout the report. Students who materially assisted in the research were Ruth Braham, Pamela Collins, Robert Kuchnaw, and Karen Wilde. We greatly appreciate the advice given by Dr. Fred C. Bates of St. Louis University, Boynton W. Beckwith of United Air Lines, Dr. Roscoe R. Braham of the University of Chicago, Dr. John A. Flueck of Temple University, H. C. S. Thorn of the Environmental Science Services Administration and Dr. Horace W. Norton of the University of Illinois.

3 CONTENTS Page INTRODUCTION 1 HAIL-DAY DATA 3 1. Data and Analytical Procedures 4 2. Natural Variability 4 3. Theoretical Frequency Distributions 9 4. Experimental Design and Tests of Hypothesis Results for the Weather Bureau Hail-Day Data 19 CROP-HAIL INSURANCE DATA Data and Analytical Procedures Natural Variability 36 Days with loss 36 Dollars and acres of loss Theoretical Frequency Distributions 43 Yearly data 43 Daily data Experimental Design and Tests of Hypothesis 59 Yearly data 59 Daily data Results for the Yearly Data Results for the Daily Data 74 Algebraic computations 74 Monte Carlo trials Summary of Results Using Yearly and Daily Insurance Data 88 NETWORK HAILSTREAK DATA Data and Analytical Adjustments Natural Variability of Hailstreak Area and Energy Values Experimental Design and Empirical Frequency Distributions Test of Hypothesis Results for the Paired Hailstreak Data 107 RESULTS OF APPLYING THE STATISTICAL METHODOLOGY Comparison of Various Tests, Designs, and Types of Data Application of Methodology for Other Climatic Areas 116 iii

4 Page SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS Summary Conclusions Recommendations for Additional Research 120 BIBLIOGRAPHY 120 APPENDIX 124 iv

5 TABLES Table Page 1. Number of hail days per area in 5-year periods during Kolmogrov-Smirnov "goodness of fit" test for the Weather Bureau hail days Test for randomness of the climatological data series for Weather Bureau hail days Average and extreme number of days of loss per month Annual and daily loss values for study areas Correlation coefficients between annual number of days of loss, annual dollar losses, and annual acre losses Correlation coefficients of annual loss values between areas Test for randomness of the climatological data series for yearly insurance data Sample estimates of the gamma and log-normal distributions for yearly insurance data Kolmogrov-Smirnov "goodness of fit" test for yearly insurance data Sample estimates of the log-normal parameters for daily insurance data Chi-square "goodness of fit" test for the daily insurance data Kolmogrov-Smirnov "goodness of fit" test for the insurance damage days The least squares simple regression parameters for Monte Carlo trials of daily insurance data (a = 0.05) The least squares simple regression parameters for Monte Carlo trials of daily insurance data for a 40% decrease Comparison by ranking of areas and type of data measurement for a 20% decrease using the gamma test Comparison by ranking of the type I and type II errors with the experimental units for a 20% decrease and using the sequential normal test in area Comparison by ranking of theoretical distributions with design and test for a 20% decrease at an a level of 0.05 and a ß level of v

6 Table Page 19. Rank comparison of the experimental unit with a given decrease and type of data measurement using a 1-sample test, sequential analysis, and a continuous design in area Number of years required to detect various decreases in daily insurance data for selected designs and a = Comparison between algebraic and Monte Carlo estimates of years to detect various decreases, for an a level of 0.05 and for area 3 daily monetary insurance data Number of complete hailstreaks, no-hail rain cells, and rain cell combinations possible in study area during Median and extreme values for 77 hailstreak areas and energies Comparison of areas with the "best" test and design of various types of data measurement for detecting a 20% decrease (a = 0.05, ß = 0.20) Comparison of the type I and type II errors and experimental unit with "best" tests for various types of data and a 20% decrease Number of years required to detect various decreases using "best" designs for each type of hail data (a = 0.05) 114 vi

7 ILLUSTRATIONS Figure Page 1 Location of the Weather Bureau study areas 5 2 Recurrence interval for Weather Bureau summer hail-day data Temporal variability of Weather Bureau hail-day data 8 4. Frequency distributions of Weather Bureau spring hail-day data Frequency distributions of Weather Bureau summer hail-day data Frequency distributions of Weather Bureau fall hail-day data Frequency distributions of Weather Bureau annual hail-day data Illustration of the sequential test procedure Comparison of the number of years required to obtain significance for various tests and experimental designs using Weather Bureau hail-day data in area 5 (summer and annual data - a =.05) Comparison of the number of years required to obtain significance for different areas using Weather Bureau hail-day data and a poisson 1-sample sequential test (20% decrease - a =.05) Comparison of the number of years required to obtain significance for different decreases using a poisson 1-sample sequential test and Weather Bureau hail days in area 5 (a =.05) Comparison of the number of years required to obtain significance for different type I errors using a poisson 1-sample sequential test and Weather Bureau hail days in area 5 (20% decrease) Comparison of the number of years to obtain significance for the null and alternative hypotheses for the poisson sequential test using Weather Bureau hail days in area Poisson 0C function for Weather Bureau hail-day data in area Poisson ASN function for Weather Bureau hail-day data in area Study areas of crop-hail insurance data Annual amount of dollar loss in areas 1 and Annual amount of acre loss in areas 3 and Histograms and theoretical frequency curves for yearly hail insurance data (dollars loss) Histograms and theoretical frequency curves for yearly hail insurance data (acres damaged) Histograms and theoretical frequency curves for daily monetary insurance data 50 vii

8 Figure Page 22. Histograms and frequency curves for daily acreage damaged insurance data Frequency distributions of hail damage days for area Frequency distributions of hail damage days for area Frequency distributions of hail damage days for area Frequency distributions of hail damage days for area Comparison of the number of years required to obtain significance for various tests and designs using yearly insurance data in area 1 (dollars) Comparison of the number of years required to obtain significance for various tests and designs using yearly insurance data in are a 1 (acres) Comparison of the number of years required to obtain significance for yearly insurance data for different areas (20% decrease) Comparison of the number of years required to obtain significance for yearly insurance data for different decreases in area 1 (dollars) Comparison of the number of years required to obtain significance for yearly insurance data for different type I errors in area 1 (20% decrease) Comparison of N 0 and N 1 for the 1-sample sequential tests using yearly acreage data in area 1 (continuous design) Comparison of the number of years required to obtain significance for various tests and designs using daily insurance data in area 1 (dollars) Comparison of the number of years required to obtain significance for various tests and designs using daily insurance data in area 1 (acres) Comparison of the number of years required to obtain significance for the different areas using daily insurance data (20% decrease - acres) Comparison of the number of years required to obtain significance for different decreases using daily insurance data in area 1 (acres) Comparison of the number of years required to obtain significance for different type I errors using daily insurance data in area 1 (20% decrease) 83 viii

9 Figure Page 38. Comparison of the number of years required to obtain significance for the null and alternative hypotheses of the normal sequential test using daily insurance data in area 1 (continuous design - acres) Comparison of the number of years required to obtain significance for various tests and designs for Monte Carlo trials of daily insurance data in area 3 (dollars - a =.05) Comparison of the number of years required to obtain significance for various tests and designs for Monte Carlo trials of daily insurance data in area 3 (acres - a =.05) Comparison of the number of years required to obtain significance for Monte Carlo trials of monetary and acreage daily insurance data in area 3 (a =.05-40% decrease) Comparison of the number of years required to obtain significance for different decreases in Monte Carlo trials of daily insurance data in area 3 (dollars and acres - a =.05) Comparison of the number of years required to obtain significance for different significance levels for Monte Carlo trials of daily insurance data in area 3 (40% decrease) Hail observing points in study area and complete hailstreaks on 9 June Log-normal frequency distributions of the areal extent of hailstreaks and energy of hailfall within the hailstreak Empirical distributions of areal extent and energy of hailfall Comparison of the number of years required to obtain significance for various significance levels for the paired storm design using energy and areal extent as parameters (20% decrease) Comparison of the number of years required to obtain significance for various decreases for the paired storm design using energy and areal extent as parameters Schematic representation of validity zones for various tests and designs 119 ix

10 INTRODUCTION This report is Part 2 of the final report to the National Science Foundation of the Hail Evaluation Techniques Project, NSF GA-482. This report describes one of the four phases in the project. This phase is concerned with the use of historical hail data in Illinois to determine the necessary length of hail suppression programs to detect significant seeding effects. The results can be used for the planning and evaluation of future suppression programs. Part 1 of the final report describes the results of the other three phases of the project. Most prior and current hail suppression activities in the United States have often been plagued with controversies and questionable results common to rain-enhancement efforts (Hagen and Butchbaker, 1967, Stout, 1961). One scientifically oriented project in Colorado did show reduction in hail intensity over a 5-year period (Schleusener and Auer, 1964). However, the relative infancy of hail suppression activities suggests that preliminary statistical studies in regard to data collection, size of study area, statistical design, and duration of hail suppression experiments should be performed prior to actual experimentation. Such studies should serve to eliminate some of the problems that have plagued many precipitation modification experiments. The evaluation of a cloud seeding experiment to increase precipitation or to decrease hail is a problem of tremendous complexity. For example, many physical and biological experiments can be conducted under various controlled conditions, but in the case of a weather modification experiment, many of the important variables such as pressure, temperature, and wind cannot be controlled. Present day forecasting methods are not sensitive enough to predict the amount of rain or hail that would have fallen had cloud seeding not been conducted. To resolve this evaluation problem, experimenters have turned largely to one of two basic experimental plans with their various modifications in order to attach statistical significance to the results. The first of these is the target-control method. In this design, the rainfall or hailfall in one area (the "seeded" or target area) is compared with that in a "control" area in which it is assumed that none of the seeding agent is present. The comparison is

11 -2- usually achieved by 1) the construction of a regression line between the historical rainfall data of the target and that of the control area, and 2) the determination of departures of the seeded observations from the line, these departures in turn being tested to determine if they are likely to have occurred by chance. If a successful application of this method is to be made, there must be a high degree of correlation between the rainfall or hailfall of the two areas. An often used variation in this method is to construct a regression line for both the seeded and non-seeded years and test to determine if the two lines coincide (Dennis and Kriege, 1966). The second frequently used plan for precipitation modification experiments employs randomization of seeded and non-seeded days over a single (target) area. This plan allows for proper randomization and several statistical tests may be employed. The Arizona experiment (Battan, 1966) and the Project Whitetop experiment (Braham, 1966) were designed along these lines. A modification of this method is one in which two nearby experimental areas are seeded with random choice of area, which is called the crossover design (Smith et al., 1965). A 2.5-year project designed to study techniques for evaluating potential hail suppression activities in Illinois was begun during 1966 with primary support from the National Science Foundation under Contract NSF GA-482 (Changnon, 1967d). One major phase of this project concerned the study of all available historical hail data in Illinois with the primary purpose of using these data 1) to choose the optimum type of statistical design for field projects, and 2) to define the duration of an experiment needed to detect various degrees of change that might be produced by suppression efforts. It was ascertained that there are only two types of long-term historical hail data available in Illinois and in most other areas the U. S. Weather Bureau point (station) records of hail days, and the crop insurance records of monetary loss and areal extent of damage. A third type of data became available from the operation of a 400-mi 2 dense rain-hail network in east central Illinois during the first year of the project (1967). Individual hailstorm areas (hailstreaks) were carefully delineated from the network data, and although no long-term historical data were available for hailstreaks, these data were employed in the study to furnish desired information for potential projects involving paired storm designs.

12 -3- This report (Part 2) contains the development of a statistical methodology for the planning and evaluation of hail modification experiments in Illinois (and areas of similar hail climate) involving the above mentioned sources of data. Nomograms based on Illinois data were constructed which give the length of time necessary to verify different levels of hail reduction for a specified type I and type II error, for daily summer seeding periods, for annual seeding periods, for different sized areas in different locations within the state, and for different statistical designs. Several levels of reduction (5, 10, 20, 40, 60, and 80%) were assigned uniformly to the various hail data. The hail data were expressed as areal totals and the areas studied ranged from 400 to 4000 mi 2. These were chosen to match sizes of past hail suppression experiments and those likely to be used in future experiments. The first, second, and third sections of this report treat the analyses of the Weather Bureau hail-day data, the crop-hail insurance data, and the network hailstreak data, respectively. In each section detailed descriptions are presented of the data, the natural variability of the phenomena, the analytical techniques employed, the theoretical frequency distributions of the data, and the results of applying the methodology. The results contain nomograms of the sample size required for the detection of various reduction levels for various sizes and locations of data areas, for various seasons and combinations of experimental designs, and for various statistical tests. The fourth section of the report presents a comparison of the various statistical tests, designs, and data sources, plus a discussion of applying the methodology developed. The last section is a conclusion with recommendations for future research, and the appendix contains detailed listings of data employed in the study. HAIL-DAY DATA This section is an investigation of Weather Bureau hail-day data for possible use in weather modification programs. This study was made because relatively long historical records are available and easy to obtain. Theoretical distributions were fitted to data for spring, summer, fall, and annual hail days to describe their variability. The fitted distributions of

13 -4- summer and annual data were used to determine sample size for the 1-sample poisson and negative binomial tests using random and continuous designs. The test procedure used was the sequential analysis approach. Portions of this study were used to prepare a research paper (Changnon and Schickedanz, 1969). 1. Data and Analytical Procedures Previous research with historical hail data in Illinois has shown that 1) 85 cooperative substations of the U. S. Weather Bureau have quality hail-day records of at least 15-yr duration in the period (Changnon, 1967a) and 2) data are available for 10 first-order stations in and adjacent to Illinois. Examination of all Illinois stations with quality hail data during the period, when the greatest density of stations was available, indicated that there were three regions with relatively high station densities. Five of the stations in these three regions encompass areas of nearly equal size, and a boundary was constructed for each to delineate an area of 1000 mi 2 that was generally oriented SW-NE. The names and locations of the stations in these three areas (areas 1, 2, and 3) are depicted in Fig. 1. When three other stations north of area 1 were combined with those in area 1, a 3000-mi 2 area was formed (area 4, Fig. 1). Four stations in central Illinois with records for the period were used to define a 500-mi 2 area, labeled area 5. Areas ranging from 500 to 3000 mi 2 were chosen to match sizes of areas that might likely be selected for hail suppression experiments. The dates of hail at each station in an area were combined to develop a list of hail days for each area. The area data were summarized for each season and on an annual basis (see appendix, Table A). However, results are presented in the report only for two periods for the summer (June-August) to show results during the crop-damage season, and for the entire year to provide results relating to crop and property damage throughout the year. 2. Natural Variability Average and extreme hail-day values for the five areas and the summer and total period, as based on 5-yr periods, are shown in Table 1.

14 -5- Figure 1. Location of the Weather Bureau study areas.

15 -6- Table 1. Number of hail days per area in 5-year periods during Summer Area 1 Area 2 Area 3 Area 4 Area 5 Total Median Average Maximum Minimum Median Average Maximum Minimum Comparxson of either the summer or annual median or average values for the three 1000-mi 2 areas (areas 1, 2, and 3) reveals little significant difference between the areas. All three areas are located in relatively high hail frequency areas of Illinois (Changnon, 1967a). The differences in area size affect the frequency as expected, with the lowest averages from the 500-mi 2 area (area 5) and the highest from the largest area (area 4). Although the three equal-sized areas had similar averages, their extremes were considerably different. For instance, area 3 had a 5-yr maximum of only 15 summer hail days, whereas area 1 had a maximum of 25 days and area 2 had 28 days. The frequency distributions of summer hail days were used to construct recurrence interval graphs for each area and station, and those for the five areas are portrayed in Fig. 2. At least once in 10 years the smallest area (area 5) will experience 3 or more hail days in summer, whereas the largest area (area 4) will have 6 or more hail days. Hail-day differences in the three 1000-mi 2 areas also are revealed in Fig. 2. The tendency for area 2 to have some very large and very small summer hail-day values is reflected by the steepness of its curve. The once-in-50-yr value in area 2 is 15 hail days, whereas that in area 1 is 9 and that for area 3 is 8 hail days. However, the once-in-2-yr value for area 2 is lower than those of the other two areas. The temporal variability of the area hail days is displayed in Fig. 3, which has curves based on 5-yr moving totals. The curves for areas 1-4 all display low values for the periods ending in , and these low values

16 -7- Figure 2. Recurrence interval for Weather Bureau surrmer hail-day data.

17 Figure 3. Temporal variability of Weather Bureau hail-day data.

18 -9- are related to the statewide low incidences of hail accompanying the droughts of this period (Huff and Changnon, 1959). All curves show a maxima for the 5-yr periods ending in the period, but thereafter the shapes of certain curves are not comparable. Area 1 shows an almost constant decrease in hail days reaching a minimum in 1959, whereas area 2 shows a constant increase through Interestingly, the curve of area 3 (the other l000-mi 2 area) exhibits features of both, relating well (decreasing) to area 1 from 1945 through 1952, but with a trend of increase similar to that of area 2 after Measures of the natural spatial variability of hail days and the areal extent of hail within the areas were obtained from correlations between the annual hail-day frequencies of pairs of stations in each area. The coefficients between most stations were less than 0.5, indicating very little relationship between their hail-day frequencies. The adequacy of the data from a few point records to represent all the regional hail days was investigated. Initially, the sampling adequacy of the 8 stations in area 4 was checked by a process of data deletion (Changnon, 1967c). Area-mean averages of summer hail days were developed using combinations of any 2 stations, any 3 stations, and on through the 8 possible stations. These averages displayed a curvilinear trend, and a quadratic equation was developed from them. Its solution showed that the highest (true) summer average for a 5-yr period was 19 hail days (2 more than that from the 8 stations, Table 1), and that 12 stations in the area were necessary to achieve true sampling of hail days within the 3000-mi 2 area. This station frequency indicated a density of 1 station per 250 square miles. The five stations in areas 1, 2, and 3 represented densities of 1 station per 200 square miles, and the four in area 5 represented a density of 1 station per 125 square miles. Thus, it appears that the sampling densities in areas 1, 2, 3, and 5 were adequate to define the hail days in these areas, whereas those in area 4 underestimated the frequency of summer hail days by about 12%. 3. Theoretical Frequency Distributions The well-known poisson distribution was fitted to the seasonal and annual hail-day data for the 5 areas in Figs. 4-7 to describe the variability and to form the basis for subsequent statistical evaluation procedures. The probability function for this distribution is:

19 -10- Figure 4. Frequency distributions of Weather Bureau spring hail-day data.

20 -11- Figure 5. Frequency distributions of Weather Bureau summer hail-day data.

21 -12- Figure 6. Frequency distributions of Weather Bureau fall hail-day data.

22 -13- Figure 7. Frequency distributions of Weather Bureau annual hail-day data.

23 -14- where: e = the base of the natural logarithms µ = the average number of hail days per year x = the number of hail days in a year To determine if the poisson distribution is adequate for describing the hail-day data, the sample data were tested for equality of the sample mean and variance (Thorn, 1957a). The distributions were then tested for "goodness of fit" with the Kolmogrov-Smirnov test. This test was used because it is more valid for small samples than is the Chi-square test. Since both parameters of each distribution are estimated from experimental data, the more common tables of D n were not used nor are they valid. Liffiefors (1967) has recently computed new tables which take this factor into consideration. If the data were inadequately described by the poisson distribution, they were then fitted by the moment estimates of the negative binomial distribution. The probability function for this distribution is: where: x = the number of hail days in a year a parameter of the distribution and a measure of the correlation between storms, where S 2 is the variance of number of hail days in a year and is the average number of hail days per year a parameter of the distribution which is proportional to the mean r = the complete gamma function The moment estimates of the negative binomial distribution were then tested for efficiency. If the efficiencies of the moment estimates were unsatisfactory, the maximum likelihood estimates of the negative binomial distribution were used, as suggested by Thorn (1957a).

24 -15- The results of this procedure are shown in Table 2 along with the parameter estimated. If the often cited 0.05 significance level is used, it is seen that of the 20 distributions fitted (spring, summer, fall, and annual for the 5 areas), 13 were fitted by the poisson, 2 by the moment estimates of the negative binomial distribution, 5 by the maximum likelihood estimates of the negative binomial distribution, and 1 (spring season in area 2) did not fit either distribution. A tendency is shown for the summer data to be fitted by the negative binomial distribution and the annual data to be fitted by the poisson. This occurs because summer data are more likely to be a series of dependent events, and hence a distribution such as the negative binomial which allows for dependence is required. This tendency was hypothesized by Thorn (1957a) and was shown in his data. 4. Experimental Design and Tests of Hypothesis Before valid statistical tests and conclusions can be drawn from the data, the data must be tested for randomness in the climatological data series. The procedure followed was that described by Swed and Eisenhart (1943). The results of this test procedure are in Table 3. The probabilities of obtaining a "U" smaller than or greater than expected are 0.17 or more for all data except the annual data in area 2 and area 3. Therefore, at the often cited 0.01 and 0.05 levels of significance, the null hypothesis of homogeneity of the climatological data series would be accepted for all areas and the alternative hypotheses of trend would not be accepted, except for the annual data of areas 2 and 3. Therefore, the latter two data sets are non-random climatological series. The lack of homogeneity of the series is not a serious hindrance to the application of the statistical tests so long as proper randomization is present in the design and the analysis is the classical non-sequential analysis. However, for continuous designs without randomization and when the test procedure is based on the sequential analysis, the presence of trend in the data can lead to erroneous conclusions. The distributions for the annual data from areas 2 and 3 were used in subsequent statistical tests, but the interpretation of results from the two data sets must allow for the factors discussed above.

25 Table 2. Kolmogrov-Smirnov "goodness of fit" test for the Weather Bureau hail days. n Spring hall days >0.20 Summer hail days * >0.20 Fall hail days >0.20 Annual hail days >0.20 Spring hail days * 1.30 < <0.01 Summer hail days * 2.20 < >0.20 Fall hail days * 1.79 < >0.20 Annual hail days < >0.20 Spring hail days >0.20 Summer hail days >0.20 Fall hail days >0.20 Annual hail days >0.20 Spring hail days >0.20 Summer hail days >0.20 Fall hail days >0.20 Annual hail days >0.20 Spring hail days >0.20 Summer hail days >0.20 Fall hail days Annual hail days >0.20 If P(x 2 > x 2 o ) is low, reject the hypothesis that i.e., the poisson distribution is inadequate. If C > 20, the moment estimates of the negative binomial are sufficient. If P(D > D n ) is low, reject the hypothesis that the data fit the specified distribution. * Maximum likelihood estimate required for negative binomial estimate. Probability of obtaining a larger value of Chi-square from random sampling.

26 -17- Table 3. Test for randomness of the climatological data series for Weather Bureau hail days. U o Observed no. N a of runs above No. of observations Area Season and below median above median P(U U o ) P(U > U o ) 1 Summer Annual Summer Annual 4 15 <0.01 > Summer Annual Summer Annual Summer Annual If P(U U o ) is low, a trend is present. If P(U > U 0 ) is low, oscillations are present. In choosing the optimum design and estimating the minimum duration of a hail modification experiment, the first necessity is to obtain an estimate of the distribution parameters for days which would have been seeded. be done by one of the following methods (Schickedanz, 1967): the seeded sample using the Monte Carlo technique with various This can 1) simulate changes in the distribution parameters; 2) assume that the number of hail days for each period was decreased a certain percentage each year, or 3) present the number of hail days in terms of a particular statistical test. In this last method, it is sufficient to first compute the components of the test for a sample of the non-seeded distribution. Then, with certain assumed changes in the distribution parameters, the differences required for significance can be obtained through algebraic relations. with hail-day data is Since the only effect that can be tested a reduction or increase in the total or average number of days, method 3 was chosen for the Weather Bureau hail-day analysis. Once this was chosen, four different seeding designs were considered. One possible design using the hail-day data is that in which individual storms are seeded. However, the hail-day data in any area are too sparse to define characteristics (areal size or intensity) of individual hailstorms

27 -18- (Changnon, 1968b) and thus cannot be used in evaluating project designs involving seeding of individual storms. Another possible design is one in which the seeded days are selected at random. However, this design reduces the sample size to one-half if the randomization factor is one-half. A hail day is an event too infrequent in most 1500 to 4000 mi 2 areas of Illinois to squander approximately half of the potential (forecasted) hail days to "no-seed" trials. A design in which the yearly unit (total or average number of hail days per year) is randomized would be completely erroneous. The third type of design considered was the "target-control" continuous seed regression approach wherein the data from the seeded days are compared with the data from a nearby control area. However, a relatively large correlation coefficient (r) between hail-day frequencies of the two areas is necessary for a target-control approach to be effective. The quantity (1-r 2 ) is the percentage of the total variation in the variable of one area (target) that is unexplained by the occurrence in another area (control). Thus a correlation coefficient of indicates that 64% of the target variable is unexplained, and that the unexplained variation is reduced only 36% when an areal control is employed. The correlation coefficient between the annual hail days of areas 2 and 5, the two nearest non-overlapping areas, was only +0.57, and those between all other possible areas were less. Hence, the target-control design was discarded for use with the hail-day data. A similar conclusion was reached from the Colorado hail suppression experiments, which employed other measures of hail (Schleusener et al., 1965). The final design considered in reference to hail-day data was a continuous seeding design on a single area without any control area. Thorn (1957b) has suggested that the lack of an areal control can be compensated for by the favorable aspects of the sequential analysis approach. Thus, this final statistical design involving continuous seeding on all potential forecasted hail days in an area was considered in the context of a sequential analysis. This design can be based on hail-day frequencies, and potential frequency changes from seeding can be evaluated with a statistical technique suggested by Thorn (1957b). This approach is based on the sequential analysis test procedure (Wald, 1947). In the non-sequential analysis, the a (type I error)

28 -19- and N (number of samples) are fixed, and ß (type II error) is the dependent variable. In the sequential test, a and ß are fixed and the observations are tested sequentially. The test is made by constructing, from theoretical considerations (Thorn, 1957b), acceptance and rejection lines for the sequential test. These lines are shown in Fig. 8 as two parallel lines. As each sample is drawn, the accumulation of hail days by years ( x) is plotted against the year number m. If the accumulation curve falls above the region (band) formed by the acceptance and rejection lines, the suppression effect is rejected. If the curve falls below the band, the suppression effect is accepted, and if the accumulation curve falls within the band, the experiment is continued. If the sequential test is applied to a number of sample sequences from the same underlying distribution, the test would terminate at various year numbers (m). These m's would then form a frequency distribution with some mean value. This mean is designated as the average sample number (ASN), and the computation of it prior to experimentation yields the average number of years required to come to a decision. Theoretical ASN equations for the poisson and negative binomial distributions are available in the literature (for details of applying the test, see Thorn, 1957a). On the average, the sequential method of testing requires fewer observations than non-sequential methods (Wald, 1947). The ASN's were computed for the various areas using the summer and annual data. The results indicated that this test is the optimum one to use if the assumptions involved in the sequential analysis are met by the data sample being tested. 5. Results for the Weather Bureau Hail Day Data The comparison of the number of years required to obtain significance for the continuous and random experimental designs for summer and annual data is shown in Figs. 9a-f. The sequential continuous test requires less time to obtain significance than comparative tests for the random design, and the annual data require less time than the sammer data. For instance, for a 20% reduction and a ß value of 0.20, 78 years are required to obtain significance for a random design involving summer hail (Fig. 9c); whereas 39 years are required for a continuous design using summer data, this same 39-yr summer data value compares with 11 years for a continuous design with annual data or a difference of 28 years.

29 -20- Figure 8. Illustration of the sequential test procedure.

30 -21- Figure 9. Comparison of the number of years required to obtain significance for various tests and experimental designs using Weather Bureau hail-day data in area 5 (summer and annual data - a =.05).

31 -22- The comparison of the number of years required to obtain significance in the Weather Bureau hail-day data for various areas is depicted in Figs. l0a-d. Results for summer data (Figs. 10a and 10c) reveal significant geographical differences at the 20-% reduction level. For instance, the l000-mi 2 areas (areas 1, 2, and 3) require 40, 80, and 30 years, respectively, for a continuous design with a = 0.05, and ß = Man-made decreases in summer hail frequencies in areas 3 and 4 would require less time to detect than comparable ones in the other areas, and changes in area 2 would require considerably more time than in the other areas. The lowest summer values in Fig. 10 are for the largest area (area 4) which has the largest average number of hail days (Table 2), and this suggests that as the average of an area is increased, less time is required to obtain significance. Although not shown in the figures, the negative binomial test requires more observations than the poisson test. When the maximum likelihood solution is required, even more time is required to obtain significance. For the annual data (Figs. 10b and 10d) the size of area is no longer as important as it was with the summer data. The smallest area (area 5) requires the least amount of time to obtain significance. Results for the annual hail-day data indicate that areal variations are masked by other factors. The considerably higher detection values for area 2 were carefully investigated, including a test for randomness in its climatological series (Table 3). The summer data for this area had an upward trend in the period, although the trend was not significant at the commonly used 0.10 level for random climatological series, (Thorn, 1966). However, the annual series for area 2 as well as that for area 3 was found to have significant upward trends (Fig. 3), and the sequential test is known to be sensitive to non-randomness in the climatological data series. Therefore, the larger number of years in area 2 could be attributed to these trends. The fallacy of this argument is that the other area, area 3, needs little time. However, it should be noted that since the poisson distribution was sufficient for the area 3 annual data, area 3 would require lower observations for this reason alone. Also, the trend toward non-randomness in the data is much stronger in area 2 than in area 3 (Fig. 3, Table 3). Nevertheless, the fact that an upward trend is present and a downward trend is being tested implies that the test is conservative; that is, the

32 Figure 10. Comparison of the number of years required to obtain significance for different areas using Weather Bureau-hail-day data and a poisson 1-sample sequential test (20% decrease - a =.05).

33 -24- numbers from Fig. 10b represent, on the average, the maximum number of years required to obtain significance. It is very likely that the area 2 values in reality are less than indicated. The comparison of the number of years required to obtain significance for the various decreases is shown in Figs. 11a-d. Results for the various reductions in hail days from continuous seeding designs reveal that it would be easier to detect comparable reductions in the annual data than in the summer data. For instance, to prove a 20% reduction in the summer data from area 5 (500 mi 2, ß = 0.20) would require, on the average, 40 years, whereas proof of the same reduction in the annual data (seeding on all hail days) would require only 11 years. Values presented in Fig. 11 also show dramatic decreases in the number of years required to obtain significance of a given ß level as hail-day reductions change from 20 to 40%. If hail suppression projects could produce 60-% or greater reductions in hail days in Illinois or other locales with comparable climate and topography, then proof could be shown in relatively short periods of time, 5 years or less for a reasonable choice of type II error. A comparison of the number of years to obtain significance for different type I errors is shown in Figs. 12a-d. For a very conservative sequential test, a = 0.01 (Fig. 12a), 65 and 18 years are required to obtain significance for g = 0.1 and 0.5, respectively. For a more liberal choice of the 0.10 significance level, 51 and 11 years, respectively, are required to obtain significance for 20% decreases in summer hail days and a continuous seed design. A sequential test always carries the concern that the test may not terminate and no decision regarding seeding effects will be made; that is, the accumulation of hail days by years will remain in the band formed by the rejection and acceptance lines (Fig. 8). An even greater concern is that because of the a and g choice, and the decrease that is being tested, the accumulation may fall above the band and the test will terminate with a decision to reject the seeding effect. The number of years required to obtain significance (N o ) in the previous figures of this section represent the number required to obtain a decision to accept the suppression effect; that is, the accumulation will fall below the band formed by the rejection and acceptance lines. However, in reality some of the numbers may be fictitious in that the test would have terminated sooner with the decision of rejecting the seeding effect.

34 Figure 11. Comparison of the number of years required to obtain significance for different decreases using a poisson 1-sample sequential test and Weather Bureau hail days in area 5 (a =.05).

35 Figure 12. Comparison of the number of years required to obtain significance for different type I errors using a poisson 1-sample sequential test and Weather Bureau hail days in area 5 (20% decrease).

36 -27- Figs. 13a-f represent a comparison of N o with the number of years (N ) required to reach the decision of rejecting the seeding effect. The shaded regions in Fig. 13 represent the domain in which the sequential test would have terminated prior to the acceptance of the seeding effect. For decreases of up to 60%, this region never extends beyond type II errors of 0.15 for the 0.05 significance level. In Fig. 13c there is a comparison between the regions of the various significance levels where the dotted area is the region for the 0.10 significance level. It is seen that if a more liberal choice of type I error is made, the zone extends to larger ß values. Therefore, in interpreting the results of the previous graphs (Figs. 9-12), these remarks should be kept in mind. Up to this point, it was assumed that the true values of y (the reduced mean due to seeding) and y (the historical mean) were the "true values" of the parameter y of the poisson distribution. What happens at other values of y, if the estimates of y and y are not the "true" values 9 The function L (y), or the operating characteristic (the probability of accepting H o when µ is the true value) gives insight into this aspect. The OC (operating characteristic) curves for the summer data in area 5 are shown in Fig. 14. The OC curves show that for a given hypothesized decrease, large effects are easier to detect than small effects. For the 40-% curve, and for a mean of 1.0 hail days, the ability to detect is very good, power = 0.94, whereas for a mean of 2.0 hail days it is very low, power = Hence, if seeding could reduce the true mean to 1.0 it would very likely be detected. The relationship between the true mean and the ASN is shown in Fig. 15. The ASN represents the number of years required to reach a decision. The maximum point of each curve represents the center of the interval of indifference, which is the area where the decision would be to continue the testing procedure. This illustrates that if the true mean for the 20-% hypothesized decrease is close to the center (1.45 hail days), it would be detected only after a relatively long time (20.6 years). On the other hand, if the true mean is reduced by hail suppression to 1.0 (1 hail day), it would be detected in less than 7 years.

37 -28- Figure 13. Comparison of the nurrber of years to obtain significance for the null and alternative hypotheses for the poisson sequential test using Weather Bureau hail days in area 5.

38 Figure 14. Poisson OC function for Weather Bureau hail-day data in area 5.

39 Figure 15. Poisson ASN function for Weather Bureau hail-day data in area 5.

40 -31- CROP-HAIL INSURANCE DATA In this section, the number of dollars paid to policy holders for loss of crop yields due to hail, and the number of insured acres damaged by hail were investigated as a source of data for the verification of hail modification experiments. These data were considered 1) because of the excellent liability coverage in Illinois, and 2) because the eventual success of a hail modification experiment should be judged by its economic benefit. Theoretical frequency distributions were fitted to the data, and sample size was computed for five designs using daily and yearly insurance data. These designs included 1) randomization of days over a single target area into seeded and non-seeded days with the non-seeded days being the control, 2) random choice of days to be seeded over a single target area with the historical record being the control, 3) continuous seeding (on all potential hail days) with the historical record being the control, 4) seeding in a target area chosen at random with another area being designated as the control, and 5) continuous seeding (all potential hail days) in a target area with a nearby area being the control. In the statistical analysis, both the non-classical (sequential) and the classical (non-sequential) analyses were employed. The components of the particular test being used were computed for the non-seeded distributions; then, with assumed changes in the distribution parameters, the sample size was computed through algebraic relations. These values were computed for the gamma and for the normal 1- and 2-sample tests, and for the poisson and negative binomial 1-sample tests. For the yearly insurance data, the poisson and negative binomial tests were not used, and the year was substituted in place of the day as the experimental unit. Monte Carlo techniques were then employed with the daily insurance data to obtain a limited number of seeded and non-seeded distributions from which computations of sample size were made. These techniques were employed to obtain a somewhat more realistic decrease in the parameters of the non-seeded distributions, and to obtain duration estimates for a non-parametric test.

41 Data and Analytical Procedures Crop-hail insurance data is a very meaningful expression of the effect of hail suppression, if it is available for a large portion of an area extensively covered by crops. Importantly, these insurance records show the amount of loss in dollars and the number of acres damaged on a daily and regional basis. Thus, a reduction in hail is directly related to the economy of the region. Unfortunately, insurance data on property damage from hail are not readily available, but the hail damage to crops exceeds that to property in Illinois by a factor of 12 to 1 (Changnon, 1960). For the present investigation, detailed records on individual paid claims for all losses in Illinois during the period were obtained from the Crop-Hail Insurance Actuarial Association. These data and those relating to liability (amount of area insured) were available on a county basis. In accordance with the sizes of areas chosen in the hail-day analysis, which were selected to approximate the size of potential seeding areas, a pair of small areas (each about 1500 mi 2 ) and a pair of large areas (each about 4000 mi 2 ) were chosen for a study of various experimental designs that could be envisioned for this data. The choice of nearby pairs of areas allows an evaluation of target-control and crossover experimental designs, as well as the evaluation of single areas with randomization by days or non-randomization (continuous seed). The study areas were delineated on a county boundary basis because of the basic data format, and paired areas of extensive liability and approximately similar size were chosen from the data. The four areas chosen for investigations of the insurance data are shown in Fig. 16. Areas 1 and 2 were 1531 and mi2 } respectively, and areas 3 and 4 were 3800 and 3826 mi 2, respectively. The average areal coverage of liability (number of square miles with insurance) during the 19-yr period was 80% in area 1, 80% in area 2, 75% in area 3, and 74% in area 4. The daily values of dollar and acreage loss for the two smaller areas appear in Table B of the appendix, and those for the two larger areas appear in Table C of the appendix. Although hail insurance data are a realistic measure for evaluation of hail suppression activities, direct comparison of the loss in one month with that in another, or comparison of the data in one year with that in another, cannot be accomplished without certain adjustments to the data. The problems

42 -33- Figure 16. Study areas of crop-haul insurance data.

43 -34- of change during a crop season and between years include these facts: 1) a given crop's susceptibility to damage fluctuates considerably during the crop season, 2) the amount of liability changes between years, and 3) the value of the dollar changes between years. To make valid areal comparisons of the values and results of different study areas, another adjustment was required to allow for the fact the areas were not of the exact same size. Decker (1952) used an adjusted dollar and areal index for fitting hail-damage frequency distributions, and for evaluating the probabilities of hail damage for various crop-reporting districts in the state of Iowa. The adjustment indices used in this study are similar to those used by Decker. The individual daily loss values in this research were multiplied by the following adjustment index. (3) where: ADL - adjustment value for dollar loss values for a given month and year SSI = seasonal susceptibility index LI = liability index AI = area index PI = price index The seasonal susceptibility (to damage) index was 10 dollars divided by the median monthly loss cost. The 10 value was used to eliminate fractions that existed in the median loss costs. Loss cost is a number derived from insurance data that represents the total storm-day losses divided by liability for the area with loss, and multiplied by $100 (Changnon and Stout, 1967). Eight years of Illinois insurance records were used in an earlier study to determine the median monthly values, considered to be a numerical expression of the crop's susceptibility to damage (Changnon, 1967b). The median loss cost for corn and soybeans in July is $ which is nearly ten times those in May and October ($0.0001). Hence, the SSI for May and October became 1.0 (10 divided by ), and that for July became The other indices were 0.2 for June, 0.29 for August, and 0.5 for September (see appendix, Table D).

44 -35- The liability index was the total areal liability in dollars in a given year divided by the annual price index and 10 dollars. Insurance records on the actual areal extent of liability in each year were not available for all years in the period, and the only available annual statistic on liability was the monetary amount of liability which is closely related to the areal extent of coverage. The liability varied considerably during the 19-year period, expanding in area 1 from a low of $1.6 million in 1948 to a high of $27.2 million in The change in one of the larger areas (number 4) was from $6.5 million in 1948 to $60.2 million in Since the liability measure was in dollars, the problem of fluctuation due to dollar value changes was handled by dividing the liability values by a price index. This value was then divided by 10 6 dollars to make the LI a non-dimensional number of low magnitude. For instance, the LI for area 1 was 0.53 in 1948 and in 1966; in area 4 the 1948 LI was 2.08 and the 1966 value was (see appendix, Table E). The area index was the total number of acres in a specific area divided by 10 6 acres. The 10 6 value was used to make the area index a non-dimensional number of low magnitude. The area index was used to normalize for the difference in size between areas, and resulting AI values were 0.98 for area 1, 1.02 for area 2, 2.43 for area 3, and 2.45 for area 4. The price index was the price Illinois farmers received for all farm products in any given year adjusted to a base of 100 (Illinois Cooperative Crop-Reporting Service, 1958). This price index integrates the changing dollar value on an agricultural basis and also helps to account for temporal changes in crop types and their quality. These indices ranged from a low of 2.16 in 1964 to a high of 3.10 in This scheme of developing ADL values produced smaller ADL for the two larger areas 3 and 4, and the adjusted dollar values in these areas were lower than those for the smaller areas even though the original unadjusted dollar values in the large areas for the same dates were larger than those in the smaller areas. For instance, the unadjusted 11 July 1966 dollar loss in area 4 was $329,787 compared with $268,635 in area 1, but the adjusted loss value for area 4 for this day was $248 compared with $1109 for area 1.

45 -36- The adjustment factor used for each of the daily values of acreage loss was determined by the formula: CO where AAL - the adjustment value for acres of loss in a given year SSI = seasonal susceptibility index LI = liability index AI = acre index These indices were the same as those described for the dollar loss adjustments (ADL). The PI was not employed in the acreage adjustment formula since the dollar change did not directly affect the temporal variability in acreage. The daily dollar and acreage values which were adjusted by the ADL and AAL indices were those used in the statistical analyses of the various experimental designs. The dollar and acreage adjustment indices for each year appear in Table E of the appendix. Unless stated otherwise, all loss values presented in this paper are the adjusted values (Tables B and C of the appendix). It should be realized that these adjustments cannot account for all the factors involved, but the indices were developed from the only county-yearly data available for adjusting insurance data. Inherent in the insurance data are other factors, such as changing farm practices and crop types which are not measured on a county basis and cannot be adjusted for. Hopefully, the PI partially accounts for some of these factors. Also, inherent in the data is a ±5-% variation due to the subjectivity'in the field measurements of loss. 2. Natural Variability The insurance loss data for dollars and acres exhibit a great amount of variability between days, months, and years, as well as between areas. Certain of the basic data from the four study areas were selected for presentation in this section to reveal their time and space variability. Days with loss. The average and maximum monthly and annual numbers of days of loss for each area are shown in Table 4. Minimum values were zero for

46 -37- all months. Comparisons of area 1 with 2, and area 3 with 4 reveal that the two southernmost areas (2 and 3) averaged a few more loss days per year than did 1 and 4, respectively, but the maximum annual values were comparable. The greater frequencies in areas 2 and 3 relate to more losses, on the average, in May and June (Table 4) as the hail season advances northward across Illinois (Changnon, 1963). Table 4. Average and extreme number of days of loss per month. May June July Area 1 Area 2 Area 3 Area 4 Average Maximum Average Maximum Average Maximum August Average Maximum September Average Maximum October Average Maximum Annual Average Maximum Minimum There were 315 hail loss days in area 1 during , and on 149 days, or 47%, loss occurred in comparable area 2. There were 399 hail loss days in area 2, and on 250 days the hail loss occurred only in area 2. Area 3 had 612 hail loss days in the 19-yr period, and on 288 of these days,

47 -38- or 47%, losses occurred in comparable area 4. Area 4 had 530 hail loss days, and on 242 days hail loss did not occur in area 3. Although these paired areas were geographically close, approximately only half of the days with losses in one area were loss days in the other area. Dollars and acres of loss. The average and extreme annual values of dollar loss and acre loss for all four areas are listed in Table 5. The average and extreme annual values of area 2 are slightly larger than those of area 1, and two of those of area 3 are slightly larger than those of area 4. Thus, both of the southernmost areas had slightly larger losses than did their comparable areas to the north. Inspection of the average daily values of loss (Table 5) reveals very comparable values between the values of the two small areas and between those of the two larger areas (3 and 4). Thus, the areal differences shown in the annual values are related to the fact that areas 2 and 3 averaged a few more hail days per year. When a day of loss occurred in an area, it produced, on the average, a quantity of loss (dollars or acres) approximately equal to that in the same sized area. The temporal variability in the annual loss values is displayed in Figs. 17 and 18. The dollar-value curves for the two smaller areas (Fig. 17) reveal certain large shifts. For instance, area 1 had a $10 loss in 1949 and a $1031 loss in 1950, a 100-fold increase, and area 2 went from a $165 loss in 1961 to a $4110 loss in The year-to-year fluctuations in acres of loss for the two larger areas (Fig. 18) also display considerable variability. During the 19-yr sampling period there were 18 changes between acres of loss per year, and comparison of these for the two areas revealed that eleven times the changes in the values were harmonious (increased or decreased together), but seven times there was disagreement. These disagreements in natural data between adjacent areas reflect on the validity of the recent Russian hail suppression experiments (Sulakvelidze, 1966). Their major proof of success was based on comparison of insurance loss data in a seeded area with that in an adjacent control area. The temporal trends of the values of the two areas for 3 or 4 years prior to the seeding year were alike, and then the seeded area curve decreased and the control area increased during the seeded year. Several such situations can be found in the dollar loss and acre loss curves of Figs. 17 and 18 which were based on natural data.

48 Table 5. Annual and daily loss values for study areas. Dollars Acres Area 1 Area 2 Area 3 Area 4 Area 1 Area 2 Area 3 Area 4 Annual average Annual maximum Annual minimum Average per loss day Maximum per loss day Minimum per loss day

49 -40- Figuve 17. Annual amount of dollar loss in areas 1 and 2.

50 -41- Figure 18. Annual amount of acre loss in areas 3 and 4.

51 -42- The resulting distribution of the daily loss values is very skewed. The few extremely large values in Table 5 indicate the wide range in the values with the dollar values ranging from $0.06 to a high $ within one area. A great number of the daily losses consisted of very low values with 51% of the daily dollar values being $10 or less whereas only 12% exceeded the average of $116 per loss day. The annual values of dollar loss and acre loss in each of the four areas were correlated with each area's frequency of hail loss days. The resulting correlation coefficients (v) and the probability of obtaining a larger correlation from random sampling [P(r 2 2 >r 0 )] are shown in Table 6. The correlations for dollar loss vs hail days are slightly larger than those for hail days vs acre loss, and all except area 3 are significant at the 0.05 level. However, the highest dollar-day coefficient, for area 1, indicates that only 42% of the variation in annual dollar losses are explained by the number of days with loss per year. Table 6. Correlation coefficients between annual number of days of loss, annual dollar losses, and annual acre losses. Hail days vs dollar loss < < <0.01 Hail days vs acre loss < Dollar loss vs acre loss < < < <0.01 The correlations between the annual dollar losses and annual acre losses (Table 6) are all relatively high. This good relationship between area of hail (acres) and amount of loss (dollars) does not agree with the findings on area-energy relations for the individual hailstreaks (see paired storm section, page 103) which showed no correlation. This disagreement may relate to the fact that the annual totals tend to smooth out many of the area-energy differences

52 -43- that may exist on a given day or hour in an area, or that energy is not an adequate measure of monetary loss to crops by hailstorms. To further check this relationship, the 315 daily values of dollar loss in area 1 were correlated with their corresponding daily acre loss values. The coefficient was +0.84, also indicating a good relationship between daily amount of loss and areal extent of damaging hail. These varying results for area, energy, and amount of loss indicate that use of areal extent as an indirect measure of energy dissipated from hail fall, or vice versa, in hail suppression projects is likely not valid if the experimental units are individual (paired) storms. However, areas of hailstreaks may be a measure of monetary loss, and certainly areal extent of loss would provide a reasonably good estimate of the amount of damage within areas of 1500 to 4000 mi 2 if the units were daily or annual values of damaging hail. The annual loss values (days, dollars, and acres) of area 1 were correlated with those of area 2, the other 1500-mi 2 area (Table 7). The coefficients indicated that the area 1 acre-loss values explained only 35% of the variation in acre loss of area 2, whereas the annual number of days of loss in area 1 explained 50% of the variation in those of area 2. Correlations of area 3 values with those of 4 in Table 7 show lower coefficients for dollars and acres than found for areas 1 and 2. Table 7. Correlation coefficients of annual loss values between areas. Days of hail Dollar loss Acre loss r P(r 2 >r 0 2 ) r P(r 2 >v Q 2 ) r P(r 2 >r o 2 ) Area 1 vs Area < < <0.01 Area 3 vs Area < Theoretical Frequency Distributions Yearly data. The temporal variability of the yearly monetary and acre damage data for all four areas was illustrated in Figs. 17 and 18. The annual insurance data were then tested for randomness in the climatological data series using the procedure of Swed and Eisenhart (1943), and the results of the

53 -44- test are shown in Table 8. The probabilities of obtaining U's smaller or larger than expected from random sampling are all greater than 0.10 for all areas and for both trend and oscillation patterns in the data. Therefore, at the often cited 0.01 and 0.05 levels of significance, the null hypothesis of homogeneity of the climatological data series would be accepted and the alternative hypotheses of trend and oscillations would not be accepted. Therefore, the data were treated as homogeneous data series on the basis of this test, and the justification for proceeding with further statistical analysis of the yearly insurance data was established. Table 8. Test for randomness of the climatological data series for yearly insurance data. U Observed N a number of runs above Number of Area Type of measurement and below median observations above median P(U U e ) P(U > U e ) 1 Dollars loss Acres damaged Dollars loss Acres damaged Dollars loss Acres damaged Dollars loss Acres damaged If P(U U e ) is low, trend is present. If P(U > U e ) is low, oscillations are present. The gamma and the log-normal distributions were then fitted to the data shown in Fig. 19 and 20. The density function for the gamma distribution is:

54 -45- Figure 19. Histograms and theoretical frequency curves for yearly hail insurance data (dollars are loss).

55 -46- Figure 20. Histograms and theoretical frequency curves for yearly hail insurance data (acres damaged).

56 -47- The symbols 3 and γ are location and shape factors, respectively, and are estimated by the method of maximum likelihood (Thorn, 1958). The density function for the log-normal distribution is where: y - In x µ y = mean of the In x a y = standard deviation of the In x The sample estimates of the log-normal and gamma distributions for yearly insurance data are tabulated in Table 9. Table 9. Sample estimates of the gamma and log-normal distributions for yearly insurance data. o l r B N Log-normal Gamma Gamma Size of Log-normal standard shape location Area sample mean deviation factor factor Dollars Area Area Area Area Acres Area Area Area Area Comparison of frequency curves in Fig. 19 indicates that the gamma distribution provides a better fit than the log-normal for the yearly insurance data. To test this premise, the Kolmogrov-Smirnov "goodness of fit" test was applied to the data. The results of this test are shown in Table 10. Comparison

57 -48- of the D n values indicates that except for the damaged acres in area 3 the maximum differences between the observed and theoretical frequency distributions are smaller for the gamma distribution than for the log-normal distribution. Thus, the gamma distribution is the better fit. Table 10. Kolmogrov-Smirnov "goodness of fit" test for yearly insurance data. D n = F n (x)-f(x) P(D D n ) Area Type of data Gamma Log-normal Gamma Log-normal 1 Dollars loss >.200 >.200 Acres damaged > Dollars loss Acres damaged >.200 > Dollars loss >.200 >.200 Acres damaged >.200 > Dollars loss >.200 >.200 Acres damaged >.200 >.200 F (x) n F(x) = Observed distribution. = Theoretical distribution. If P(D D n ) is low, reject the hypothesis that the observed data fits the theoretical distribution. Since both parameters of each distribution are estimated from experimental data, the tables of Liffiefors (1967) were used again. Since tables of the sampling distribution of D n include only the 0.20 probability level, the probabilities of obtaining a smaller D n from random sampling for most of the areas cannot be compared. However, certain conclusions can be drawn from the probabilities. First, good fits were obtained for all the gamma distributions with the exception of dollars for area 2, and for all log-normal distributions with the exceptions of acres (area 1) and dollars (area 2). Secondly, with the exception of acres for area 1, even these probabilities are close to the 0.05 level of significance.

58 -49- Daily data. In the initial analysis of the daily data (Figs. 21 and 22), the days with crop damage were separated from the many days without crop (hail) damage. A mixed distribution function was then estimated based on two assumptions. First, there is a non-zero probability of hail on a particular day. Secondly, when damage does occur, the amount of damage is distributed as a log-normal or gamma variable. The general form of the mixed distribution function [G(x)] can be written as. G(x) = P(X < a) = P(X = 0) + P(X > 0) P(X < a X > 0) (7) where: P(X < a) = probability of receiving less than a specified amount of hail damage P(X = o) = probability of receiving no hail damage P(X > 0) = probability of receiving some hail damage P(X < a X > 0) = probability of receiving less than a specified amount of hail damage, given that hail damage has occurred The term P(X < a X > 0) is given by: (8) The density function, f(x), can be specified as any distribution. For this study, mixed distributions with the log-normal density function from Eq. 6 were used. The sample estimates of the log-normal parameter are shown in Table 11. The Chi-square "goodness of fit" test was then applied to the non-zero portions of the mixed distribution functions for the assumptions of gamma and log-normal distributions. The Chi-square test was based on the method described by Hahn and Shapiro (1967) with one modification. The number of class intervals were chosen using the relation 5 log 10 N, where N is the number in the sample. This method insures that the choice of class interval boundaries will depend on the theoretical values and not on the sample values. It also

59 -50- Figure 21. Histograms and theoretical frequency curves for daily monetary insurance data.

60 -51- Figure 22. Histograms and frequency curves for daily acreage damaged insurance data.

61 -52- insures that, except for modification of class interval limits due to rounding and measurement errors, equal numbers of expected values will result in each interval. The above rule also insures that there will be at least five expected values in each interval as long as the sample is 40. This procedure of Chi-square makes comparisons between different distributional fits more objective. Table 11. Sample estimates of the log-normal parameters for daily insurance data. Log-normal mean Log-normal standard deviation Area Dollars Acres Dollars Acres Area Area Area Area The results of the Chi-square test are shown in Table 12. Again, if the probability is small of obtaining a Chi-square value greater than the observed value from random sampling, the hypothesis that the sample data fit the specified distribution is rejected. From the table it is seen that none of the daily insurance data can be fitted by the gamma distribution. The data also are poorly fitted by the log-normal distribution for the area 3 monetary data and for acreage data in areas 3 and 4. The dollar data for area 2 is close to the 0.05 significance level, and the log-normal distribution was used for subsequent statistical tests. With the exception of areal comparisons and Monte Carlo trials, the computation of sample size was based on data from area 1. Although the log-normal distribution did not fit either data set from area 3 nor the acreage data from area 4, log-normal distributions with parameters estimated from these data were used in the areal comparisons. For the Monte Carlo trials, in which the parametric test is compared to the non-parametric tests, the log-normal distribution also was used, and the loss of power due to using an inappropriate distribution was readily demonstrated. Subsequent work has indicated that the data in area 3 and 4 could be fitted by a truncated log-normal distribution.

62 -53- Figure 23. Frequency distributions of hail damage days for area 1.

63 -54- Figure 24. Frequency distributions of hail damage days for area 2.

64 -55- Figure 25. Frequency distributions of hail damage days for area 3.

65 -56- Figure 26. Frequency distributions of hail damage days for area 4.

66 -57- Table 12. Chi-square "goodness of fit" test for the daily insurance data. N Degrees Type of Number of of 2 x o P(x 2 > x 2 o ) Area data loss days freedom Log-normal Gamma Log-normal Gamma 1 Dollars <0.010 Acres < Dollars <0.010 Acres < Dollars <0.010 <0.010 Acres <0.010 < Dollars <0.010 Acres <0.010 < X = value of Chi-square observed from sample. 2 2 If P(x > X ) is low, reject the hypothesis that the sample data fits the theoretical distribution. The number of days on which hail damage occurred in the four areas is shown in Figs The results of the Kolmogrov-Smirnov "goodness of fit" test for all damage day data, and for various classes of daily loss, are shown in Table 13 for each area. Ten of the cases were inadequately described by the poisson distribution, that is, probability values are less than For these ten cases the test of sufficiency indicated that the moment estimates of the negative binomial were not sufficient. However, for the daily-loss categories of $150 loss and $200 loss, all distributions could be fitted by the poisson distribution or by the moment estimates of the negative binomial distribution, even though the moment estimates were insufficient for the $150 loss category in areas 2 and 3. It is obvious from Table 13 that as more of the low daily loss values are excluded from the distribution, the better the fit becomes in all areas. Since the low loss days did not fit the distributions well, only the data for the hail days producing $150 loss or more and that for hail days of $200 loss or more were selected for further analysis.

67 Table 13. Kolmogrov-Smirnov "goodness of fit" test for the insurance damage days. Average Negative Goodness of fit no, of binomial hail Poisson test test of Type of data Area days K P of adequacy sufficiency Poisson N.B. Poisson N.B. Total damage days < <0.01 Days > $50 loss Days > $100 loss >0.20 Days > $150 loss >0.20 Days > $200 loss >0.20 Total damage days < Days > $50 loss <0.01 Days > $100 loss <0.01 Days > $150 loss >0.20 Days > $200 loss >0.20 Total damage days < Days > $50 loss < <0.01 Days > $100 loss < >0.20 Days > $150 loss >0.20 Days > $200 loss >0.20 Total damage days < Days > $50 loss Days > $100 loss <0.01 Days > $150 loss ~ >0.20 Days > $200 loss > If P(x > X 18 2 ) is low, reject the hypothesis that x = S, i.e., the poisson distribution is inadequate. If C > 20, the moment estimates of the negative binomial are sufficient. If P(D > D n ) is low, reject the hypothesis that the data fit the specified distribution. c

68 Experimental Design and Tests of Hypothesis Yearly data. Initially, the yearly insurance data were considered as the experimental unit in the various designs. An experimental unit using yearly data does not appear as practical as the daily experimental unit since the yearly sample size is much smaller, and much of the areal-temporal variability is masked by other factors. Nevertheless, sample size was computed for the various designs and tests using yearly data to check its potential applicability. The normal sample test was used with all of the experimental designs. Under the assumption that the yearly data were log-normal distributed, the following formula was used to obtain the number of observations required for the 2-sample non-sequential test (Davies, 1954): (9) where: µ a = the normal deviate for a probability level µ ß = the normal deviate for ß probability level D = difference in means it is desired to detect S 2 = the variance Various reductions of 0.20, 0.40, 0.60, and 0.80 were assumed and applied to the non-transformed hail data. The corresponding scale change was effected on the transformed scale by the addition of the logarithm of (1-6). The variances were assumed to be equal, since the variance of the log-normal distribution is unaffected by scale changes in the variate. Eq. 9 was then applied with S 2 equal to the variance of the logarithms and D equal to the logarithms of (1-6). For the random 1-sample test, Eq. 9 is used directly to compute sample size. For the 1-sample continuous design, Eq. 9 is divided by 2.0. With the daily single area random design, both samples must be obtained from the same area. Therefore, Eq. 9 must be multiplied by 2.0. Gabriel (1967) states that under the simplifying assumptions of normal precipitation and equal variances in two areas, a crossover design needs only

69 -60- (1-R)/2 of the number of observations needed by a single area design and that the crossover design requires 1/[2(1 + R)] of the number of observations needed by a target-control design. The relationship between the crossover design and the single area design can be expressed as: (10) where: S = number of observations required for a single area design R = the correlation coefficient between the two areas C = the number of observations required for the crossover design The combination of the relations of the above paragraph yields: (11) where: R = correlation coefficient between the two areas S = the number of observations required for the single area design TC = the number of observations required for the target-control design Eqs. 10 and 11 were applied to the yearly insurance data to obtain the number of years required for the target-control and crossover designs. Since the gamma distribution fits these data, results also were obtained using the 2-sample gamma test. This test assumes that both samples must have common shape factors. The test statistic as given by Schickedanz (1967) is:

70 -61- where: = the number of observations in the sample from the first distribution = the number of observations in the sample from the second distribution = the observations in the sample from the first distribution = the observation in the sample from the second distribution = the complete gamma function for respectively

71 -62- Since the seeded sample was not available, the effect of seeding was expressed as percentage changes in the mean. Therefore, the substitutions were made in Eq. 12. sample were not available, the products Also, since the individual observations of the seeded could not be estimated. Therefore, it was assumed that is equal to and under these conditions Eq. 12 reduces to

72 -63- Taking natural logarithms of both sides of Eq. 13 yields: If one assumes n 1 to be a constant equal to c, then N = c + n 2, and solving for n 2 provides an equation for the sample size of a non-sequential test for the yearly data. Sample one becomes the historical record and sample two becomes the seeded sample. Thus, (14) The approximate power of the maximum likelihood ratio test against a specific alternative is given by Fix (1954) as (15) where is the value of the non-central Chi-square (x 2 ') corresponding to the a level of significance. The power obviously depends on, the non-centrality parameter. The degrees of freedom f are the same as those associated with the likelihood ratio test. Wilks (1938) showed that for large samples, -2 In is asymptotically that of Chi-square with 1 degree of

73 -64- freedom. Therefore, A is estimated, noting that for large samples it is equivalent to the likelihood ratio function evaluated for the values of the parameters specified by H (Lehmann, 1959). a Fix (1954) has computed tables of the non-central Chi-square for the 0.05 and 0.01 size of the test. In these tables is the tabular value corresponding to values of P and f. In order to obtain the approximate power of the likelihood ratio test, it is sufficient to enter -2 In in the tables, and then the values of P for a given degree of freedom can be obtained by interpolation. In order to solve Eq. 14 the tabular value of -2 In was obtained from Fix's tables, and thus the value of In was obtained for a specified significance level and power. The number of years required to obtain significance for the likelihood ratio test with the random daily design was obtained by setting n = n 1 = n 2, N = 2n and solving Eq. 14 for n. The above procedure was used to compute the sample size for the random yearly design and yearly continuous design. The results for a sequential test with the log-normal distribution can be obtained by using the sequential test method of Wald (1947) for the testing of the reduction of a mean from a normal distribution. All that is required is a slight modification to allow for the logarithms. To apply this test, one must assume that the historical record is indeed the population for each area. The ASN equations were applied to the data from all insurance areas, and the sample size was computed for the random and continuous designs using the 1-sample test. The gamma distribution was also used in a sequential test. The ASN equations for the sequential test were derived for the gamma distribution using the methods of Wald (1947) and Thorn (1957b). The following ASN equation was derived. (16) where: ß = the location parameter of the gamma distribution ß1 = the historical location parameter ß 0 = the value of the location parameter to which ß is reduced

74 -65- The hypothesis H a = ß ß1 is that seeding produced no worthwhile effect and The hypothesis H ß ß 0 is that seeding reduced the historical location 0 = parameter from ß, to 1 ß0. Eq. 16 was then applied to the yearly dollar loss data and to the yearly acreage data to obtain the average number of years to obtain significance. Daily data. The procedures used in evaluating the yearly data were also applied to the daily data. In addition, the poisson and negative binomial 1-sample tests were applied to distributions of the number of damage days for the random and continuous designs. The methods used were the same as those used for Weather Bureau hail days previously described. Also, Monte Carlo techniques were used to obtain a limited number of seeded and non-seeded distributions from which computations of sample size were made. These methods were employed to obtain a somewhat more realistic decrease in the parameters of the non-seeded distributions, and to obtain estimates for a non-parametric test. For the daily data, the seeding effect was first simulated by superimposing a scale decrease on the areal parameters of the non-seeded distributions. The number of years required to obtain significance was then obtained through algebraic relations. The duration of a seeding experiment based on daily data was obtained by the same equations that were used to estimate sample size for the Weather Bureau hail-day data and the crop insurance yearly data.

75 -66- The seeding effect was then simulated by random generation of data. The selection of a random number, T j, is equivalent to selecting a cumulative probability value, F(x), at random. The inverse function of any probability function can be expressed as: (17) where: G[F(x)] = a function of cumulative probabilities. Therefore, a random variate X can be selected from the log-normal distribution by using the relationship above. Sample values of daily damage x were then generated from the log-normal distribution of area 3. As the hail damage values X were generated, they were designated seeded or non-seeded days by randomization in pairs. Every seeded value was decreased by in (1-6), where 6 is the desired percentage decrease on the non-transformed scale to detect. This experimental design is the single area design with randomization by days over the seeding area. As each pair of data were generated, the differences between the means of the seeded and non-seeded samples were tested for significance by using a "t" test. The anti-logarithms of the sample values were computed and the "t" test also was performed for the non-transformed values. The non-transformed values were also tested for significance using the Mann-Whitney U test. For each value of n, the number in the non-seeded sample, a tabulation was made of whether the test was significant. This was continued until a specified number of samples had been generated. The process was then repeated, so that a frequency distribution of the number of runs significant at a particular sample size was obtained. The percentage number of significant runs at each sample size is equivalent to the power of the test for that particular sample size, since the probability of accepting the alternative hypothesis is defined as the power of the test. The procedure was performed for the 0.10, 0.05, and the 0.01 significance levels. The number of damaging hail days per year was used with these data to obtain the number of years required to obtain significance.

76 -67- The data from area 3 and 4 were used in a generation scheme to obtain data for the crossover design. The first daily value in area 3 was considered. A number was selected at random from the interval (0, 1). If the number was P(H), the probability of hail for area 3, it was designated to be a hail day. If it was a hail day, the value x (A 3 ) was selected at random from the area 3 log-normal probability curve. A random choice of seed or non-seed was then made for area 3. The first day of area 4 was then designated to be the opposite of day 1 for area 3 in regard to seeding. The random choice was then made with regard to whether the day was a hail or non-hail day. If a hail day was designated, a value (A 4 ) was selected from the log-normal probability curve of area 4. Next, the second day of area 3 was determined to be a hail or non-hail day at random and the above prooedure was repeated. The randomization of the seed and non-seed values for the remainder of the values in each area was accomplished by randomization in pairs, so that the treatment effect (seeding) would appear an equal number of times in each area. As the values A 3 and A 4 were generated, the differences A 3 -A 4 were then computed, and the differences for the A 3 seeded days were separated from the A 4 seeded days. The "t" test was then applied to the A 3 -A 4 differences for a specified number of sample runs, and the number of times significance was obtained at each sample, size was tabulated. Next, the continuous seed non-sequential design was simulated using the random generation procedure. This was a test in which the seeded period was to be tested against the historical data sample. First, a historical record of 600 values was simulated by random generation from the log-normal distribution of area 3. The simulation of the seeded sample was accomplished by generation of the data from the area 3 log-normal probability curve, and a tabulation was made of the number of times that the Mann-Whitney U test was significant at a particular sample size. The next design considered was the target-control design. The correlation coefficients were computed between the possible target and control areas, and all were found to be 0.22 or less. This greatly reduces the advantage of the control area in the target-control design. Hence, no further computations were made on the target-control design.

77 Results for the Yearly Data The comparison of the number of years required to obtain significance for the various tests and experimental designs using yearly values of dollars at different levels of decrease is shown in Figs. 27a-f. The gamma test is seen to be better than any of the other tests. This was an expected result because the gamma distribution was the best fitting distribution and hence permits more information to be extracted from the data (Schickedanz, 1967; Schickedanz and Decker, 1968). It is obvious also that the sequential test is superior to the non-sequential test, if one is willing to accept the assumptions involved in the sequential analysis. The 1-sample test yields a smaller sample size than does the 2-sample test. It should be noted that in the 1-sample test, and in the 2-sample (continuous) test, the historical record is assumed to be the population. However, in the 2-sample test (random) both samples are randomly drawn, and one sample is designated as the target (seeded) and the other as the control (non-seeded). The crossover design falls between the random and continuous designs involving the normal sequential 1-sample test. The target-control design, because of poor correlation between areas, and the non-sequential 2-sample random test, because of the complete randomization, require the largest sample sizes. It is obvious from the graphs in Fig. 27 that any experimental design using yearly units of dollars is a very poor choice. For a 20% decrease and ß level of 0.20, approximately 60 years are required to obtain significance for the best test and design. Even for a 80% decrease and a ß level of 0.20, the number of years required to verify the results vary from 2 to 111 years depending on the type of design employed. It should be noted that for the gamma non-sequential case, the value of 3 at 0.01 is not on the graphs. To obtain these values, tables of the non-central Chi-square are necessary, and the tables by Fix (1954) do not include values for ß = The number of years to obtain significance for the various tests and experiments as designs using yearly acre-loss data is shown in Figs. 28a-f. Although yearly acre data, in general, would provide an answer sooner than designs using dollar data, the yearly experimental unit remains a poor experimental unit to use. For a 20% decrease and ß = 0.20, approximately 40 years are required to obtain significance with the best test (Fig. 28c).

78 -69- Figure 27. Comparison of the number of years required to obtain significance for various tests and designs using yearly insurance data in area 1 (dollars).

79 -70- Figure 28. Comparison of the number of years required to obtain significance for various tests and designs using yearly insurance data in area 1 (acres).

80 -71- Comparison of the area size and location effects on the results is shown in Figs. 29a-f. Comparison of the dollar loss results for the two smaller areas (Figs. 29a-b) for the gamma test shows that it is easier to detect significant differences in area 2 than in area 1. The same relation is true for the acre data. For the two larger areas, area 3 requires smaller sample size than area 4, although there is very little difference between the dollar curves for these two larger areas. Comparison of the dollar-loss curves for areas 4 and 1 (which is smaller and a part of area 4, see Fig. 16) shows that the larger area requires the smaller sample size. However, comparison of the acre-loss data for areas 3 and 2 (which is a part of area 3) shows that the smaller area requires a smaller sample size. This reversal in results probably occurs because the variability in acres is closely associated with areal extent. That is, tendency for smaller sample size with larger means is offset by the increased non-homogeneity factor introduced into the data by increasing the size of area. This conclusion seems to be justified by the fact that for the normal tests, Figs. 29c-e, the relation is the same for dollars and acres. However, the possibility also exists that it may be a result of the different tests employed. Some evidence of this is exemplified by the results of the "goodness of fit" test (Table 10). Although the area 3 acre data are well fitted by the gamma distribution, they fit the log-normal distribution better. Hence, the gamma distribution is not the best fitting distribution and this may be the cause of these results. For the crossover design, Fig. 29f, the smaller areas require the larger sample size. This is true in spite of the fact that the dollar and acre correlation coefficients between the smaller areas are better than those between the larger areas (Table 7). The comparison of the number of years required to obtain significance for the various tests and decreases is shown in Figs. 30a-f. Detection of a 5% change for a type II error of 0.20 and with an optimum test and design (Fig. 30a) requires a sample size of greater than 400 years. For the same type II error and test (Fig. 30a), a 20% change requires 59 years and a 60% decrease, requires 5 years. The least desirable test (in terms of sample size), the 2-sample random (Fig. 30e) requires over 40,000 years at a power level of 0.80 to detect a 5% decrease, and 50 years to detect the 80% decrease.

81 -72- Figure 29. Comparison of the nurrber of years required to obtain significance for yearly insurance data for different areas (20% decrease).

82 -73- Figure 30. Comparison of the number of years required to obtain significance for yearly insurance data for different decreases in area 1 (dollars).

83 -74- The comparison of the number of years required to obtain significance for different type I errors is shown in Figs. 31a-f. For the gamma sequential test, continuous design (using acres), it is seen that if one is very conservative (a level = 0.01), 47 years are required to obtain significance for a type II error of For a more liberal choice (a = 0.10), 35 years are required to obtain significance for a 20% decrease. For the normal 2-sample random design (acres) and for a type II error of 0.20, 6000 years are required; for a type I error of 0.10, 2700 years are required. A sequential test, as explained for the Weather Bureau hail-day data (page 24), always carries the danger that the test may not terminate or that it may terminate with a decision to reject the seeding effect rather than to accept it. Thus, in previous figures of this section, some of the numbers, which represent years required to accept the suppression effect, may be fictitious in that the test would have terminated sooner with the rejection decision. Figs. 32a-f show a comparison of the number of years required to obtain the 'accept' decision (N 0 ) with the number required for the 'reject' decision (N 1 ). The shaded regions in these figures represent the domain in which the sequential test would have terminated because of rejection prior to reaching the acceptance of the seeding effect. It is seen that this region never extends beyond a type II error of 0.15 and the 0.05 significance level for the gamma sequential test. For the normal sequential test, it never extends beyond 3 = 0.05 for the 0.05 significance level. Fig. 32c presents a comparison between the regions of the various significance levels, it shows that if a more liberal choice of type I error is made, the zone extends to larger ß values. Therefore, in interpreting the results of the previous figures, these remarks should be kept in mind. 6. Results for the Daily Data Algebraic computations. The number of years and days to obtain significance for the various tests and experimental designs using daily dollar data are shown in Figs. 33a-f. The bottom scale is the number of years and the top scale is the number of days required for a given decrease. The conversion factors between days and years are listed in Table F of the appendix. The normal sequential test is seen to provide reliable answers faster than any other test. Again,

84 -75- Figure 31. Comparison of the number of years required to obtain significance for yearly insurance data for different type I errors in area 1 (20% decrease).

85 -76- Figure 32. Comparison of N 0 and N 1 for the 1-sampte sequential tests ustng yearly acreage data in area 1 (continuous design).

86 -77- Figure 33. Comparison of the number of years required to obtain significance for various tests and designs using daily insurance data in area 1 (dollars).

87 -78- as in the testing of yearly units, the sequential test is superior to the non-sequential test if one is willing to accept the assumptions involved in the sequential analysis. The second best test is the poisson test based on damage days with greater than $150 loss. The crossover design yields smaller sample sizes than does the random non-sequential design or the target-control design. The comparison of years required to obtain significance for the various tests and experimental designs for daily acre-loss data is shown in Figs. 34a-f. For a 20% acreage decrease, and ß = 0.20, 13 years are required to obtain significance for the log-normal sequential continuous test, and 160 years are required for the normal non-sequential, 2-sample test. These values compare with 23 and 197 years, respectively, for the two former tests and designs using daily dollar-loss data. Regional differences in the number of years required to obtain significance from the daily data are shown by the curves for various areas (Figs. 35a-e). Comparison of the acre data from the two smaller areas (areas 1 and 2) using the normal sequential test (Fig. 35a) shows that it is easier to detect significant differences in area 2 than in area 1. Experimentation in area 4, a larger area containing area 1, requires less time to show significance than large area 3 which contains area 2. A geographical reversal exists in that the small southern area 2 requires less time than the small northern area 1, whereas the larger southern area 3 requires more time than the larger northern area 4. This can be attributed directly to the fact that the average damage days (May-October period) for areas 1, 2, 3, and 4 are , 21.00, 27.89, and 32.21, respectively. Therefore, the size of area, which is reflected in the number of damage days per area, has a very real influence on the size of sample required to obtain significance with these daily data. The comparison of the number of years to obtain significance for the various decreases is shown in Figs. 36a-e. To detect a 5% change, for a type II error of 0.20 and for the optimum test and design, requires a sample size of 238 years. A 20% decrease and the same error requires 12.6 years, and the 80% decrease requires less than a year. The 2-sample random test, the least desirable in terms of sample size (Fig. 36c), requires over 300 years at a power level of 0.80 for a 5% decrease, and 2.2 years for an 80% decrease.

88 -79- Figure 34. Comparison of the nurrber of years required to obtain significance for various tests and designs using daily insurance data in area 1 (acres).

89 -80- Figure 35. Comparison of the number of years required to obtain significance for the different areas using daily insurance data (20% decrease - acres).

90 -81- Figure 36. Comparison of the number of years required to obtain significance for different decreases using daily insurance data in area 1 (acres).

91 -82- The comparison of the number of years required to obtain significance for different type I errors at a 20% decrease is shown in Figs. 37a-f. The normal sequential test, continuous design (acres), for a very conservative level (a = 0.01), requires 14.5 years to obtain significance for a type II error of For a more liberal choice (a = 0.10), 12.5 years are required to obtain significance for a 20% decrease. For the normal 2-sample random design using acres and for a type II error of 0.20 (Fig. 37c), 189 years are required. For a type I error of 0.10, 123 years are required. A comparison of the number of years required to obtain significance for the normal sequential test is depicted in Figs. 38a-d. Again, the domain in which the sequential test would have terminated prior to the acceptance of the seeding effect is represented by the shaded areas. This area never extends beyond ß = 0.08 for the 0.05 significance level. Fig. 38c is a comparison of the regions for the various significance levels, and shows that, for the more liberal choice of type I error (a = 0.10), the zone extends to ß = 0.13 for the 20% decrease. Monte Carlo trials. The comparison of the number of years required to obtain significance for the various tests and experimental designs for generated data using dollars and acres is shown in Figs. 39a-c and 10a-c. As with sample computations presented earlier, the continuous design produced the least number of years to obtain significance, followed by the crossover and the random designs. The transformed data required less time to obtain significance than the non-transformed data, so that for this case the transformed "t" test had more power than the non-trans formed "t" test. The non-parametric "U" test required less duration than parametric "t" test. This results from the fact that the log-normal distribution did not adequately fit the data from area 3. This illustrates the loss in power when using a test based on an inappropriate distribution. As seen here, the non-parametric test is more powerful than the parametric test for this set of data. For a sample size of 10 years, the difference in power for the parametric and non-parametric random test is Curves on Figs. 41a-f allow comparisons between dollar-loss and acre-loss data. For the 0.05 level and a 40% decrease, the acres-damaged data is the best in all designs except for the crossover design where, for type II errors of 0.26 or less, the dollar data is better. This could be attributed to the fact that some undesirable combination can occur with random choice of two areas which have a high degree of variability inherent in the data.

92 Figure 37. Comparison of the number of years required to obtain significance for different type I errors using daily insurance data in area 1 (20% decrease).

93 -84- Figure 38. Comparison of the number of years required to obtain significance for the null and alternative hypotheses of the normal sequential test using daily insurance data in area 1 (continuous design - acres).

94 Figure 39. Comparison of the nunber of years reauired to obtain significance for various tests and designs for Monte Carlo trials of daily insurance data in area 3 (dollars - a =.05).

95 Figure 40. Comparison of the number of years required to obtain signifioanoe for various tests and designs for Monte Carlo trials of daily insurance data in area 3 (acres - a =.05).

96 -87- Figure 41. Comparison of the number of years required to obtain significance for Monte Carlo trials of monetary and acreage daily insurance data in area 3 (a =.05-40% decrease).

97 -88- Figs. 42a-f allow comparisons of the number of years to obtain significance for various decreases involving different tests and designs. Here again, the size of decrease that the experimenter is able to obtain becomes an important factor, as it does in all tests and designs. Figs. 43a-f show the effect of various type I errors on the ability to detect significant effects. Some overlapping of the 0.10 and 0.05 curves is present for low ß values. This occurs because the number of Monte Carlo trials performed was too small to adequately differentiate in this region of values. The curves in Figs were fitted to the generated data by a least squares simple regression where the power is y and the sample size is x. Five different equations were fitted to the data, and the curve with the highest regression was chosen to depict the generated results for a given set of data. The correlation coefficient, r, the intercept, a, the slope parameter, b, and the best fitting equation for the 0.05 significance level are listed in Table 14. The corresponding data for the 0.10 and 0.01 significance levels and for a 40% decrease are listed in Table 15. Sample size can be converted to years by the conversion factors listed in Table F of the appendix. 7. Summary of Results Using Yearly and Daily Insurance Data A rank comparison of area size and types of data to detect significant reductions in hail measurement for the gamma test (20% decrease) is shown in Table 16. For the yearly unit of data, acres of loss in all 4 areas require shorter sampling intervals to obtain significance than dollars of loss in any area. For the daily units of data, other factors become important and the trend for acreage preference is not as marked as with the yearly data. Also, for the yearly and daily data, the number of damage days does not appear to have a pronounced effect on the detection ability (rank). The areal size has little or no effect for the yearly data, but seems to be a factor with the daily data. A very interesting feature is that for yearly acres or dollars data the two southern areas (2 and 3) require less time to obtain significance than do the northern areas (1 and 4). For the daily data, the geographical location is no longer as important as with the yearly data, but the number of days and

98 -89- Figure 42. Comparison of the number of years required to obtain significance for different decreases in Monte Carlo trials of daily insurance data in area 3 (dollars and acres - a =.05).

99 -90- Figure 43. Comparison of the number of years required to obtain significance for different significance levels for Monte Carlo trials of daily insurance data in area 3 (40% decrease).

100 Table 14. The least squares simple regression parameters for Monte Carlo trials of daily insurance data (a = 0.05). Dollars Acres 40% 60% 80% 40% 60% 80% 1) Continuous Design "U" Test V a b Equation y = a + b In x 1/y = a + b/x y = a + b In x y = a + b In x 1/y = a + b/x 1/y - a + b/x 2) Crossover Design "U" Test V a b Equation 1/y = a + b/x 1/y - a + b/x 1/y = a + b/x y = a + b x 1/y = a + b/x 1/y = a + b/x 3) Random Design "U" Test r a b Equation y = a + b In x 1/y = a + b/x 1/y = a + b/x 1/y = a + b/x 1/y - a + b/x 1/y = a + b/x 4) Crossover Design "t" Test Non-Transformed r a b Equation y = a + b x y = a + b x y = a + b In x y = a + b x 1/y - a + b/x y = a + b In x 5) Random Design "t" Test Transformed V a b Equation y - a + b x y = a + b In x 1/y = a + b/x y = a + b x y = a + b in x y = a + b In x 6) Random Design "t" Test Non-Transformed V a b Equation y - a + b x y = a + b x y = a + b In x In y = a + b x y = a + b in x In y = a + b In x

101 -92- Table 15. The least squares simple regression parameters for Monte Carlo trials of daily insurance data for a 40% decrease. a = 0.01 a = 0.10 Dollars Acres Dollars Acres 1) Continuous Design "U" Test V a b Equation y - a + b x y - a + b x y = a + b In x y = a + b In x 2) Crossover Design "U" Test r a b Equation y - a + b x y - a + b x y = a + b In x In y = a + b In x 3) Random Design "U" Test V a b Equation y = a + b x in y = a + b In x y = a + b In x y = a + b In x 4) Crossover Design "t" Test Non-Transformed v a b Equation In y = a + b In x y = a + b x In y = a + b In x i n y = a + b l n x 5) Random Design "t" Test Transformed T a b Equation y - a + b x y = a + b x y = a + b x y = a + b x 6) Random Design "t" Test Non-Transformed r a b Equation In y = a + b In x In y = a + b x In y = a + b x y = a + b x

102 -93- the size of area are shown to be the most important factors. In general, as the average number of damage days and the size of area increase, the sample size required to obtain significance is decreased in daily data. Table 16. Comparison by ranking of areas and type of data measurement for a 20% decrease using the gamma test. Average number of Size damage days of area Rank of areas" (May-October) (mi 2 ) Location Yearly Data Area 2 (acres) South Area 3 (acres) South Area 4 (acres) North Area 1 (acres) North Area 3 (dollars) South Area 2 (dollars) South Area 4 (dollars) North Area 1 (dollars) North Daily Data Area 4 (acres) North Area 3 (acres) South Area 4 (dollars) North Area 2 (acres) South Area 3 (dollars) South Area 1 (acres) North Area 2 (dollars) South Area 1 (dollars) North "Areas ranked in descending order according to ability of detection at all ß levels A comparison of the type I and type II errors with the experimental units (days or years) is shown in Table 17. This table illustrates that regardless of the choice of a and ß, the daily experimental unit is a more efficient unit for detecting significant differences than is the yearly experimental unit.

103 -94- Table 17. Comparison by ranking" of the type I and type II errors with the experimental units for a 20% decrease and using the sequential normal test in area 1. Type I error Type II error Experimental unit Day Day Day Day Day Day Year Year Year Year Year Year *Ranked in descending order according to ability to detect Table 18 presents the various distributions ranked by their detection ability for a 20% decrease, and these allow comparison of the designs and tests. In general for both yearly and daily data, the sequential test is shown to be better than the non-sequential test, the continuous design better than the random, and the 1-sample test better than the 2-sample test. The continuous design poisson 1-sample test has less power to detect significance than does the normal 1-sample non-sequential test. The crossover random design (normal 2-sample test) and the single area random design (normal 1-sample test) using the non-sequential analysis require smaller sample sizes than does the random design using the poisson 1-sample test and the sequential analysis. Finally, the target-control continuous design and the single area completely randomized design require the largest sample sizes. Although the 1-sample test generally outperforms the 2-sample, the crossover random 2-sample test is more powerful than the normal and poisson 1-sample random tests. Table 19 shows a ranking based on the length of detection period with the associated experimental units for given percentage decreases in hail and the

104 -95- Table 18. Comparison by ranking of theoretical distributions with designs and test for a 20% decrease at an a level of 0.05 and a ß level of Rank of Data collection distributions* and design Analysis Test Yearly Gamma Single area, Continuous Sequential Likelihood ratio 1-sample Gamma Single area, Random Sequential Likelihood ratio 1-sample Log-normal Single area, Continuous Sequential Normal, 1-sample Gamma Single area, Continuous Non-sequential Likelihood ratio 2-sample Gamma Single area, Random Non-sequential Likelihood ratio 2-sample Log-normal Crossover, Random Non-sequential Normal, 2-sample Log-normal Single area, Random Sequential Normal, 1-sample Log-normal Single area, Continuous Non-sequential Normal, 1-sample Log-normal Single area, Random Non-sequential Normal, 1-sample Log-normal Continuous, Non-sequential Normal, 1-sample Target-control Log-normal Single area, Random Non-sequential Normal, 2-sample Daily Log-normal Single area, Continuous Sequential Normal, 1-sample Log-normal Single area, Random Sequential Normal, 1-sample Log-normal Single area, Continuous Non-sequential Normal, 1-sample Poisson (> $150 loss - Single area, Continuous Sequential Poisson, 1-sample discrete) Poisson (> $200 loss - Single area, Continuous Sequential Poisson, 1-sample discrete) Log-normal Random, Crossover Non-sequential Normal, 2-sample Log-normal Single area, Random Non-sequential Normal, 1-sample Poisson (> $150 loss - Random Sequential Poisson, 1-sample discrete) Poisson (> $200 loss - Random Sequential Poisson, 1-sample discrete) Log-normal Continuous, Non-sequential Normal, 2-sample Target-control Log-normal Single area, Random Non-sequential Normal, 2-sample *Distributions ranked in descending order according to ability of detection

105 -96- types of data measurement. An 80% reduction in the yearly data (acres or dollars) is required in order to have a length of detection period less than that of a 20% reduction in the daily loss data. Also, a 40% reduction in the yearly data is required at the same power level to have detection properties approximating a 10% reduction in the daily data. It is also seen that a 5% reduction in the daily data is easier to detect than a 20% reduction in the yearly data. Table 19. Rank comparison of the experimental unit with a given decrease and type of data measurement using a 1-sample test, sequential analysis, and a continuous design in area 1. Rank of Percentage Experimental measurement type* decrease unit Acres 80 Daily Dollars 80 Daily Acres 60 Daily Dollars 60 Daily Acres 40 Daily Dollars 40 Daily Acres 80 Yearly Dollars 80 Yearly Acres 20 Daily Dollars 20 Daily Acres 60 Yearly Dollars 60 Yearly Acres 10 Daily Dollars 10 Daily Acres 40 Yearly Dollars 40 Yearly Acres 5 Daily Dollars 5 Daily Acres 20 Yearly Dollars 20 Yearly Acres 10 Yearly Dollars 10 Yearly Acres 5 Yearly Dollars 5 Yearly *Type of measurement ranked in descending order according to ability of detection

106 -97- The number of years required to obtain significance for various percentage reductions, selected tests, and designs is listed in Table 20. Even with the liberal type II error of 0.5, over 100 years are required to detect a 5% decrease for any design and test. The optimum test and design with respect to sample size requires approximately 6 years for a 3 level of 0.5 and for a 20% decrease. A 40% decrease would be detected in less than 10 years using several designs. Even with a more stringent type II error of 0.1, significant 40% decreases could be obtained in less than 5 years, with certain design-test combinations. Table 20. Number of years required to detect various decreases in daily insurance data for selected designs and a = Number of years required to obtain significance for given percentage decrease Design ß 5% 10% 20% 40% 60% 80% Continuous seeding in area 1 using daily dollar losses and a 1-sample test, sequential analysis Continuous seeding in area 1 using daily dollar losses $150 and a 1-sample test, sequential analysis Target control seeding in areas 1 and using daily dollar losses Crossover between areas 1 and using daily dollar losses Random seeding in area 1 using daily dollar losses and a 2-sample test, non-sequential analysis

107 -98- A comparison between the algebraic and Monte Carlo computations of sample size is shown in Table 21. From the table the following conclusions can be drawn: 1) the non-parametric test has slightly more power than the parametric test; 2) the normal test and the transformed "t" test have nearly the same power to detect, indicating that the algebraic computations of sample size are sufficiently accurate for percentage decrease computation (the departure at 0.10 is due to extrapolation errors in extending the regression equation to this region); and 3) there is a loss in power when the "t" test is applied directly to the skewed data. Table 21. Comparison between algebraic and Monte Carlo estimates of years to detect various decreases, for an a level of 0.05 and for area 3 daily monetary insurance data. Crossover design (2-sample test) Single area random (2-sample test) Albegraic Monte Carlo Algebraic Monte Carlo Non- Trans. Non- "U" trans. "U" "t" trans. Decrease Normal test test "t" test Normal test test test "t" test (Number of years to obtain significance) 40% ß = ß = % ß = ß = % ß = ß = NETWORK HAILSTREAK DATA This section describes an investigation of individual hailstorm characteristics. These data are another form of hail measurement that could be used in verifying hail suppression experiments.

108 Data and Analytical Adjustments The operation of a dense hail-observing network in central Illinois during April-September of 1967 furnished detailed data on the surface hail patterns on 26 days of hail (Changnon, 1968a). Areas of hail continuous in time and space, defined as hailstreaks, were ascertained within the 400 mi 2 study area which contained 98 cooperative hail observers and 49 other instrumented sites each with a recording raingage modified to record time of hail (Changnon, 1966) and a 1-ft 2 foil -covered hailpad (Wilk, 1961) to record hailstone sizes and energy of the hailfalls (Fig. 44). Insurance adjustor data also were obtained for those hailstreaks that produced crop damage. Only those hailstreaks that had at least 3 locations within the streak boundary with time of hail and at least 2 locations with measurable energy values, and that occurred entirely within the study area, were used in the analysis. The recording raingage data allowed the mapping and recording of the individual rain cells, with and without hailstreaks, that crossed the study area during the hailfall periods (Changnon et al., 1967). In the 6-month data collection period 77 hailstreaks so defined occurred within the area. For each of these the areal extent was measured, and the area-mean energy imparted by the hailstones was calculated using all hailpad energy values from within the hailstreak. An example of the hailstreaks with their area and energy values for a hail period on 9 June 1967 is presented in Fig. 44. There were 13 other hailstreaks in this 1-hr period, bat these did not have complete life histories within the study area. The purpose of the individual hailstreak study was to provide expressions of the natural differences between temporally related hailstreaks. These data were then used to determine required sample size to verify potential suppression experiments that might be based on hailstreak data from a pair of similar clouds, where one member of the pair is randomly seeded. To more nearly simulate an actual field experiment, certain limiting criteria were defined for selecting hailstreaks for comparison. First, any two hailstreaks to be compared had to occur within a 1-hr period. This derived from a basic assumption that any hailstreaks occurring in this area within a 1-hr period likely derived from separate convective clouds that had similar meteorological characteristics at about the same time prior

109 Figure 44. Hail observing points in study area and complete hailstreaks on 9 June 1967.

110 -101- to their production of hail in the study area. That is, each would have derived from a cloud that fulfilled any one of several possible criteria of cloud selection, such as moisture content and height, and their development times were sufficiently close to fit within a realistic operational approach. Secondly, all rain cells occurring over the network and not producing hail during the entire period of hail had to be determined and used to represent potentially chosen clouds that did not produce any hail. Thirdly, if a 1-hr period of hailstreaks was separated from another such period by 4 hours or more, the two periods and their hailstreaks were considered to be separate entities for a potential seeding experiment and in our analysis. Hence, two or more discrete 1-hr hail periods could occur in the area on a given day, and two days in 1967 did have two such hail periods (Table 22). Table 22. Number of complete hailstreaks, no-hail rain cells, and rain cell combinations possible in study area during Number of pairs Number of Both members One member Neither member Date of 1-hour Number of rain cells with associated with associated with associated hail period hailstreaks with no hail hail hail hail 4/ /5 (AM) /5 (PM) / / /21 (early PM) /21 (late PM) / / / / / / / / / / / / / / Totals

111 -102- Comparisons were made between the area and energy values of all possible pairs of rain cells, with or without hail, in a given 1-hr hail period. Using the example illustrated on Fig. 44, for which all rain cells had associated hail, the area and energy values of hailstreak number 1 were compared with those of hailstreaks 2, 3, 4, and 5; those of number 2 with 3, 4, and 5; those of number 3 with those of 4 and 5, and those of number 4 with 5. Thus, for the 9 June period there were ten pairs of hailstreaks for which comparisons were made. The number of hailstreaks and potential pairs for the 23, 1-hr hail periods of 1967 appear in Table 22. The area and energy values of the 77 individual hailstreaks are presented in Table G of the appendix. 2. Natural Variability of Hailstreak Area and Energy Values The areal extent and area-mean energy values of the 77 hailstreaks exhibited considerable variability in the given hail periods. The median and extreme values obtained are listed in Table 23. The difference between the area extremes is large, and that between the extreme energy values is exceptional, more than a factor of The differences between the median and average values are indicative of the wide distribution of the values and of the influence of the few large values. Table 23. Median and extreme values for 77 hailstreak areas and energies. Area Energy (mi 2 ) (lb /ft 2 ) Average Median Maximum Minimum The greatest difference between hailstreak area values in a single 1-hr period was 30 mi 2 on 28 May when the smallest hailstreak covered 2 mi 2 and the largest covered 32 mi 2. The greatest energy difference in a single hail period occurred on the early afternoon of 21 April when one hailstreak had a mean

112 -103- energy of Ib/ft 2, as determined from 2 hailpad values inside the hailstreak, whereas another had a mean energy of only Ib/ft 2. Such great natural differences are indicative of the difficulties that will be associated with verifying hail suppression experiments designed around paired clouds. The log-normal distribution was fitted to the hailstreak data and the result is shown in Fig. 45. The difference between the area and energy data is readily demonstrated. The area data is nicely fitted by the non-truncated log-normal distribution, but the energy data requires a truncated log-normal distribution with a truncation point of The truncated distribution was obtained by deleting from the sample all values less than or equal to , and by making the transformation (x ) on the remainder of the sample. The log-normal mean and variance were then estimated from the transformed sample. The fact that only 47 values were left out of a sample of 77 illustrates the severity of the truncation. The area values and energy values of the 77 hailstreaks were correlated to measure the degree of their relationship using data based on dense point measurements. The correlation coefficient was which meant that the areal extent of a hailstreak has no relationship with its area-mean energy value. This would strongly suggest that measurements of hail area could not be used to derive suitable estimates of the force of hail for individual storms. 3. Experimental Design and Empirical Frequency Distributions In the paired storm design, a pair of clouds are selected with similar characteristics, and one member of the pair is then chosen at random to be seeded. The 147 pairs of rain cells from Table 22 in which both members had associated hailstreaks were assumed to have originated from clouds with similar characteristics. Thus, the associated hailstreaks were assumed to be hail that would have been produced from clouds meeting the paired storm design criteria. One member of each pair was selected at random and designated as seeded. The differences between the areas and between the energy values of the seed and no-seed hailstreaks were then computed, and the cumulative ogives for the empirical distributions of differences were formed. These distributions were designated as the natural distributions, that is, the distributions expected if seeding had not occurred. The values of the seeded

113 -104- Figure 45. Log-normal frequency distributions of the areal extent of hailstreaks and energy of hailfall within the hailstreak.

114 -105- hailstreaks of each pair were then reduced 20, 40, 60, and 80% and the respective cumulative ogives were formed (Fig. 46). The differences in the curves are assumed to be the effect that seeding would have on the natural differences. A mixed distribution was then estimated based on two assumptions. First, there is a non-zero probability of obtaining a pair of rain cells which have associated hail with each member. Secondly, when such a pair occurs, the differences of area (energy) are distributed in the form of the cumulative distributions of Fig. 46. The general form of the mixed distribution function is the same as Eq. 7, that is: where: P(X < a) = probability of receiving less than a specified difference of area (energy) P(X = 0) = probability of having experienced a pair of rain cells with only one member having associated hail, or neither member having associated hail P(X > 0) = probability of having experienced a pair of rain cells with both members having associated hail P(X < a X > 0) = probability of receiving less than a specified difference of area (energy) given that a pair of rain cells which has both members seeded has occurred. The term P(X < a X > 0) is given by: The density function, f(x), is specified to be the derivative of the cumulative ogives from Fig. 46.

115 -106- Figure 46. Empirical distributions of areal extent and energy of hailfall.

116 Test of Hypothesis Because of the dependence between members of pairs, the appropriate test to use is the Wilcoxin Match Pair Signed Rank test. To derive the number of years required to obtain significance for assumed decreases of 20, 40, 60, and 80%, sample values were generated from the curves of Fig. 46. As each sample value was generated, a tabulation was made of whether significance was obtained for the Wilcoxin Matched Pair Signed Rank test. This was continued until a specified number of values had been generated. The process was then repeated, so that a frequency distribution of the number of runs significant at a particular sample size was obtained. Again, the percentage number of significant runs at each sample size is equivalent to the power of the test. 5. Results for the Paired Hailstreak Data The number of years required to obtain significance for various significance levels using the paired hailstreak data is shown in Figs. 47a-d. Comparison of these curves reveals that the energy data requires much more time to obtain significance than does the areal extent data. Unfortunately, areal extent is poorly correlated with the energy of the hailstreak. These nomograms illustrate that the smaller the decrease desired to detect, the more important the choice of significance level becomes. For example, if one chooses the 0.01 level of significance instead of 0.10, it would require 10 more years to detect a significant difference for a 20% decrease at ß = However, this difference is only 0.42 year for a 60% decrease. The number of years required to obtain significance for various decreases can be compared in Figs. 48a-c. For a significance level of 0.05 and ß = 0.20, 0.15 year is required to detect a 80% difference in area, whereas more than 10 years are required to detect a 20% decrease in the area values. The number of years was obtained by assuming that 1) there would be 25 of the 1-hr periods per year with 2 or more storms each producing one or more hailstreaks, and 2) one of these storms in each pair could be seeded on an operational basis. There were 21 such 1-hr periods during 1967 in a 400 mi 2 area, and there were 17 such 1-hr periods of 2 or more hailstreaks in 1968 in a 1000 mi 2 area (Changnon, 1968c).

117 Figure 47. Comparison of the number of years required to obtain significance for various significance levels for the paired storm design using energy and areal extent as parameters (20% decrease).

118 -109- Figure 48. Comparison of the Timber of years required to obtain significance for various decreases for the paired storm design using energy and areal extent as parameters.

119 -110- RESULTS OF APPLYING THE STATISTICAL METHODOLOGY 1. Comparison of Various Tests, Designs, and Types of Data Table 24 lists the number of years required to obtain a 20% decrease (a = 0.05, B = 0.20) in all ten study areas based on the "best" test and design (best in that these required the smallest sample size) discovered for the various types of data measurement. Results for the Weather Bureau annual data show no direct relationship between the years required for a 20% decrease and the area size or the average number of hail days. However, the Weather Bureau summer data suggest some trend toward smaller numbers of years required with the larger averages of hail days and the larger areas. The size of area and average number of hail days have little effect on the years required for the annual acre insurance data. However, sizes and averages are important in the daily acre insurance results which indicate that larger areas and larger numbers of hail days reduce the number of years required to obtain significance. In general, annual Weather Bureau hail days and the daily insurance data yield the smaller sample sizes, with averages of 8.5 and 9.2 years, respectively. These values are 40% lower than the number of years required on the paired storm (hailstreak) design, and are markedly lower than the other averages in Table 24. The type I errors, type II errors, and the experimental unit for the various types of data are compared for a 20% decrease in Table 25. For a 3 value of 0.20, the average number of years to obtain significance for an a level of 0.05 is 24.0; the more liberal choice of a = 0.10 requires 20.7 years on the average, and the conservative level of 0.01 requires 29.8 years. For a 3 value of 0.50, the average number of years to obtain significance for a = 0.05 is 9.5; the more liberal level of a = 0.10 requires an average of 7.1 years, and the conservative estimate of a = 0.01 requires 13.1 years on the average. For a significance level of 0.05, the average number of years to obtain significance varies from 24.0 to 9.5 as ß varies from 0.20 to Table 26 shows the number of years required to obtain significance for various decreases based on the "best" designs and tests for the various types of hail data. If the 20% and 40% decreases for all tests, designs, and 3 = 0.20 are averaged, the number of years required for the 20% decrease is %

120 -111- Table 24. Comparison of areas with the "best" test and design of various types of data measurement for detecting a 20% decrease (a = 0.05, 3 = 0.20). Number of years Average number Area size to obtain Area of days (mi 2 ) significance Weather Bureau Annual Hail Days Average 8.5 Weather Bureau Summer Hail Days Average 42.6 Insurance Data, Yearly-Acres Average 36.1 Insurance Data, Daily-Acres Average 9.2 Monte Carlo Trials 20% decrease not available Paired Storms, Areal Extent ECIN* 25 (No. of Pairs) Paired Storms, Mean Energy 20% decrease not available *East Central Illinois Network

121 -112- Table 25. Comparison of the type I and type II errors and experimental unit with "best" tests for various types of data and a 20% decrease. Number of years Type II error Experimental to detect (ß) Type of data unit decrease a = Weather Bureau hail days Year Weather Bureau hail days Summer Insurance data (acres) Year Insurance data (acres) Day Monte Carlo Day not available 0.2 Paired storms (area) Storm Paired storms (energy) Storm not available Average 20.7 a = Weather Bureau hail days Year Weather Bureau hail days Summer Insurance data (acres) Year Insurance data (acres) Day Monte Carlo Day not available 0.5 Paired storms (area) Storm Paired storms (energy) Storm not available Average 7.1 a = Weather Bureau hail days Year Weather Bureau hail days Summer Insurance data (acres) Year Insurance data (acres) Day Monte Carlo Day not available 0.2 Paired storms (area) Storm Paired storms (energy) Storm not available Average 24.0

122 -113- Table 25 (continued) Number of years Type II error Experimental to detect (ß) Type of data unit decrease a = Weather Bureau hail days Year Weather Bureau hail days Summer Insurance data (acres) Year Insurance data (acres) Day Monte Carlo Day not available 0.5 Paired storms (area) Storm Paired storms (energy) Storm not available Average 9.5 a = Weather Bureau hail days Year Weather Bureau hail days Summer Insurance data (acres) Year Insurance data (acres) Day Monte Carlo Day not available 0.2 Paired storms (area) Storm > Paired storms (energy) Storm not available Average 29.8 a = Weather Bureau hail days Year Weather Bureau hail days Summer Insurance data (acres) Year Insurance data (acres) Day Monte Carlo Day not available 0.5 Paired storms (area) Storm Paired storms (energy) Storm not available Average 13.1

123 -114- Table 26. Number of years required to detect various decreases using "best" designs for each type of hail data (a = 0.05). "Best" Number of years required to obtain significance for given percentage decrease design ß 5%_ 10%_ 20% 40% 60% 80% Weather Bureau hail days Annual Continuous seeding in area 5 using hail-day data 1-sample test sequential analysis Summer Continuous seeding in area 5 using hail-day data 1-sample test sequential analysis Insurance data Yearly Continuous seeding in area 1 using daily acre losses 1-sample test sequential analysis Target-control seeding in area 1 and 2 using daily , , , acre losses non-sequential , , , analysis Crossover between areas 1 and 2 using , , daily acre losses non-sequential 0.5 6, , analysis Random seeding in area 1 using daily , , , acre losses nonsequential analysis , , ,

124 -115- Table 26 (continued) Number of years required to obtain significance "Best" for given percentage decrease design ß 5% 10% 20% 40% 60% 80% Daily Continuous seeding in area 1 using daily acre losses 1-sample test sequential analysis Continuous seeding in area 1 using daily acre losses $150 1-sample test sequential analysis Target-control seeding in areas 1 and 2 using 0.2 2, daily acre losses nonsequential analysis Crossover between areas 1 and 2 using daily acre losses nonsequential analysis Random seeding in area 1 using daily acre 0.2 2, losses non-sequential analysis Monte Carlo trials* (acres) Daily Continuous seeding in area 3 using acre losses 2-sample test non-sequential analysis non-parametric Crossover between areas 3 and 4 using daily acre losses non-parametric *Monte Carlo trials were not performed for the smaller percentage decreases because of the great amount of computer time involved.

125 -116- Table 26 (concluded) Number of years required to obtain significance "Best" for given percentage decrease design ß 5% 10% 20% 40% 60% 80% Random seeding in area 3 using daily acre losses nonparametric Paired storm data Randomized seeding within pairs non-sequential 2- sample test using areal extent greater than that for the 40% decrease. This indicates that it is highly desirable to obtain reductions of 40% or more in hail data if one desires to detect seeding effects quickly and within small error bounds. 2. Application of Methodology for Other Climatic Areas The methodology developed here is applicable to other climatic areas. In some instances different distribution functions may be required, and the quality of data may not equal that of the Illinois data. The application of actual numerical results from the Illinois research to other areas should be done with considerable caution. In particular, the data from other climatic areas under consideration should be checked to see if they exhibit averages and variances similar to the Illinois data. It is believed that the Illinois nomograms can be used as first approximations of the sample requirements in many other areas.

126 -117- SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 1. Summary A statistical methodology involving the analysis of three basic types of historical hail data on an area approach was developed for the planning and evaluation of hail suppression experiments. The methodology was then used to generate nomograms relating the number of years required to detect significant results to 1) type I error, 2) type II error, and 3) power of the test for various statistical tests and experimental designs. These nomograms were constructed for various sized areas and geographical locations within the state. The research involved the sequential analysis approach as well as the classical non-sequential approach to statistical analyses. Theoretical ASN equations were derived for the two-parameter gamma distribution and were used to construct some of the nomograms. 2. Conclusions The methodology appears to be adequate for obtaining useful estimates of the time required to detect an effect of seeding with specified power. The methodology is_ applicable to other climatic areas provided the appropriate historical record and theoretical distribution functions are used, although the Illinois nomograms may be only first approximations of those for other hail climates. Any 1-year experiment in hail suppression will involve risks in deriving meaningful conclusions, regardless of how elegant the statistical analysis. For this length of experiment, a 40% decrease is the minimum decrease which can be detected with a significance level of 0.05 and type II error of 0.5 (that is, the probability of rejecting a seeding-produced decrease in hail when the decrease actually existed). In order to detect a small decrease for a 1-year experiment, the type II error increases rapidly. For a significance level of 0.05 and a power (that is, the probability of detecting a seeding decrease when seeding decrease is present) level of 0.50, the minimum number of years required to detect a 20% decrease with a continuous design is 4.1 years for Weather Bureau hail days and 4.7 years for summer daily insurance data. For a power level of 0.80, the values are 11.0 and 12.6

127 -118- years, respectively. With randomization in the experiment, this value increases; or, the value increases if one relies on the classical non-sequential analysis. The insurance acreage data required less time than monetary loss data to obtain significance. Although there are problems inherent in the insurance data (changing liability and seasonal variability in crop susceptibility), these data afford the best measure of detecting seeding effects from a hail suppression program, provided the experiment is conducted in areas with liability coverage of at least 60%. The decision of which design and test to use is illustrated schematically in Fig. 49. Considering the diagram to be a representation of all tests and designs employed in this research, zone A contains the family of designs and tests which yield the least amount of time to obtain significance, although some of the assumptions and techniques may be questionable. Zone C incorporates the designs and tests with the most stringent requirements in regard to randomization and assumptions (hence, most valid), but its designs require the longest time to obtain significance. Since the designs and tests in zone C often require exorbitant sample sizes, and those in zone A are not always valid, the family of designs and tests in zone B are the most logical to use. This zone would include the 1-sample tests with both the sequential and non-sequential analysis approach, and some randomization in the design. Hence, the recommendation resulting from this research is to use the single area design in which all potential storms are seeded on a particular day with the randomization being applied to days rather than storms. The randomization factor could vary from 1/2 to 1/5 with preference being given the smaller fractions. That is, 50 to 20% of the days should be retained for a control in the experiment. If the requirements for sequential analysis are fulfilled by the data sample involved, the sequential approach is the best one to use in verifying the results. The risks involved in the sequential analysis, as opposed to the non-sequential approach, are outweighed by the reductions in the sample size required. This view is supported by the fact that observations and field operations in the dimension of time are expensive, difficult, and time consuming.

128 Figure 49. Schematic representation of validity zones for various tests and designs.

129 Recommendations for Additional Research The research presented in this report points to several areas in which additional research would be quite useful. First, the possibility of using individual stations rather than areal averages for the basis of the statistical analysis should be investigated to see if the power of the test could be improved, hence reducing the sample size required. Secondly, in this research a constant percentage decrease in the regional data was assumed in the methodology. It would be desirable to consider other forms of decreases, such as percentage decreases of individual stations with random error distributions superimposed on these differences. In this research, all crops were used as the basic parameter in the insurance data. The study based on individual crops might possibly reduce the variability since the time period in which the crop susceptibility to damage is greatest varies somewhat from crop to crop. For example, corn and soybeans, the major Illinois crops, are much alike in susceptibility to damage, but differ from wheat and other small grains. Finally, for similar research in other areas, it would be helpful to develop better adjustment factors for removing the effect of temporal changes in farming practices and liability from the insurance data. BIBLIOGRAPHY Battan, Louis J., 1966: "Silver-iodide seeding and rainfall from convective clouds," J. of Appl. Meteor., 5: Braham, R. R., Jr., 1966: Final report of project Whitetop, a convective cloud randomized seeding project. Part 1, Chicago Department of the Geophysical Sciences, University of Chicago, 156 pp. Changnon, S. A., Severe summer hailstorms in Illinois during Trans Acad. Sci., 53, Changnon, S. A., 1963 Monthly and semi-monthly distributions of hail days in Illinois. Chicago, Research Report 17, Crop-Hail Insurance Actuarial Association, 17 pp.

130 -121- Changnon, S. A., 1966: Meteor., 5: "Note on recording hail incidences," J. of Appl. Changnon, S. A., 1967a "Method of evaluating substation records of hail and thunder," Mo. Wea. Rev., 95, Changnon, S. A., 1967b: "Areal-temporal variations of hail intensity in Illinois," J. of Appl. Meteor., 6, Changnon, S. A., 1967c Summary of 1966 hail research in Illinois, Chicago, Research Report 33, Crop-Hail Insurance Actuarial Association, 37 pp. Changnon, S. A., 1967d: Hail evaluation techniques project, Urbana, Annual Report, NSF Contract GA-482, Illinois State Water Survey, 11 pp. Changnon, S. A., P. T. Schickedanz, and H. Q. Danford, 1967: "Hail patterns in Illinois and South Dakota," Proceedings of Fifth Conference on Severe Local Storms, pp Changnon, S. A., and G. E. Stout, 1967: "Crop-hail intensities in central and northwest United States." J. of Appl. Meteor., 6, Changnon, S. A., 1968a: Summary of 1967 hail research in Illinois, Chicago, Research Report 39, Crop-Hail Insurance Actuarial Association, 51 pp. Changnon, S. A., 1968b: "Effect of sampling density on areal extent of hail," J. of Appl. Meteor., 7, Changnon, S. A., 1968c. Hail evaluation techniques, Urbana, Part I, Final Report NSF GA-482, 48 pp. Changnon, S. A., and P. T. Schickedanz, 1969: "Utilization of hail-day data in designing hail suppression projects," Mo. Wea. Rev., 97: Davies, Owen L., 1954: The design and analysis of industrial experiments, New York: Hafner Publishing Company, First Edition, 636 pp. Decker, W. L., 1952: "Hail-damage frequency distributions for Iowa, and a method of evaluating the probability of a specified amount of hail damage," Trans. Amer. Geophys. Union, 33: Dennis, A. S., and D. F. Kriege, 1966: "Results of ten years of cloud seeding in Santa Clara County, California," J. of Appl. Meteor., 5:

131 -122- Fix, E., 1954: "Tables of noncentral x 2 " University of California Publications in Statistics, Berkeley, University of California Press, Volume 1 (1949), pp Gabriel, K. R., 1967: "The Israeli artificial rainfall stimulation experiment; statistical evaluation for the period ," Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Hagen, L. J., and A. F. Butchbaker, 1967: "Climatology of hailstorms and evaluation of cloud seeding for hail suppression in southwestern North Dakota," Proceedings of Fifth Conference on Severe Local Storms, pp Hahn, G. J., and S. S. Shapiro, 1967: Statistical models in engineering, New York, John Wiley and Sons, Inc., 355 pp. Huff, F. A., and S. A. Changnon, 1959: Hail climatology of Illinois, Urbana, Report of Investigation 38, Illinois State Water Survey, 46 pp. Illinois Cooperative Crop Reporting Service, 1958: Illinois agricultural statistics. Springfield, Bulletin 58-2, 110 pp. Lehmann, E. L., 1959: Testing statistical hypotheses, New York, John Wiley and Sons, Inc., First Edition, 369 pp. Liffiefors, H. W., 1967: "On the Kolmogrov-Smirnov test for normality with mean and variance unknown," J. of the Amer. Stat. Assoc, Schickedanz, P. T., 1967: A Monte Carlo method for estimating the error variance and power of the test for a proposed cloud seeding experiment, Unpublished Ph.D. Thesis, University of Missouri, Columbia, June 1967, 113 pp. Available from University Microfilms, Ann Arbor, Michigan. Schickedanz, P. T., and W. L. Decker, 1968: The determination of optimum design and minimum duration of cloud seeding experiments, Proceedings of the First Statistical Meteorological Conference, Schleusener, R. A., and A. H. Auer, 1964: Hailstorms in the high plains, Ft. Collins, Final Report NSF Grant G-23706, Colorado State University, 100 pp. Schleusener, R. A., J. D. Marwitz, and W. L. Cox, 1965: "Hailfall data from a fixed network for the evaluation of a hail modification experiment," J. of Appl. Meteor., 4:61-68.

132 -123- Smith, E. J., E. E. Adderley, and F. D. Bethwaite, 1965: "A cloud-seeding experiment in New England, Australia," J. of Appl. Meteor., Stout, G. E., 1961: Review of hail suppression activities in the United States and Canada since 1956, Chicago, Research Report 8, Crop-Hail Insurance Actuarial Association, 9 pp. Sulakvelidze, G. K., 1966: "Results of the Caucasus anti-hail expedition of 1965," Trudy, Issue 7, Swed, F. S., and C. Eisenhart, 1943: "Tables for testing randomness of grouping in a sequence of alternatives," The Annals of Math. Stat., 14: Thorn, H. C. S., 1957a: "The frequency of hail occurrence," Technical Report 3, Advisory Committee on Weather Control, pp Thorn, H. C. S., 1957b: "A method for the evaluation of hail suppression," Technical Report 4, Advisory Committee on Weather Control, pp Thorn, H. C. S., 1958: 86: "A note on the gamma distribution," Mo. Wea. Rev., Thorn, H. C. S., 1966: Some methods of climatological analysis, Geneva, Switzerland, Technical Note 81, World Meteorological Organization, 156 pp. Wald, A., 1947: 212 pp. Sequential Analysis, New York, John Wiley and Sons, Inc., Wilk, K. E., 1961: Radar investigations of Illinois hailstorms. Urbana, Scientific Report 1, AF 19(604)-4940, Illinois State Water Survey, 42 pp. Wilks, S. S., 1938: "The large sample distribution of the likelihood ratio for testing compositive hypotheses," The Annals of Math. Stat., 9:60-62.

133 -124- APPENDIX Table A. Seasonal and Annual Numbers of Hail Days in Five Areas Defined for U. S. Weather Bureau Data Area 1 Year Spring Summer Fall Annual

134 -125- Table A (Continued) Area 2 Year Spring Summer Fall Annual

135 -126- Table A (Continued) Area 3 7 Year Spring Summer Fall Annual l l

136 -127- Table A (Continued) Area 4 Year Spring Summer Fall Annual

137 -128- Table A (Concluded) Area 5 Year Spring Sunnier Fall Annual

138 -129- Table B. Daily Crop-Hail Insurance Loss Data For Areas 1-2,

139 -130- Table B (Continued)

140 131 Table B (Concluded)

141 132 Table C. Daily Crop-Hail Insurance Loss Data for Areas 3-4,

142 133 Table C (Continued)

143 134 Table C (Continued)

144 135 Table C (Concluded)