THE SIGNIFICANCE OF SYNOPTIC PATTERNS IDENTIFIED BY THE KIRCHHOFER TECHNIQUE: A MONTE CARLO APPROACH

Transcription

1 INTERNATIONAL JOURNAL OF CLIMATOLOGY Int. J. Climatol. 19: (1999) THE SIGNIFICANCE OF SYNOPTIC PATTERNS IDENTIFIED BY THE KIRCHHOFER TECHNIQUE: A MONTE CARLO APPROACH ROBERT K. KAUFMANN*, SETH E. SNELL, SUCHARITA GOPAL and RAY DEZZANI Department of Geography, 675 Commonwealth A enue, Boston Uni ersity, Boston, MA 02215, USA Recei ed 10 February 1998 Re ised 21 October 1998 Accepted 11 No ember 1998 ABSTRACT The Kirchhofer technique for identifying spatial patterns in climate data depends on the threshold and minimum size used to define the groups. As a result, the analyst cannot be sure whether the groups defined by the Kirchhofer technique represent meaningful meteorological phenomena or whether they emerge due to random chance. Monte Carlo techniques are used to generate a three-dimensional surface that can be used to evaluate the statistical significance of the groups identified from the historical record for summertime maximum daily temperature in the mid-continental region of the US. Statistically meaningful groups can be identified using stringent criteria for threshold and minimum group size. Choosing among these values depends on how the groups are used and the importance of parsimony to that process. The reliability of critical values can be evaluated by comparing the statistical significance of results for individual groups generated by existing techniques. Copyright 1999 Royal Meteorological Society. KEY WORDS: Kirchhofer technique; synoptic patterns; Monte Carlo techniques 1. INTRODUCTION The Kirchhofer technique (Kirchhofer, 1974) is used widely to classify synoptic patterns (Barry et al., 1987; Yarnal, 1993). Nonetheless, the interpretation of results is clouded by uncertainty. The number and size of groups identified by the Kirchhofer technique depends on the threshold and minimum group size (Key and Crane, 1986). There are no criteria to choose these parameters. As a result, analysts cannot be sure whether the synoptic patterns identified by the Kirchhofer technique represent groups that are generated by a meaningful meteorological phenomena or whether they emerge due to random chance. This paper develops a distribution to evaluate the statistical significance of synoptic patterns identified by the Kirchhofer technique (Kirchhofer, 1974). Section 2 describes the Kirchhofer methodology and how the lack of criteria for threshold and minimum group size introduces uncertainty about the statistical significance of the groups identified. Section 3 illustrates how this uncertainty affects the results generated by an historical analysis of maximum daily temperature patterns in the mid-continental region of the US. Section 4 describes a methodology to generate critical values that can be used to evaluate the statistical significance of groups identified by the Kirchhofer technique. In Section 5, these critical values are used to identify statistically meaningful groups from the historical data. Section 6 concludes with a description of how the reliability of this methodology can be validated. * Correspondence to: Department of Geography, 675 Commonwealth Avenue, Boston University, Boston, MA 02215, USA. Tel.: ; fax: ; kaufmann@bu.edu Contract/grant sponsor: National Science Foundation; Contract/grant number: SBR CCC /99/ $17.50 Copyright 1999 Royal Meteorological Society

2 620 R.K. KAUFMANN ET AL. 2. IDENTIFYING SYNOPTIC PATTERNS Kirchhofer (1974) developed a technique that can be used to identify synoptic patterns from climate data over time and space. The Kirchhofer technique for analyzing lattice data standardizes observed values at each location based on normalization scores (Z ij ): Z i, j = (x i, j x j) (1) s j in which x i, j is the value of a climate variable at location i on day j, x is the mean value of the lattice on day j, and s is the S.D. of the lattice values on day j. The use of spatial averages and deviations assumes spatial stationarity. In this paper, maximum daily temperatures are analyzed. These values are not spatially stationary, therefore, the Kirchhofer technique is modified slightly. Rather than normalizing lattice values in relation to the spatial average and S.D., they are normalized relative to the temporal mean and deviation: Z i, j = (x i, j x i, j ) (2) s i, j in which Z i, j is the time normalized value at point i on day j, x i, j is the value at location i on day j, x i, j is the average for location i on day j, and s i, j is the S.D. for location i on day j. This modification prevents the Kirchhofer technique from defining the first-order north-to-south temperature trend as a synoptic pattern. The lattice of normalized values is compared with other lattices to calculate a summary Kirchhofer score. The score is used to classify daily synoptic patterns. Point-to-point comparisons of these scores determine the similarity between days. A similarity score is computed for each combination of days in the data set using a sum-of-squares procedure: N K A B = (Z ia Z ib ) 2 (3) i=1 in which K A B is the Kirchhofer score, Z ia is the normalized lattice value of the observation at location i on day A and Z ib is the normalized lattice value of the observation at location i on day B, and N is the number of points analyzed (Yarnal, 1984). In the classification procedure, daily lattices are considered similar relative to some threshold value for K A B. In addition to the overall score, the similarity of subareas of the lattice are checked to ensure that even if certain areas of the lattice agree on certain days, they will be considered dissimilar if the lattices are very different in other areas. Typically, row and column similarity scores (K r and K c ) are calculated in addition to a score for the entire lattice. If K r 1.9N r or K c 1.9N c for any row or column, in which N r and N c are the number of points in the rows and columns, respectively, the days are considered dissimilar. As stated previously, the location values in this analysis are normalized relative to temporal means and S.D.s. This avoids the error described by Blair (1998) in the calculation of row and column scores. The row and column thresholds used in this analysis are quite lenient and are based on values used in a previous study (Key and Crane, 1986). The row and column thresholds chosen correspond to a product moment correlation coefficient (r) of 0.05 using the derivation of Wilmott (1987). The threshold for the entire lattice is somewhat more strict. Yarnal (1985) considers days similar if the computed Kirchhofer score is less than 0.5N. Others use a threshold of 1.0N to group days (Moritz, 1979). Daily synoptic patterns are identified through key days. In an iterative procedure, the day in the sample that has the most other days similar to it is denoted the first key day. This lattice, together with lattices from similar days, is removed from the analysis and the process is repeated. This process continues until there no longer are any days in the analysis with enough similar days to constitute a new category. This cut-off, termed minimum group size, is chosen by the analyst. The number and size of groups identified by the Kirchhofer technique depends on the threshold and minimum group size (Key and Crane, 1986). There are no criteria to choose these parameters. As a result,

3 SYNOPTIC PATTERNS IDENTIFIED BY THE KIRCHHOFER TECHNIQUE 621 the analyst cannot be sure whether the synoptic patterns identified by the Kirchhofer technique represent groups of days that are generated by a meaningful meteorological phenomena or whether they emerge due to random chance. Random chance can play a significant role in defining groups given the large number of comparisons required by the procedure, which is approximately the number of observations squared. The maximum number of groups that the Kirchhofer technique can identify from a sample corresponds to the size of the sample. In theory, the Kirchhofer technique could identify a separate group for each day. Such a result would indicate that there are no reoccurring patterns each day is unique and constitutes a group of 1. If the historical record does contain synoptic patterns generated by some meaningful meteorological phenomena, the Kirchhofer technique will identify fewer groups than days in the sample and the number of days in the meaningful group(s) will be greater than 1. The number and size of groups is determined in part by the threshold and minimum group size. A large threshold will create a few groups with a large number of days per group. Group size is reduced by choosing a smaller threshold (fewer days will meet the criteria for similarity). Under these conditions, the technique will identify smaller groups but the number of groups created is increased. Regardless of the threshold, increasing the minimum group size reduces the number of groups identified. 3. ANALYSIS OF SUMMERTIME MAXIMUM TEMPERATURE The effect of threshold and minimum group size on the number of groups identified by the Kirchhofer technique is illustrated by analyzing the historical record for summertime maximum daily temperature in the mid-continental region of the US. Daily maximum temperature data for June, July and August from 16 NOAA ground weather stations are used in the analysis (Figure 1). These daily data are obtained from the National Climatic Data Center (NCDC, 1994). Equations (2) and (3) are used to analyze these data using combinations of threshold (K A B /N= , increment 0.25) and minimum group size ( , increment 10). These combinations affect the number of groups identified (Figure 2). Decreasing both the minimum group size and threshold increases the number of groups identified. Consistent with this effect, the maximum number of groups is identified at the back of Figure 2. Conversely, few groups are identified in the front corner of Figure 2, where the threshold size and minimum groups size is large. The surface in Figure 2 begs the question, are any of the groups identified by the Kirchhofer technique meaningful? That is, do the groups represent noise, in which case the Kirchhofer technique falsely indicates the presence of synoptic patterns, or do the groups represent synoptic patterns generated by physically meaningful meteorological phenomena? To answer this question, the authors use Monte Carlo techniques to generate a distribution that can be used to evaluate the statistical significance of the groups that are identified by various combinations of threshold and minimum group size. These critical values allow the determination of whether the number of groups identified by the Kirchhofer technique using a given combination of threshold and minimum group size is larger (in a statistically meaningful sense) than the number of groups identified from a set of randomly generated data using the same threshold and minimum group size. 4. A STATISTICAL DISTRIBUTION FOR THE KIRCHHOFER TECHNIQUE Monte Carlo techniques are employed to generate a three-dimensional surface that can be used to evaluate the statistical significance of the groups identified from the historical data. To do so, daily deviations for a 16-point lattice are simulated for a period similar to that of the historical analysis. For each location, 5796 observations are generated, which corresponds to the number of days in the June, July, August sample period. For each location, each observation represents the deviations from the mean daily maximum temperature for that location (Equation (2)). This normalization assures spatial stationarity.

4 622 R.K. KAUFMANN ET AL. The observations are generated by drawing randomly from a normal distribution with a mean value of zero and a S.D. of This S.D. is estimated from the historical data for one of the points in the lattice (Clarendon, TX, NCDCc =1761) on day 200, which is about the mid-point of the June, July, August period. The S.D. does not vary greatly among the points on day 200 and sensitivity analysis indicates that the value of the S.D. has relatively little effect on the number of groups identified by the Kirchhofer technique from the simulated data. This data generating process is repeated 100 times to generate 100 experimental datasets. Each experimental dataset represents deviations in maximum daily temperature as generated by a random process for a 16-point lattice. Each data set is analyzed with the Kirchhofer technique using the range of values for threshold and minimum group size that is used to analyze the historical data. The number of groups identified by each combination of threshold and minimum group size for each of the 100 experimental datasets is ranked in descending order. The value at position 5 corresponds to the 5% significance level for that combination of threshold and minimum group size. Fewer than 5% of the time will the Kirchhofer technique identify more groups than the value at position 5 from data that are generated randomly, and therefore contain no reoccurring patterns. Figure 1. The 16-point lattice used to analyze maximum summertime temperature in the US. The open circles represent the weather stations from which the data are obtained

5 SYNOPTIC PATTERNS IDENTIFIED BY THE KIRCHHOFER TECHNIQUE 623 Figure 2. The number of groups identified by various combinations of threshold and minimum group size from the historical data The values at position 5 are compiled to form the surface in Figure 3. This surface represents the critical values that can be used to evaluate the statistical significance of the groups identified by various combinations of threshold and minimum group size. Less than 5% of the time will the Kirchhofer method identify more groups from data with no meaningful groups than the critical value in Figure 3. Based on this meaning, any portion of the surface in Figure 2 that lies above the surface in Figure 3 identifies groups of days that represent statistically meaningful synoptic patterns. 5. ANALYSIS OF RESULTS Comparison of the surfaces in Figures 2 and 3 indicates that the ability of the Kirchhofer technique to identify statistically meaningful synoptic patterns depends on the threshold and minimum group size. The number of groups identified from the analysis of historical data exceeds the critical threshold in only parts of the surface (Figure 4). These areas represent combinations of threshold and minimum group size that identify statistically meaningful synoptic patterns. Using a large threshold and a small minimum group size, the Kirchhofer technique identifies fewer groups in the historical data than the critical values generated from the experimental data sets (Figure 4). Using a threshold of 0.50 and a minimum group size of 60, the Kirchhofer technique identifies 15 groups from the historical data. On the other hand, a value of 15 groups appears at position 45 of the 100 values generated from the experimental data sets using this combination of threshold and minimum group size. This result implies that 45% of the time, the Kirchhofer technique will identify 15 or more groups from

6 624 R.K. KAUFMANN ET AL. data that contain no synoptic patterns. Clearly, the 15 groups identified by a threshold of 0.5 and a minimum group size of 60 from the historical data are not statistically significant, nor are the groups identified by other combinations of threshold and minimum group size that do not exceed the critical values in Figure 3. To identify statistically meaningful synoptic patterns, we must use stringent criteria for threshold and minimum group size. By reducing the threshold size, the probability that a given day will be assigned to a particular group is also reduced. By increasing the minimum group size, the number of similarities required to constitute a group increases. By tightening the criteria for group membership, groups generated by physically meaningful meteorological phenomena (the signal) are separated from those generated by random chance (noise). Consistent with this effect, the number of groups identified from the historical data is greater than the critical threshold when combinations of small thresholds and large minimum group sizes are chosen (Figure 4). For example, using a threshold of 0.25 and a minimum group size of 50 identifies eight groups from the historical data. On the other hand, this combination identifies three or fewer groups in 95 of the 100 experimental data sets. This implies that at least five of the eight groups identified by a 0.25 threshold and a minimum group size of 50 are statistically meaningful. The presence of meaningful groups in the historical dataset can be inferred indirectly by the absence of groups relative to those generated by the analysis of the experimental data sets. The historical data contain many fewer small groups than contained in the experimental data. For example, the value at position 95 for a threshold of 0.25 and a minimum group size of 20 is 34, even at this strict threshold. This implies that 95% of the time, the Kirchhofer technique will identify 34 or more groups from data that Figure 3. The critical value (p 0.05) for various combinations of threshold and minimum group size generated from the analysis of the experimental data sets

7 SYNOPTIC PATTERNS IDENTIFIED BY THE KIRCHHOFER TECHNIQUE 625 Figure 4. The combinations of threshold and minimum group size that can be used to identify statistically meaningful synoptic pattern from the historical data contain no pattern using a threshold of 0.25 and a minimum group size of 20. The analysis of the historical data identifies only 27. Less than 5% of the time will the combination of 0.25 and 20 for threshold and minimum group size identify so few groups from data lacking synoptic patterns. The paucity of groups identified by this combination of threshold and minimum group size (and other combinations in this portion of the surface) may imply that these groups are missing. Because the number of days to be classified is finite, the absence of a large number of small groups may imply the presence of a non-random data generating process. This non-random process may generate patterns that cause the Kirchhofer technique to classify the days elsewhere. That elsewhere is the front left of the surface in Figure 2, where the threshold is low and the minimum group size is large. Consistent with the results described above, the front left position of the surface contains the combinations of threshold and minimum group size that identify more groups than found in the randomly generated data. The surface in Figure 4 can be used to define combinations of threshold and minimum group size that identify statistically meaningful groups. Nonetheless, the surface does not provide criteria for choosing among this subset of combinations. Thirteen groups can be defined if a threshold of 0.25 and a minimum group size of 40 are chosen. Alternatively, one group can be defined if a threshold of 0.75 and a minimum group size of 1200 are chosen. The set of criteria used to define the groups depends on how groups are to be used and the importance of parsimony to that process. The combination of 0.25 and 40 identifies 13 groups from the historical data. This number exceeds the critical value of 8. This critical value implies that some (8) of the 13 groups identified from the historical data may not represent physically meaningful meteorological phenomena. On the other hand, the combination of 0.75 and 1200 identifies only one group. The critical value for this combination is zero. This single group is highly unlikely from a statistical perspective, and therefore may represent a physically meaningful meteorological phenomenon. If parsimony is important, the analyst

8 626 R.K. KAUFMANN ET AL. may want to chose the group identified by the combination of 0.75 (threshold) and 1200 (minimum group size). 6. CONCLUSION This paper identifies a method by which analysts can evaluate the statistical significance of groups identified by the Kirchhofer method. This is critical because the technique identifies groups of days that emerge due to random chance and are therefore not truly related. As a result, previous analyses of climate data that used this technique probably overstated the number of synoptic patterns present. It is not possible to determine the degree to which the number of groups is overstated. The critical values derived for this analysis apply to this data set only. The critical value for a given threshold and minimum group size vary positively with the number of days in the sample period. The reliability (power) of the critical values is uncertain because they are not derived from a priori theory. Future research should focus on the reliability of critical values. This effort could proceed in two steps. In the first step, techniques based on the Markov random field could be used to estimate spatial interpolations from individual groups identified as being statistically significant by the Kirchhofer technique. Analysis of these groups should identify statistically meaningful spatial relations. In the second step, the spatial interpolations would be compared across groups. If the groups identified by the Kirchhofer technique using the critical values derived in Section 4 are statistically meaningful, the spatial relations estimated for individual groups should be statistically different from one another. If this procedure validates the power of the critical values, this research would provide a more rigorous method to evaluate the statistical significance of synoptic patterns, which could be quite useful in many areas of climate research. ACKNOWLEDGEMENTS The FORTRAN code used to calculate the Kirchhofer scores was provided by Jeff Key. This research was supported by a grant from the National Science Foundation SBR REFERENCES Barry, R.G., Crane, R.G., Schweiger, A. and Newell, J Arctic cloudiness in spring from satellite imagery, J. Climatol., 7, Blair, D The Kirchhofer technique of synoptic typing revisited, Int. J. Climatol., 18(14), Key, J. and Crane, R.G A comparison of synoptic classification schemes based on objective procedures, J. Climatol., 6, Kirchhofer, W Classification of European 500 mb Patterns, Arbeitsbericht der Schweiz. Meteorol. Zurich Zentral., 43. Moritz, R.E Synoptic Climatology of the Beaufort Sea Coast, Occasional Paper No. 30, Institute of Arctic and Alpine Research, University of Colorado, Boulder, CO, 176 pp. NCDC, Summary of the Day Cooperati e Network Weather Data, TD CD-ROM, EarthInfo, Inc, Boulder, CO. Wilmott, C.J Synoptic weather map classification: correlation versus sums-of-squares, The Prof. Geogr., 39(2), Yarnal, B Synoptic Climatology in En ironmental Analysis, Bellhaven, London. Yarnal, B A procedure for the classification of synoptic weather maps from gridded atmospheric pressure surface data, Comput. Geosci., 10(4), Yarnal, B A 500 mb synoptic climatology of Pacific northwest coast winters in relation to climate variability, to , J. Climatol., 5(7),