HAC robust trend comparisons among climate series with possible level shifts

Transcription

1 Research Article Received: 26 June 2013, Revised: 12 June 2014, Accepted: 13 June 2014, Published online in Wiley Online Library: 14 July 2014 (wileyonlinelibrary.com) DOI: /env.2294 HAC robust trend comparisons among climate series with possible level shifts Ross R. McKitrick a * and Timothy J. Vogelsang b Comparisons of trends across climatic data sets are complicated by the presence of serial correlation and possible step-changes in the mean. We build on heteroskedasticity and autocorrelation robust methods, specifically the Vogelsang Franses (VF) nonparametric testing approach, to allow for a step-change in the mean (level shift) at a known or unknown date. The VF method provides a powerful multivariate trend estimator robust to unknown serial correlation up to but not including unit roots. We show that the critical values change when the level shift occurs at a known or unknown date. We derive an asymptotic approximation that can be used to simulate critical values, and we outline a simple bootstrap procedure that generates valid critical values and p-values. Our application builds on the literature comparing simulated and observed trends in the tropical lower troposphere and mid-troposphere since The method identifies a shift in observations around 1977, coinciding with the Pacific Climate Shift. Allowing for a level shift causes apparently significant observed trends to become statistically insignificant. Model overestimation of warming is significant whether or not we account for a level shift, although null rejections are much stronger when the level shift is included. Keywords: autocorrelation; trend estimation; mean shift; HAC methods; climate models 1. INTRODUCTION Many empirical applications involve comparisons of linear trend magnitudes across different time series with autocorrelation and/or heteroskedasticity of unknown form. Vogelsang and Franses (2005, herein VF) derived a class of heteroskedasticity and autocorrelation robust (HAC) tests for this purpose. The VF statistic is similar in form to the familiar regression F-type statistics but remains valid under serial dependence up to but not including unit roots in the time series. For treatments of the theory behind HAC estimation and inference, see Andrews (1991), Kiefer and Vogelsang (2005), Newey and West (1987), Sun et al. (2008), and White and Domowitz (1984) among others. Like many HAC approaches, the VF approach is nonparametric with respect to the serial dependence structure and does not require a specific model of serial correlation or heteroskedasticity to be implemented. Unlike most nonparametric approaches, the VF approach avoids sensitivity to bandwidth selection by setting the bandwidth equal to the entire sample. Here, we extend the VF approach to the case in which one or more of the series has a possible level shift. Our assumption throughout is that a researcher considers a one-time level shift as a fundamentally different process than a continuous trend. Consequently, if the null hypothesis is that two series have the same trend and one series exhibits a trend while the other exhibits a level shift and no trend, a rejection of the null would be considered valid because the two phenomena are distinct and a prediction of one is not confirmed by observing the other. Accounting for level shifts does not necessarily increase the likelihood of rejecting a null of trend equivalence. In the top panel of Figure 1, a comparison of the linear trend coefficients would suggest they are similar, but clearly, y 1 differs from y 2 in that the former is steadily trending while the latter is trendless with a single discrete level shift at the break point T b. By contrast, in the bottom panel, a failure to account for the shift would overstate the difference between the trend slopes. In each case, the influence of the shift term is highlighted by the fact that if the trend slope comparisons were conducted over the preshift or post-shift intervals, they might indicate opposite results to those based on the entire sample (with the shift term omitted). The basic linear trend model is written as y it ¼ a i þ b i t þ u it (1) where i =1,,n denotes a particular time series and t =1,,T denotes the time period. The random part of y it is given by u it, which is assumed to be covariance stationary (in which case y it is labeled a trend stationary series, that is, stationary around a linear time trend, if one is present). For a series of length T, we parameterize the break point by denoting the fraction of the sample occurring before it as λ = T b /T. * Correspondence to: R. R. McKitrick, Department of Economics, University of Guelph, Guelph, N1G 2W1 Canada. ross.mckitrick@uoguelph.ca a Department of Economics, University of Guelph, Guelph, N1G 2W1, Canada b Department of Economics, Michigan State University, Lansing, MI, U.S.A. 528 This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.

2 HAC ROBUST TREND COMPARISONS WITH LEVEL SHIFTS Figure 1. Schematics of two series to be compared The following issues must be addressed in order to derive an HAC robust trend comparison test in the presence of a possible level shift. (i) If λ is known, and specifically is known to be in the (0, 1) interval, the VF test score can be generalized, as we show in Section 3, but the distribution is shown to depend on λ and the critical values change. It will turn out that the form of the VF statistic and its critical values are the same whether one is testing hypotheses involving the trend coefficients or other parameters in the trend function. (ii) If λ is unknown, it must be estimated along with the magnitude of the associated shift term. But this gives rise to a problem of nonidentification if we want to allow for the possibility that the true value of the level shift parameter is zero. The regression model with level shift takes the form y it ¼ a i þ g i DU t ðþþb λ i t þ u it (2) where the dummy variable DU t (λ)=0 if t λt and 1 otherwise (we will typically suppress the λ term where it is convenient to do so). Hence, for series i, estimation of (2) by ordinary least squares (OLS) yields an estimated intercept of ^a i up to T b and ^a i þ ^g i thereafter. In our empirical application, we are primarily interested in testing hypotheses about the trend slope parameters, b i, while controlling for the possibility of a level shift. If it is reasonable to view λ as known, then inference about the trend slopes will proceed in a straightforward way with DU t (λ) included in the model even in the case where the true value of g i is zero. However, if it is more reasonable to treat λ as unknown and we want to be robust to the possibility that g i is nonzero, then inference about the trend slopes (b i ) becomes more delicate because λ is not identified when g i is zero. We would face a similar identification problem if we wanted to test the null hypothesis that g i itself is zero and λ is unknown. There is now a well-established literature in statistics and econometrics for carrying out inference where a parameter is not identified under the null hypothesis but is identified under the alternative hypothesis. See for example Davies (1987), Andrews (1993), Andrews and Ploberger (1994), and Hansen (1996) among others. One solution to this identification problem involves the use of a supremum function, which is akin to a data-mining approach. In the present case, we can compute the VF statistic for equality of trends for a range of λ allowed to vary across (0, 1) and find the largest VF statistic, the sup-vf statistic. To be robust to the possibility that there are no level shifts in the data, that is, to be robust to the critique that the date of the level shift was chosen to data-mine an outcome for the equality of trends test, we work out the null distribution of the sup-vf statistic for equality of trends for the case where g i = 0. This yields a trend equality test that is very robust to the possibility and location of potential level shifts. Although our focus is on the problem of trend inference allowing for possible unknown level shifts, our extension of the VF approach is general enough to include tests of the null hypothesis of no level shift, and we report some limited results in the paper for these tests. A potential application of tests for a level shift is the homogenization of weather data. Many long observational records are believed to have been affected by possible equipment and/or sampling changes, changes to the area around monitoring locations, and so forth (see Hansen et al., 1999; Brohan et al., 2006 for examples in the land record; Folland and Parker, 1995; Thompson et al., 2008 for examples in sea surface data). A typical method for detecting and removing level shifts is to construct a reference series that is not expected to exhibit the discontinuity, such as the mean of other weather station records in the vicinity, and then look for one or more jumps in a record relative to its reference series. While the application of the VF approach to testing for a level shift is potentially quite useful in many empirical settings, the problem of testing for a level shift in a trending series with a known or unknown shift date has already received some attention in the econometrics 529

3 R. R. MCKITRICK AND T. J. VOGELSANG literature (Vogelsang (1997) and Sayginsoy and Vogelsang (2011)) and the empirical climate literature (see Gallagher et al. (2013) and references therein). Each proposed method has inherent strengths and weaknesses. A complete comparison of the VF approach to existing tests for a level shift would be a substantial undertaking and is beyond the scope of this paper, but we draw some contrasts in Sections 4 and 5. The question of whether or not a level shift is present in trending data can strongly affect the resulting trend calculations and tests of equality of trend slopes. If a change point λ is known, the analysis in Section 3 applies, and if a change point is suspected but the date is unknown, the analysis in Section 4 applies. In our application, we focus on the case where there is at most one level shift in each series. In other applications, such as those involving very long weather series, one might suspect there are multiple shifts. If they occur at known dates, then our extension of the VF approach is general enough to apply. However, should it be more reasonable to model the shift dates as unknown or should there be uncertainty regarding the number of shifts, this greatly complicates the analysis, especially from a computational perspective, and is beyond the scope of this paper. In addition, if one thinks level shifts occur frequently and with randomness, then there is the additional difficulty that the range of possible specifications could, in principle, include the case in which the level changes by a random amount at each time, which is equivalent to having a random walk, or unit root component in u it.ify it has a unit root component, inference in models (1) and (2) becomes more complicated. More importantly, it is difficult to give a physical interpretation to a unit root component of a temperature series. See Mills (2010) for a discussion of temperature trend estimation when a random walk is a possible element of the specification. In our application, we think it is reasonable to assume that the observed series are well characterized by a trend and at most one level shift at a known date and that the errors are covariance stationary. We focus on the prediction of climate warming in the troposphere over the tropics. As shown in Section 6, climate models predict a steady warming trend in this region because of rising atmospheric greenhouse gas levels, but none predict a step-change, so trends and shifts can be regarded as distinct phenomena. A number of studies (summarized later) have shown that models likely overstate the warming trend, but there is disagreement as to whether the bias is statistically significant. McKitrick et al. (2010) used the original VF test to examine this issue over the interval, coinciding with the record available from weather satellites. We extend their analysis to the interval using data from weather balloons. This long span encompasses a date at which a known climatic event caused a level shift in many observed temperature series. If the shift is nontrivial in magnitude, the comparison would thus be akin to that in Figure 1, such that failure to take it into account could bias the comparison either toward overstating or understating the difference in trend slopes. The event in question occurred around 1978 and is called the Pacific climate shift (PCS). This manifested itself as an oceanic circulatory system change during which basin-wide wind stress and sea surface temperature anomaly patterns reversed, causing an abrupt step-like change (level shift) in many weather observations, including in the troposphere, as well as in other indicators such as fisheries catch records (see Seidel and Lanzante, 2004; Tsonis et al., 2007; Powell and Xu, 2011, and extensive references therein). For our purposes, we do not try to present a specific physical explanation of the PCS or even evidence which its origin was exogenous to the climate system, only that it was a large event at an approximately known date, the existence of that has been documented and studied extensively and that resulted in a shift in the mean of the temperature data. We first present results based on assumption that the PCS occurred at a known date (Section 6.2) and then based on the assumption that the PCS is not known to have occurred or that the date of occurrence is unknown (Section 6.3). We find, in some cases, that the shift term is significant at the 5% or 10% level, confirming the overall importance of controlling for this possibility when comparing trends. If the date of the PCS is taken as given and exogenous, we find that the models project significantly more warming in both the lower troposphere and mid-troposphere than are found in weather balloon records over the interval. This finding remains robust if we treat the date of the PCS as unknown and apply the conservative data-mining approach. In fact, this finding is robust whether or not we include a level shift in the regression model: we reject equivalence of the trend slopes between the observed and model-generated temperature series either way. The evidence against equivalence is simply stronger when we control for a level shift and this is true whether we treat the date of the shift to be known or unknown. We also find that if the date of the PCS is assumed to be known then (a) the appearance of positive and significant trend slopes in the individual observed temperature series vanishes once we control for the effect of the level shift and (b) we find statistical evidence for a level shift in some but not all observed temperature series. If the date of the PCS is assumed to be unknown, statistical evidence remains for a level shift in the mean of the observed mid-troposphere series but is weak in the lower troposphere series. This is not surprising given that we use the data-mining robust critical value that decreases the power of detecting such a shift. 2. BASIC SET-UP WITH NO SHIFT OR KNOWN SHIFT DATE Trend models The literature on estimation and inference in model (1) is by now well established, and it may hardly seem possible that there is something new to be said on the subject. In fact, the last decade or so has seen some very useful methodological innovations for the purpose of computing robust confidence intervals, trend significance, and trend comparisons in the presence of autocorrelation of unknown form. Many of these robust estimators use the nonparametric HAC approach that is now widely used in econometrics and empirical finance literatures. In contrast, the nonparametric HAC approach is used less in applied climatic or geophysical papers although nonparametric approaches have been proposed by Bloomfield and Nychka (1992) and further examined by Woodward and Gray (1993) and Fomby and Vogelsang (2002) for the univariate case. As far as we know, McKitrick et al. (2010) is the first empirical climate paper to apply nonparametric HAC methods in multivariate settings.

4 HAC ROBUST TREND COMPARISONS WITH LEVEL SHIFTS It will be convenient to define a general deterministic trend model that contains (1) and (2) as special cases: y it ¼ β i d 0t þ δ i d 1t þ u it (3) where d 0t is a single deterministic regressor (typically the time trend in our applications) and d 1t is a k 1 vector of additional deterministic regressors (typically the intercept and shift terms) and δ i is the corresponding k 1 vector of parameters. Model (1) is thus obtained for d 0t = t, β i = b i, and d 1t =1,δ i = a i, and model (2) is obtained for d 0t = t, β i = b i, and d 1t = (1, DU t ), δ i =(a i, g i ). Notice that we are assuming that each time series y it has the same deterministic regressors. This is needed for the VF statistic to be robust to unknown conditional heteroskedasticity and serial correlation. In some applications, it might be reasonable to model some of the series as having different trend functions. For example, we know that the climate model series in the application do not have level shifts because level shifts are not part of the climate model structures. When we think series could have different functional forms for the trend, we can simply include in d 1t the union of trend regressors across all the series. While this will result in a loss of degrees of freedom, in many applications, the regressors will be similar across series, so the loss in degrees of freedom will often be small. We view this loss of degrees of freedom as a small price to pay for robustness to unknown forms of conditional heteroskedasticity and autocorrelation. We estimate model (3) using OLS equation by equation. OLS has some nice properties in our set up. Because the regressors are the same for each equation, we have the well-known exact equivalence between OLS and generalized least squares estimators that account for cross series correlation. Because we have covariance stationary errors, the well-known Grenander and Rosenblatt (1957) result applies in which case OLS is also asymptotically equivalent to generalized least squares estimators that account for serial dependence in the data. Defining the n 1 vector β =(β 1, β 2,, β n ) and the k n matrix δ =(δ 1, δ 2,, δ n ), model (3) can be written in vector notation as y t ¼ βd 0t þ δ d 1t þ u t (4) The parameters of interest are in the vector β, so it is convenient to express the OLS estimator using the partialling out result for linear regression, also known as the Frisch Waugh Lovell result (Davidson and MacKinnon, 2004 and Wooldridge, 2005) as follows. Let ed 0t and ey t denote respectively the OLS residuals from the regression of d 0t on d 1t and the regression of y t on d 1t. The OLS estimator of β can be written as ^β ¼ T t¼1 ed 2 0t 1 T ed 0t ey t (5) t¼1 and it directly follows that ^β β ¼ T t¼1 ed 2 0t 1 T ed 0t u t (6) t¼1 Note that the form of ^β in Equation (5) would be unchanged if we redefined d 0t to be the shift term and d 1t to be the intercept and trend terms; however, the definition of the ~ variables would be adjusted accordingly. This implies that the test statistic on hypotheses about the shift term will take the same form as those for the trend terms when the shift date is known The VF test We are interested in testing null hypotheses of the form H 0 : Rβ ¼ r (7) against alternatives H 0 : Rβ r where R and r are known restriction matrices of dimension q n and q 1 respectively where q denotes the number of restrictions being tested. The matrix R is assumed to have full row rank. Robust tests of H 0 need to account for correlation across time, correlation across series, and conditional heteroskedasticity as summarized by the long run variance of u t. This is defined as Ω ¼ Γ 0 þ 1 j¼1 Γ j þ Γ j where Γ j ¼ E u t u t j is the matrix autocovariance function of u t. Those familiar with the time series literature will notice that Ω is proportional to the spectral density matrix of u t evaluated at frequency zero. The VF statistic is constructed using the following estimator of Ω: ^ Ω T ¼ Γ^0 þ T 1 1 j ^j Γ þ Γ^j T j¼1 (8) where ^Γ j ¼ T 1 T t¼jþ1 ^u t^u t j. This is the Bartlett kernel nonparametric estimator of Ω using a bandwidth (truncation lag) equal to the sample size. The VF statistic for testing H 0 : Rβ = r is given by 531

5 R. R. MCKITRICK AND T. J. VOGELSANG " # VF ¼ R^β r T 1 1 RΩ^ TR R^β r ed 2 0t t¼1 =q (9) In the Supporting information, we provide a finite sample motivation for the form of ^Ω T. Also note that (8) was originally proposed by Kiefer et al. (2000, 2001) although in the different but computationally identical form: T 1 ^Ω T ¼ 2T 2 ^S t^s t (10) where ^S t ¼ t ^u j. See Kiefer and Vogelsang (2002) for a formal derivation of the exact equivalence between (8) and (10). j¼1 When only one restriction is being tested (q = 1), we can define a t-statistic version of VF as t¼1 R^β r VF t ¼ r ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi T e t¼1d 2 1RΩ^ 0t TR 2.3. Asymptotic limit of the VF statistic We now provide sufficient conditions for obtaining an asymptotic approximation to the sampling distribution of the VF statistic. A formal proof is given in the Supporting information. The fraction c (01] of the sample is ct, and we denote the integer portion of this quantity as [ct]. The symbols and d denote weak convergence and convergence in distribution, Λ is the matrix square root of Ω (i.e. Ω = ΛΛ ) and W j (c) denotes a j 1 vector of independent standard Wiener processes where j is a positive integer. Two assumptions are sufficient for obtaining the limit of VF. Define the partial sums of u t as S t ¼ t u t. The first assumption is that a functional central limit theorem holds for S t.ast 1, T 1=2 S ½cTŠ ΛW n ðþ c (11) j¼1 The second assumption is related to the deterministic regressors in the model. Assume that there is a scalar τ 0T and a k k matrix τ 1T, such that ½cTŠ T 1 τ 0T d 0t f 0 ðþds s and T 1 τ 1T d 1t c 0 f 1ðÞds s (12) t¼1 c 0 For example, in model (2), d 0t = t, d 1t = (1, DU t ), τ 0T = T, τ 1T ¼ 1 0, f 0 (s)=s, and f 1 (s) = (1, DU(λ > s)) where, in this case, DU 0 1 denotes a continuous indicator function taking the value 1 if λ > s and 0 otherwise. Stack f 0 ðþand s f 1 ðþinto s a vector fðþ s ¼ f 0 ðþ; s f 1 ðþ s and define the stochastic process ½cTŠ t¼1 Bq f c ðþ¼ c dw q ðþ s 0! 1 dw q ðþf s ðþ s 0 1 fðþf s s 0 1 ðþ ds! c 0 fðþds s (13) In the Supporting information, we show that under assumptions (11) and (12), the limit of VF under the null hypothesis (7) is given by " # 1 VF d Z 1 q 2 Bq f ðþb c q f ðþ c dc Z q =q VF 1 q (14) where Z q ~ N(0, I q ) and is independent of the random matrix 2 Bq f ðþb c q f ðþ c dc: The limit of the VF statistic can therefore be seen to be similar 0 to an F random variable; however, it follows a nonstandard distribution that depends on the deterministic regressors in the model via the stochastic process Bq f ðþ. c The critical values of VF1 q thus depend on the regressors in d it and by extension depend on the value of λ when a level shift dummy variable is included in the model. It is important to note that the critical values do not depend on which regressors are placed in d 0t (the regressor of interest for hypothesis testing). For a given value of λ, one uses the same critical values for testing the equality of trend slopes or testing hypotheses about the intercepts or testing hypotheses about level shifts in model (2).

6 HAC ROBUST TREND COMPARISONS WITH LEVEL SHIFTS Obtaining the critical values of the nonstandard asymptotic random variable defined by (14) is straightforward using Monte Carlo simulation methods that are widely used in the econometrics and statistics literatures. In the application, when we take the date of the PCS to be exogenously given at 1977:12, this yields a value of λ = For model (2) with λ = , we simulated the asymptotic critical values of VF for testing one restriction (q = 1) that we tabulate in Table 1. The Wiener process that appears in the limiting distribution is approximated by the scaled partial sums of 1000 independent and identically distributed N(0, 1) random deviates. The vector f(s) is approximated using (1, DU(t > T), t/t) for t = 1,2,,T. The integrals are approximated by simple averages; 50,000 replications were used. We see from Table 1 that the right tail of the VF statistic is fatter than that of a χ 2 1 random variable Bootstrap critical values and p-values If carrying out simulations of the asymptotic distributions is not easily accomplished using standard statistical packages, an alternative is to use a simple bootstrap that is described in detail in the Supporting information. Residuals from Equation (4) can be resampled and used to compute Ω^ from Equation (8) and VF from Equation (9); then the percentiles of the bootstrapped VF statistic in many repetitions can provide critical values and p-values. A particular advantage of the VF method is that its asymptotic null distributions do not depend on unknown correlation parameters, and it falls within the general framework considered by Gonçalves and Vogelsang (2011), where it was shown that the simple, or naïve, independent and identically distributed bootstrap will generate valid critical values. No special methods, such as blocking, are required here, and the bootstrap critical values are asymptotically equivalent to the distribution given by (14). 3. TREATING THE SHIFT DATE AS UNKNOWN Many previous authors (e.g. Seidel and Lanzante, 2004) have treated the date of the level shift as known because the PCS was an exogenous event observed across many different climatic data series. As a robustness check, we also report results where we treat the date of the level shift as unknown. We take a data-mining approach that has a long history in the change point literature. For a given hypothesis, we compute the VF statistic for a grid of possible shift dates and determine the one that gives the largest VF statistic. In other words, we search for the shift date that gives the strongest evidence against the null hypothesis. The effect on critical values of searching over shift dates must be taken into account, otherwise this approach would be a data-mining exercise that could give potentially misleading inference. The level of the test will be inflated above the nominal level compared with the case where the shift date is assumed to be known. Fortunately, it is easy to obtain critical values that take into account the search over shift dates. For a given potential shift date T b, let VF(λ) denote the VF statistic for testing a given null hypothesis. The limiting random variable given by (14) depends on λ through the level shift regressor, and we now label the limit by VF 1 q ðþto λ make explicit the dependence on the shift date used to estimate the model. For technical reasons (Andrews (1993)), we need to trim the fraction v from each end of the sample, leaving a grid of potential shift dates given by vt +1,vT +2,, T vt (in our application, we set v = 0.1). Define the data-mined VF statistic as supvf ¼ sup VFðÞfor λ λ ðv; 1 vþ λ Under the null hypothesis (7) and under the assumption there is no level shift in the data, we have supvf d sup VF 1 q λ ðv;1 vþ ðþ λ (15) where the limit follows from (14) and application of the continuous mapping theorem. Using simulation methods identical to those used for the known shift date case, we computed asymptotic critical values for supvf for v = 0.1 and q = 1 for testing hypotheses about the trend slope Table 1. Asymptotic critical values: model (2), known shift date with λ = , q =1 % VF t VF The value in the first column shows the percentage in the upper (right) tail exceeding the indicated values of VF t, and VF. Left tail critical values of VF t follow by symmetry around zero. VF, Vogelsang Franses. 533

7 R. R. MCKITRICK AND T. J. VOGELSANG parameters in model (2). These critical values are given in Table 2. Using the supvf statistic along with the critical values given by (15) provides a very conservative test with regard to the shift date. 4. TESTING FOR A LEVEL SHIFT IN A UNIVARIATE TIME SERIES As part of the empirical application, we provide visual evidence that the observed temperature series exhibit level shifts around the time of the PCS. Some formal statistical evidence regarding these level shifts can be provided by application of the VF statistic to an individual time series. Consider model (2) for the case of n = 1, and place the model in the general framework (3) with d 0t = DU t, d 1t = (1, t), β 1 = g 1, and δ 1 =(a 1, b 1 ). If we take the shift date as known, then the VF statistic for testing for no level shift (H 0 : g 1 = 0) can be computed as before using (9) with R = 1 and r = 0. The asymptotic null critical values are still given by Table 1. If we treat the shift date as unknown, we can apply the supvf statistic although the asymptotic critical values depend on which regressor is placed in d 0t. While it is true that for a given value of λ, the distribution of VF 1 q ðþis λ the same regardless of the regressor placed in d 0t, the covariance structure of VF 1 q ðþacross λ λ depends on which regressor is placed in d 0t. Therefore, the supvf statistic when testing for a zero trend slope has different asymptotic critical values than the supvf statistic for testing a zero level shift. We simulated the asymptotic critical values of supvf for testing for a zero level shift for the case of v = 0.1 and q = 1 and provide those critical values in Table 2. Other tests for a level shift at an unknown date of a trending time series have been proposed in the empirical climate literature. Reeves et al. (2007) provide a review of change point detection methods developed in the climate literature, but the review focuses on tests designed for time series variables that do not have serial correlation. In contrast Lund et al. (2007) propose a test for a level shift that allows a specific form of autocorrelation the first order periodic autoregressive model. We prefer the VF approach for two reasons. First, the VF approach is robust to more general forms of autocorrelation. Second, we formally derive and characterize the limiting null distribution of the sup statistic, and this allows us to tabulate null critical values. Lund et al. (2007) also use a sup-type statistic, but they do not provide any asymptotic theory that can be used to generate valid approximate critical values. A recent paper by Gallagher et al. (2013) develops asymptotic theory for a level shift test that treats the shift date as unknown but their analysis is confined to trend models where u it is assumed to be uncorrelated over time. What seems to be missing from the empirical climate literature are level shift tests that allow the shift date to be unknown and permit serial correlation in u it. Fortunately, there are several papers in the econometrics literature that propose level shift tests for trending series that have these properties, see Ploberger and Krämer (1996), Vogelsang (1997), and Sayginsoy and Vogelsang (2011). While clearly well outside the scope of this paper, it would be interesting to compare the sup-vf test for a shift in trend at unknown date with the other tests proposed in the literature. 5. FINITE SAMPLE PERFORMANCE OF THE VF STATISTIC In this section, we report some results from a small Monte Carlo simulation study that demonstrates the finite sample performance of the VF statistic. We compare the performance of the VF statistic with a traditional Wald statistic. The Wald statistic is configured to be robust to heteroskedasticity and serial correlation over time and to be robust to correlation across series. We use two established methods for constructing the Wald statistic. The first method is based on the parametric estimator of Ω given by ^ Ω M ¼ Γ^0 þ M 1 1 j ^j Γ þ Γ^j M Þ j¼1 which is the Bartlett kernel estimator. The value of the bandwidth, M, was chosen by the data dependent method proposed by Andrews (1991) based on the AR(1) plug-in method. This data dependent method tends to choose values of M that are small relative to T although (16) Table 2. Asymptotic critical values: model (2), unknown shift date, q = 1; 10% trimming (λ* = 0.1) % supvf trend slope supvf level shift The value in the first column shows the percentage in the upper (right) tail exceeding the indicated values of the supvf statistics for, respectively, the trend slope and level shift coefficients. VF, Vogelsang Franses.

8 HAC ROBUST TREND COMPARISONS WITH LEVEL SHIFTS M tends to be larger when serial correlation in the data is strong compared with cases where serial correlation is weak. The second method also uses (16) but with prewhitening based on autoregression models with lag 1 (AR(1)) fit to the components of ^u t. Prewhitening was explored by Andrews and Monahan (1992). We set M = 1 in the prewhitening case that makes the estimator of Ω an AR(1) parametric estimator. We compute Wald-type statistics based on these two estimators of Ω using (9) with ^Ω T replaced with either ^Ω M or the prewhitened estimator. The resulting Wald statistics are denoted by W Bart and W PW, respectively. Under the assumptions used to derive the limiting null distribution of the VF statistic, it is well known that W Bart and W PW have limiting null distributions given by χ 2 q =q where χ2 q denotes a chi-square random variable with q degrees of freedom. We use the following data generating process: y 1t ¼ g 1 DU t ðþþb λ 1 t þ u 1t (17) y 2t ¼ b 2 t þ u 2t (18) where u 1t ¼ 1 κ u 1t, u 2t ¼ pffiffiffiffiffiffiffi 1 1 1þη 2 κ u 2t þ ηu 1t, u it ¼ ρ 1 u i;t 1 þ ρ 2u i;t 2 þ ε it, ε it ~ iidn(0, 1), cov(ε 1t, ε 2s ) = 0, and u i1 ¼ u i0 ¼ 0, κ ¼ rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 ρ 2. The errors, u ð1 ρ 2 Þ 2 ρ 2 ½ 1 Šð1þρ 2 Þ it, are configured to have unit variances with covðu 1t ; u 2t Þ ¼ p ffiffiffiffiffiffiffi η. When η =0,y 1t and y 2t are uncorrelated 1þη 2 with each other. We report two sets of results. The first set of results focuses on empirical null rejection probabilities. For y 1t, we set b 1 = 0.01 and g 1 =0so that there is no level shift in y 1t. For y 2t, we set b 2 = 0.01 so that the null hypothesis of equal trend slopes holds. We set η = 0 so that the two series are uncorrelated. We report results for T = 120, 240, and 636, and a selection of values of ρ 1 = 0 and g 1 = In all cases, 50,000 replications were used, and we computed empirical rejection probabilities for the VF, ρ 1 = 0.9, and VF statistics for testing g 1 = 0 using the appropriate asymptotic critical values. The simulation results for this configuration highlight the impact of serial correlation structure on null rejection probabilities relative to the sample size. The results are tabulated in Table 3. There are two sets of results reported for each of the three statistics. The first set of results corresponds to the case where no level shift dummy variable is included in the estimated model. The second set of results corresponds to the case where the level shift dummy is included in the estimated model. In this case, we also report results for the supvf statistic. Results are organized into three blocks corresponding to the three sample sizes. Within a block, results are given for seven configurations of the autoregressive parameters ranging from no serial correlation to strong serial correlation. If the asymptotic approximations were working perfectly for the statistics, we would see rejections of 0.05 in all cases. When the serial correlation is absent, all statistics have empirical rejection probabilities close to 0.05 regardless of the sample size. Once there is serial correlation in the model, over-rejections can occur depending on the strength of the serial correlation relative to the sample size. First focus Table 3. Empirical null rejections with AR(2) errors Without level shift With level shift T ρ 1 ρ 2 W PW W Bart VF W PW W Bart VF SupVF H 0 :b 1 =b 2, 5% nominal level, b 1 = b 2 =.01, η =0, g 1 = 0. The data generating process is given by (17) and (18). 535

9 R. R. MCKITRICK AND T. J. VOGELSANG on the case of AR(1) errors (VF). Rejections tend to be close to 0.05 when W Bart is small, but as W PW increases in value, rejections tend to increase. This is especially true for the W Bart statistic where rejections exceed 0.25 when ρ 1 = 0.9 In contrast, W pw and VF suffer from less severe over-rejection problems although they tend to be over-sized when T = 120 and ρ 1 = 0.9 For a given value of ρ 1, as T increases, overrejections tend to fall for all three statistics but slowest for W Bart. Overall, for the AR(1) error case, W pw and VF have similar rejections to each other and outperform W Bart. The supvf statistic tends to over-reject more than VF when serial correlation is strong although the differences between supvf and VF decrease as the sample size increases. It is a common finding that supremum statistics tend to have more over-rejection problems than statistics that treat break dates as known. One of the reasons that W pw performs relatively well with AR(1) errors is that W pw is explicitly designed for AR(1) error structures. But, when the errors are not AR(1), W pw can suffer from over-rejection and under-rejection problems. Consider the case ρ 1 = 0.3, ρ 2 = 0.3 where W pw shows substantial over-rejections that are larger than W Bart and VF. These over-rejections tend to persist as T increases. In contrast, VF is much less distorted and rejections approach 0.05 as T increases. For the case of ρ 1 = 0.9, ρ 2 = 0.3, W pw under-rejects, and the under-rejection problem becomes more severe as T increases, whereas VF has rejections close to 0.05 for all sample sizes. The W Bart statistic tends to overreject mildly in this case. Table 4. Empirical null rejections and empirical power with AR(1) errors 536 Without level shift With level shift η g 1 ρ 1 b 2 W PW W Bart VF W PW W Bart VF SupVF H 0 :b 1 =b 2, T = 636, 5% nominal level, b 1 =.01, ρ 2 = 0. The data generating process is given by (17) and (18).

10 HAC ROBUST TREND COMPARISONS WITH LEVEL SHIFTS In general, Table 3 indicates that the VF statistic has the least over-rejection problems and is the better statistic with regard to control of type 1 error. In the second set of results, we use T = 660 to match the empirical application. We now include a level shift in y 1t with λ = and we set b 1 = 0,0.01 and g 1 = For y 2t we set b 2 = 0.01, , 0.011, , and We report results for η = 0,0.5. While we ran simulations for a wide range of values for ρ 1 and ρ 2, we only report results for ρ 1 = 0,0.9 and ρ 2 = 0 given that results for other serial correlation configurations have similar patterns to what is reported in Table 3. The results are given in Table 4. The first block of 20 rows gives results for η = 0, whereas the second block of 20 rows gives results for η = 0.5. Within each η block, results are first given for g 1 = 0 followed by results for g 1 = For each value of g 1, results are given for ρ =0 followed by results for ρ = 0.9. When b 2 = 0.01, we are observing null rejection probabilities, whereas for other values of b 2, we are observing power. First focus on the results when the null hypothesis is true, that is, b 2 = For ρ 1 = 0, we see that when the level shift dummy is included, we have rejections close to 0.05 for all statistics. However, when the level shift dummy is not included and g 1 = 0.25, we observe severe over-rejections that range from to The statistic with the least severe over-rejection problem is VF. When ρ 1 = 0.9, we have relatively strong autocorrelation in the data. When either g 1 = 0 or the level shift dummy is included in the model, there are some mild over-rejection problems ranging from for VF to for W Bart with W pw in between. Over-rejections are slightly worse when η = 0.5 compared with η = 0. As we see in Table 3, supvf tends to over-reject slightly more than VF when autocorrelation is strong. In addition, when there is a level shift in the data, supvf tends to have rejections above This happens because supvf nests the null hypotheses of equal trend slopes and no level shift. A rejection using supvf indicates a level shift and/or differences in trends slopes. Table 5. Summary of lower troposphere data series Data series Model/obs name extra forcings; no. runs Trend ( C/decade) Simple trend 95% CI ± width Trend ( C/decade) Trend + level shift 95% CI ± width Level shift ( C/decade) 95% CI ± width 1 BCCR BCM2.0 O; CCCMA3.1-T47 NA; CCCMA3.1-T63 NA; CNRM3.0O; CSIRO CSIRO GFDL2.0 O, LU, SO, V; GFDL2.1 O, LU, SO, V; GISS_AOM GISS_EHO, LU, SO, V; GISS_ER O, LU, SO, V; IAP_FGOALS ECHAM INMCM3.0 SO, V; IPSL_CM MIROC3.2_T106 O, LU, SO, V; MIROC3.2_T42 O, LU, SO, V; MPI2.3.2a SO, V; ECHAM5 O; CCSM3.0 O, SO, V; PCM_B06.57 O, SO, V; HADCM3 O; HADGEM1 O, LU, SO, V; HadAT RICH RAOBCORE Notes: Each row refers to model ensemble mean (rows 1 23) or observational series (rows 24 26). All models forced with 20th century greenhouse gases and direct sulfate effects. Rows 10, 11, 19, 22, and 23 also include indirect sulfate effects. Extra forcing indicates which models included other forcings: ozone depletion (O), solar changes (SO), land use (LU), and volcanic eruptions (V). NA: information not supplied to Program for Climate Model Diagnosis and Intercomparison (PCMDI). No. runs: indicates number of individual realizations in the ensemble mean. Trend slopes estimated using OLS, 95% CI is trend ± number shown, which is computed using VF method (Section 4). For instance, the RAOBCORE simple trend (bottom row first entry) is ± C/decade. RAOBCORE, Radiosonde Observation Bias Correction using Reanalyses; RICH, Radiosonde Innovation Composite Homogenization. 537

11 R. R. MCKITRICK AND T. J. VOGELSANG Now focus on the cases where b 2 > In these cases, y 2t has a bigger trend slope than y 2t, and we should be rejecting the null of equal trend slopes. When g 1 = 0, we see that all statistics have good power when ρ 1 = 0 and power is higher for η = 0.5 compared with η = 0. Power increases as expected as b 2 increases. Across the three statistics, VF tends to have lower power than other two statistics. This illustrates the well known trade-off between over-rejection problems and power. Note that while power of VF is lowest, its power is still relatively good in an absolute sense. If we include the level shift dummy in the model even though it is not needed (g 1 = 0), all three tests show a reduction in power as one would expect. An unexpected finding is that the supvf statistic has higher power than VF and the two Wald statistics when the level shift regressor is included but there is no shift in the data. In contrast and as expected, power of supvf is lower than the tests for the case where the level shift regressor is not included in the estimated model. The most interesting power results occur for g 1 = 0.25 and b 2 = when the level shift dummy is left out of the model. In this case, the estimator of b 1 is biased up, and one can show that the probability limit of the estimator of b 1 exactly equals For the case of ρ 1 = 0, rejections of all three statistics are close to the nominal level of This shows that an omitted level shift variable can cripple the power of the tests to detect a difference in trend slopes between two series. For larger values of b 2, the tests have power even if the level shift variable is not included. When b 2 = 0.011, power is higher if the level shift dummy is included, whereas for b 2 = 0.012, power is higher when the level shift dummy is left out. When a level shift is present in the data, supvf has less power overall than VF as expected given that supvf treats the break date as unknown and uses conservative critical values. These simulation results show that (i) the VF statistic has type 1 errors closest to the nominal level, (ii) the VF statistic has lower power which is the price paid for more accurate type 1 error; however the power of VF is still reasonably good, (iii) including a level shift dummy when there is no level shift in the data lowers power, (iv) failure to include a level shift dummy when there is a level shift in the data causes type 1 errors to be excessively larger than the nominal level and, depending on the magnitude/direction of the level shift, can make it difficult to detect slopes that are different, (v) positive correlation across series (η = 0.5) tends to increase power, and (vi) stronger serial correlation tends to inflate over-rejections under the null while reducing power. Table 6. Summary of mid-troposphere data series 538 Data series Model/obs name extra forcings; no. runs Simple trend Trend ( C/decade) 95% CI ± width Trend ( C/decade) Trend + level shift 95% CI ± width Level Shift ( C/decade) 95% CI ± width 1 BCCR BCM2.0 O; CCCMA3.1-T47 NA; CCCMA3.1-T63 NA; CNRM3.0 O; CSIRO CSIRO GFDL2.0 O, LU, SO, V; GFDL2.1 O, LU, SO, V; GISS_AOM GISS_EH O, LU, SO, V; GISS_ER O, LU, SO, V; IAP_FGOALS ECHAM INMCM3.0 SO, V; IPSL_CM4; MIROC3.2_T106 O, LU, SO, V; MIROC3.2_T42 O, LU, SO, V; MPI2.3.2a SO, V; ECHAM5 O; CCSM3.0 O, SO, V; PCM_B06.57 O, SO, V; HADCM3 O; HADGEM1 O, LU, SO, V; HadAT RICH RAOBCORE Notes same as for Table 5. Trend slopes estimated using OLS, 95% CI is trend ± number shown, which is computed using VF method (Section 4). For instance, the RAOBCORE simple trend (bottom row first entry) is ± C/decade. RAOBCORE, Radiosonde Observation Bias Correction using Reanalyses; RICH, Radiosonde Innovation Composite Homogenization.