INTRODUCTION: WHAT IS A TREND?

Transcription

1 INTRODUCTION: WHAT IS A TREND? The question of defining a trend is one which has exercised the minds of statisticians for many years. Most people would claim to be able to recognize a trend when they saw one, but few would be able to go beyond the rather vague definition in the Concise Oxford Dictionary which is that a trend is a general direction and tendency. Actually, this definition is not a bad one, in that it defines the trend in terms of prediction. This is the view taken here. In much of the statistical literature, however, a trend is conceived of as that part of a series which changes relatively slowly over time. In other words, smoothness properties play a key role in the definition. There is no fundamental reason, though, why a trend should be smooth, except that it is some what easier on the eye. What then is a trend? Viewed in terms of prediction, the estimated trend is that part of the series which when extrapolated gives the clearest indication of the future long-term movements in the series. The definition makes no mention of smoothness and it is consistent with the idea of indicating a general direction. Having defined the trend in terms of its properties when extrapolated, we need a mechanism for making this extrapolation, just as we need a mechanism for making predictions of future values of the series itself. A mechanism for making predictions of the series is provided by a statistical model and such a model is fitted in the hope that it will provide the best possible predictions. A statistical model for the trend component can be set up as part of the overall model for the series. This sub-model should be such that the optimal estimator of the trend at the end of the series gives rise to a forecast function satisfying the criterion of the previous paragraph. The optimal estimator of the trend component within the series is then defined automatically. The properties of the estimated trend within the series therefore emerge as a consequence of the required properties of the trend forecast function and the characteristics of the data. Given the above framework, trend analysis is best carried out by means of unobserved component, or structural, time series models. This is the approach described in the books by Harvey (1989), Jones (1993), Kitagawa and Gersch (1996), West and Harrison (1989) and Young (1984). The trend components in structural time series models are always set up in such a way that their forecast functions indicate the long-term movements in the series and the trend within the series can be obtained by signal extraction methods. The first section describes structural time series models and explains how they are handled statistically. The next section looks at the implied weighting patterns for such models. These weighting patterns are important for understanding the relationship between parametric and nonparametric approaches to trend analysis. After looking at kernel estimation and cubic splines, two types of tests are introduced. These are tests against nonstationary trends and tests of an increase or decrease in the level of the trend. There is a clear distinction in the objectives of the tests which would not arise if trends were regarded as being linear and deterministic. The last two sections look at multivariate trend analysis and non-gaussian observations. Most of the data sets used as illustrations can be found in Koopman et al (000). STRUCTURAL TIME SERIES MODELS The simplest time series models for trend analysis are made up of a stochastic trend component, 6t, and a random irregular term. A deterministic trend emerges as a special case. In the local level model the trend is just a random walk. Thus y t 6t / t, / t L /, t 1,..., T 6t 6t"1 1t, 1t L 1,

2 where the irregular and level disturbances, / t and 1t, respectively, are mutually independent and the notation denotes normally and independently distributed with mean zero and 1 is zero, the level is constant. The signal-noise ratio, 1 /@ /, plays the key role in determining how observations should be weighted for prediction and signal extraction. The higher is q, the more past observations are discounted in forecasting the future. Similarly a higher q means that the closest observations receive a higher weight when signal extraction is carried out. Note that, even though the forecast function is horizontal, the model is deemed to have a trend, 1 is zero, as the level changes over time. The local linear trend model is more general in that the trend component in ( ref: measure ) has a stochastic slope, *t, which itself follows a random walk. Thus 6t 6t"1 *t"1 1t, 1t L 1, *t *t"1 0t, 0t L 0, where the irregular, level and slope disturbances, / t, 1t and 0t, respectively, are mutually independent. If both 1 0 are zero, the trend is deterministic, that is 6t 60 *t, t 1,..., T. When 0 is zero, the slope is fixed and the trend reduces to a random walk with drift 6t 6t"1 * 1t. 0 to be positive, but 1 to zero gives an integrated random walk trend, which when estimated tends to be relatively smooth. The signal-noise ratio is now given by q 0 /@ /. The model is often referred to as the smooth trend model. Other things being equal, it is desirable to have a smooth trend since is easier on the eye and more appealing to policy makers. Thus if the integrated random walk and random walk plus drift trends give a similar fit, the integrated random walk may be preferred. What should not be done is to impose the smooth trend regardless of fit. State space form and the Kalman filter The statistical treatment of unobserved component models is based on the state space form (SSF). Once a model has been put in SSF, the Kalman filter yields estimators of the components based on current and past observations. Signal extraction refers to estimation of components based on all the information in the sample. Signal extraction is based on smoothing recursions which run backwards from the last observation. Predictions are made by extending the Kalman filter forward. Root mean square errors (RMSEs) can be computed for all estimators and prediction or confidence intervals constructed. The unknown variance parameters are estimated by constructing a likelihood function from the one-step head prediction errors, or innovations, produced by the Kalman filter. The likelihood function is maximized by an iterative procedure. Full details can be found Harvey (1989) and Durbin and Koopman (000b). The calculations can be done with the STAMP package of Koopman et al (000). Once estimated, the fit of the model can be checked using standard time series diagnostics such as tests for residual serial correlation. Missing observations can be handled very easily within a state space framwork. Irregularly spaced observations are treated by setting up a model in continuous time and then letting the state space form deal with the implied time-varying discrete time model; see Harvey (1989, chapter 9). Reduced form If the local level model is differenced, it can be seen that the autocorrelations are zero beyond the first lag. The reduced form is therefore an ARIMAŸ0,1,1 model

3 y t 8t 8t"1, 8t L and by equating the autocorrelations at lag one it can be shown that "q " q 4q /. Note that is restricted to the range "1 t t 0. The local linear trend model is made stationary by differencing twice. It can be shown that the reduced form is ARIMAŸ0,, : y t 8t 18t"1 8t", 8t L but only part of the invertibility region is admissible; see Harvey (1989, p69). Cycles Distinguishing a long-term trend and from short-term cyclical movements is important in many environmental problems. Short-term movements may be captured by adding a serially correlated stationary component, Et, to the model. Thus y t 6t Et / t, t 1,..., T An autoregressive process is often used for Et. Another possibility is the stochastic cycle Et cos 5c sin 5c Et"1 4t Et ' > " sin 5c cos 5c ' E t"1 4t ', t 1,..., T, where 5c is frequency in radians and 4t and 4t ' are two mutually independent white noise disturbances with zero means and common 4. Given the initial conditions that the vector (E0, E 0 ' U has zero mean and covariance E I, it can be shown that for 0 t > 1, the process Et is stationary and indeterministic with zero mean, 4 /Ÿ1 " > and autocorrelation function >ŸA > A cos 5cA, A 0,1,,... For 0 5c =, the spectrum of Et displays a peak, centered around 5c, which becomes sharper as > moves closer to one; see Harvey (1989, p60). The period corresponding to 5c is =/5c.In the limiting cases when 5c 0or=, Et collapses to first-order autoregressive processes with coefficients > and minus > respectively. More generally the reduced form is an ARMAŸ,1 process in which the autoregressive part has complex roots. The complex root restriction can be very helpful in fitting a model, particularly if there is reason to include more than one cycle. Imposing the smooth trend restriction often allows a clearer separation into trend and cycle. However, following the comments made earlier, such a model should not be forced on the data if diagnostic and goodness of fit statistics indicate that it is inappropriate. Example: Minks in Canada - The number of mink furs traded annual by the Hudson Bay Company data in Canada from 1848 to 1906 has featured as an example in many time series studies. The data display a pronounced cycle and fitting the local linear trend model would simply track the cycle. This is not what is wanted. Using the STAMP package of Koopman et al (000) to fit a smooth stochastic trend plus a stochastic cycle plus an irregular gives the breakdown shown in figure 1. The damping factor of the cycle, >, is estimated as 0.93 while the period is The trend actually reduces to a deterministic linear trend since the estimate of the slope 0, is zero. However, assuming a deterministic trend at the outset would have been inadvisable. When > is unity, the cycle becomes non-stationary. However, suppose that the model is

4 amended slightly by E, rather 4, as being fixed. In this 4 is defined 4 Ÿ1 " E and 4 v 0as> v 1. When > 1 the model becomes deterministic, but ' U the initial conditions are stochastic as ŸE0, E 0 has still zero mean and covariance E I. As a result the process is still stationary since in each realization different initial values are generated and Et has zero mean and ACF as in ( ref: 3a ). Hence the process is deterministic in the sense of Wold and it has a purely discrete or line spectrum. The frequency domain properties of a cycle plus noise model are displayed by the spectral distribution function which shows a sudden jump at 5c. A test of the null hypothesis of a deterministic cycle, > 1, against > 1is given in Harvey and Streibel (1998). Example: Rainfall in Fortaleza - An initial analysis of an annual series on the number of centimetres of rain falling in Fortaleza, a town in north-east Brazil, indicates that there may be a cycle of around 1 to 13 years and another of around 5 years. Fitting a model with a random walk trend and two stochastic cycles results in the cycles being estimated as deterministic. The random walk reduces to a constant, so the series is apparently stationary. Seasonality Observations recorded quarterly or monthly often exhibit seasonal effects. A seasonal component, +t, therefore needs to be incorporated into the model. For example, the basic structural model consists of trend, seasonal and irregular components, that is y t 6t +t / t, t 1,..., T. Seasonal patterns may evolve over time. The trigonometric form of stochastic seasonality has proved effective in modelling a wide variety of seasonal movements. In this case where s is the number of seasons and each +j,t is generated by s/ +t! j1 +j,t, t 1,T,T, +j,t cos 5j sin 5j +j,t"1 Fj,t, j 1,T, Ÿs " 1 /, ' + j,t " sin 5j cos 5j ' + j,t"1 where 5j =j/s is frequency, in radians, and Fj,t and ' F j,t are two mutually independent white noise disturbances with zero means and common F, which is the same for all j. For s even Ÿs " / s/, while for s odd, Ÿs " 1 / Ÿs " 1 /. For s even, there is a single component at j s/ ' F j,t +s/,t +s/,t"1 cos 5s/ Fs/,t, F / The seasonal pattern evolves over time, F 0in which case it is deterministic. The pattern may be of some interest in itself in which case it may be extracted by smoothing. However, the essential point is that seasonal effects can handled within the overall model so that attention can be focused on the trend. WEIGHTING PATTERNS FOR PREDICTION AND SIGNAL EXTRACTION As already noted the calculations for prediction and signal extraction are generally carried out using state space methods. This section examines the way in which observations are weighted in order to make predictions and extract trends. This can be done analytically in simple

5 cases. For more complex models and irregularly spaced observations the algorithm described in Koopman and Harvey (000) may be used to compute weights numerically. Seeing the weighting patterns is important in acquiring an understanding of what exactly the models are doing and how they relate to other methods. The term optimal is this context means minimum mean square error since it is assumed that the models are Gaussian. Without Gaussianity we would still have minimum mean square error estimators within the class of linear estimators. Prediction An expression for optimal predictor of a future observation in a local level model can be obtained from the steady-state of the Kalman filter or by using classical signal extraction theory as in Whittle(1983). It can also be obtained from the ARIMA(0,1,1) reduced form. It depends only on which is a function of q as in ( ref: r9 ). Given observations up to and including y T, the optimal predictor of future observations is the same for all lead times and is equal to the filtered estimator of the level at time T, m T. Thus.. ÿ Tl t m T Ÿ1! Ÿ" j y T"j 5! Ÿ1 " 5 j y T"j, l 1,,3... j0 j0 where 5 1 is the smoothing constant in the exponentially weighted moving average (EWMA). A negative, as implied by ( ref: r9 ), corresponds to a value of 5 between zero and one. The fact that this model underpins the EWMA was pointed out by Muth (1960). The relationship between forecasts from the linear linear trend and techniques like Holt-Winters and double exponential smoothing is discussed in Harvey (1989, ch ). Signal extraction Consider a Gaussian model consisting of two mutually independent stochastic components, 6t and Bt, that is y t 6t Bt, t 1,..., T. The classical Wiener-Kolmogorov (WK) formula for finding the weights used to extract m t., that is the optimal estimator of 6t in a doubly infinite sample, is. m t. wÿl y t! w "jl j y t! w j y tj,. j". j". wÿl + 6ŸL +yÿl, where L is the lag operator and + 6 ŸL is the autocovariance generating function (ACGF) of 6t. The ACGF of y t is +yÿl + 6 ŸL + B ŸL, though this is usually evaluated in terms of the reduced form parameters. For a stationary ARMA process, written C "1 ŸL ŸL 8t, where CŸL and ŸL are polynomials in the lag operator and 8t is white noise with the ACGF is given directly by +ŸL ŸL / where ŸL ŸL ŸL "1 and similarly for CŸL. Although formula ( ref: r1 ) is only proved for stationary models in Whittle (1983, pp.56-58), it can still be used for nonstationary models even though expressions like ( ref: r1a ) are no longer strictly ACGFs. The MSE of an estimated component may be derived from the ACGF of m t. " 6t, denoted by evaluating it at L 0. Since

6 m t. " 6t + 6ŸL +yÿl y t " 6t + 6 ŸL +yÿl " 1 6 t + 6ŸL +yÿl B t its ACGF is +m"6ÿl + 6 ŸL + B ŸL /+yÿl If Bt is white noise, / t, then / wÿl and if expressions for the weights used to extract the trend can be found, then so can the estimation MSE. Local level model Provided q 0, the WK formula for estimating 6t m t. wÿl y t 1 / 1 " 1 L / 1 " L y t Ÿ1 1 L y t, 1 On recognizing that 1/ 1 L is the ACGF of an AR(1) model with parameter " it can be seen that the weights decline symmetrically and exponentially, that is w j Ÿ1 /Ÿ1 " Ÿ" j, j 0,1,,T. Setting L 1 in ( ref: mm ) shows that the weights sum to unity. The weights for the smoothed estimator of 6t near the end of a semi-infinite sample are given in Whittle (1983, p.69) as: w j Ÿ1 /Ÿ1 " Ÿ" "j Ÿ" "jÿt"t 1, ". j t T"t. Setting t T gives the weights for the filtered estimator ( ref: R10 ), while if t z T, the weights are as for a doubly infinite sample as given in ( ref: DDD ). The MSE formula ( ref: msem ) yields MSEŸm / Ÿ1 /Ÿ1 ". The formula may be adapted to semi-infinite samples. The MSE associated with ( ref: si ) is shown by Whittle (1983, p.70) to / Ÿ1 /Ÿ1 " 1 " Ÿ" ŸT"t, t t T. The MSE of the filtered estimator, ( ref: R10 ), is obtained when t T,soitis@ / Ÿ1, while if T " t is large we get the MSE of the smoother, namely ( ref: r18g ) above. Local linear trend In the local linear trend model the weights may be found from the WK formula, ( ref: r1 ), by noting that the reduced form is ARIMAŸ0,, as in ( ref: rf ). For the smooth trend , 0 1"L 11LL. Since 1/ 1 1L L is the ACGF of an AR() process with complex roots, the weights decay according to a damped sine wave as in figure. The pattern depends on the signal-noise ratio, 0 /@ /. In the more general case 1 is not necessarily zero, the weighting function is of the form 0 1 " 0 1 " 1 1 " L 1 6 L 1 1L L

7 where the numerator is obtained from the reduced form of the trend component, which is such that 6t is an MAŸ1 process with moving-average parameter 6 and disturbance 6. The weights are now obtained from the ACGF of an ARMAŸ,1 process in which "1 t 6 t 0 and, as before, 0 t 1 and " 1 t 0. The roots of the implied autoregressive polynomial need no longer be complex, but if they are, the damped sine wave starts from j 1 rather than zero. If the roots are real they must be non-negative. The weights for the slope are obtained by first writing y t *t"1 1t"1 / t. Application of the WK filter then gives a set of weights attached to first differences of the observations, that is b t. w * ŸL y 1 1L L y t1. Since the denominator is equal 0 1 " 1 1 " L /, it can be seen that the w * Ÿ1 1, so the weights sum to one. The weights attached to the observations themselves are given by w * ŸL Ÿ1 " L L "1 and these clearly sum to zero. On the other hand, the weights for estimating the level of the trend sum to one. For the smooth trend, obtained by 1 0, it follows immediately from the ( ref: r3 ) and ( ref: bllt ) that w * ŸL wÿl. It is interesting that this weighting function for extracting the level from the observations is the same as the one for extracting the slope from the first differences. However, this is not surprising since b t. w * ŸL Ÿ1 " L y t1 m t1. " m t., and this corresponds to the identity in the model. NONPARAMETRIC SIGNAL EXTRACTION: KERNELS AND SPLINES There is a very close link between the so called nonparametric techniques for signal extraction and those based on unobserved component time series models. If the trend in a time series is regarded as a deterministic function of unspecified form, it may be estimated at all points by a weighted moving average, the shape of this moving average being termed a kernel. By adopting the rather artificial device that observations are assumed to arrive more frequently (in a given time interval) as the sample size increases, it can be shown that a suitably designed kernel will estimate the trend consistently ; see, for example, Härdle (1990). The method assumes that the non-trending part of the series is white noise, but it may be extended by adopting a hybrid procedure based on fitting an ARMA model as in Hart (1994). A RMSE can sometimes be calculated for the estimated trend as in the case of the simple moving average considered in Brillinger (1994). The implied signal extraction weights from a fitted unobserved component time series model constitute a kernel. In the random walk plus noise model, the signal-noise ratio plays a similar role to a kernel bandwidth. A lower q corresponds to a wider bandwidth. The difference is that q is treated as a parameter and estimated by maximum likelihood and the fit of the model is checked by standard time series diagnostics. Cubic splines are another way of estimated trends nonparametrically. The connection between cubic splines and the local linear trend model has been known for many years; see Wecker and Ansley (1983). It is surprising that the cubic spline fitting methodology, as expounded in Green and Silverman (1994) and elsewhere, makes little or no reference to the time series literature. The local linear trend model which gives the cubic spline is very close to

8 the smooth trend model and so figure gives a good indication of the weight patterns. Fitting a model seems to be altogether more satisfactory than using a kernel or fitting a spline. Furthermore, the model implicity provides appropriate weights near the beginning and end of the series and allows RMSEs to be computed for extracted trends at all points in the series. Predictions can also be made and, as discussed later, the model can be made robust to outliers and structural breaks by specifying t-distributions for the disturbances. TESTING AGAINST NONSTATIONARY TRENDS The stochastic trend component in structural time series model reduces to a deteministic trend as a special case. We may therefore wish to test the null hypothesis that the trend is deterministic against the alternative that is stochastic and nonstationary. Suitable test statistics which are locally best invariant (LBI) can be constructed. Standard asymptotic theory does not apply, but the test statistics have asymptotic distributions under the null which belong to the Cramér-von Mises family. In the Gaussian random walk plus noise model, ( ref: measure ) and ( ref: rwte ), the LBI test of the null hypothesis 1 0, against the alternative 1 0, can be formulated as T 1 T "! i1 i! e t t1 /s c, where e t y t " y, s T "1 T! Ÿyt t1 " y and c is a critical value; see Nyblom and Mäkeläinen (1983). The test can also be interpreted as a one-sided Lagrange multiplier (LM) test. The asymptotic distribution of the statistic is Cramér-von Mises, for which the 5% critical value is If the trend is a random walk plus drift, as in ( ref: rwd ), it becomes deterministic 1 0. Thus y t 60 *t / t, t 1,..., T, The test statistic, 1, is as in ( ref: HC ) except that it is formed from the OLS residuals from a regression on a constant and time. The asymptotic distribution is a second level Cramér-von Mises distribution for which the 5% critical value is If / t is any indeterministic stationary process, rather than white noise, the asymptotic distribution of the test statistic under the null hypothesis remains the same if s is replaced by a consistent estimator of the long-run L! A". +ŸA where +ŸA is the autocovariance of / t at lag A. Kwiatkowski et al (199), hereafter KPSS, construct such an estimator nonparametrically as T s L Ÿe T "1! t1 e e t T "1! A1 T wÿa,e! ta1 e e t e t"a +Ÿ0! wÿa,e +ŸA A1 where wÿa,e is a weighting function, such as wÿa,e 1 " A/Ÿe1, A 1,...,e. Note that the long-run variance can be interpreted as the spectrum at frequency zero while the weighting function wÿa,e is a lag window. An important decision in applying the KPSS test is the choice of e. While asymptotic theory tells us the rate at which e should increase with T, it gives no guidance as to the best choice in a small sample. Indeed the best choice depends on the

9 properties of the series, but it is precisely this information which the nonparametric approach assumes is unavailable. Serial correlation can be handled parametrically by estimating an unrestricted model and then constructing a test statistic of the form ( ref: HC ) from the innovations obtained by running the Kalman filter 1 set to zero; see Harvey (000). Following the argument in Leybourne and McCabe (1994) a parametric test will tend to have a more reliable size and higher power. Example: Nile - Annual data on the volume of the flow of the Nile (in cubic metres 10 8 is shown in figure 3. Fitting a mean and computing the test statistic ( ref: HC ) gives a value of 1.53, indicating a clear rejection of the null hypothesis that there is no random walk component. The KPSS test gives the same result with the statistics for e3 and 7 being 1.10 and 0.73 respectively. Before jumping to the conclusion that the trend is a random walk, it should be noted that the 1test also has power against structural breaks in an otherwise stationary series; see Nyblom (1989). We will return to this point later. Example: Rainfall and minks - Environmental time series may contain cyclical components which are deterministic or near deterministic. Care has to be taken if nonparametric tests are used in such situations. For example the near deterministic cycle in the mink series results in the the KPSS statistic (with time trend fitted) taking the values.0.176, , and for lag lengths,e, of 0, 5, 10, 15 and 0 respectively, so the test gives no clear message. Deterministic cyclical components, as found in the Fortaleza rainfall series, should be accounted for by including the relevant sines and cosines as explanatory variables. TESTS OF AN INCREASE OR DECREASE IN THE LEVEL Grenander (1954) considered the deterministic trend model y t ) *t Bt, t 1,..., T where Bt is a stationary indeterministic process and suggested a test of significance - which could be one-sided - on b, the OLS estimator of *. The variance of b in this case is given L O%Ÿt " t L is the long-run variance defined in ( ref: sigl ). While the stationary part of the model is handled nonparametrically, the assumption of a linear trend is very restrictive. A test constructed to have power against the alternative of a deterministic but monotonically increasing ( decreasing) trend was suggested by Brillinger (1989). The test uses the linear combination %w t y t where w t Ÿt " 1 1 " Ÿt " 1 T 1 " t 1 " t T 1, t 1,..., T The weighting pattern means that it strongly contrasts the beginning and end levels of the series. The test statistic A %w t y t %wt 1, has a standard normal distribution when the level is constant and the test is one-sided on the positive (negative) tail. The method suggested by Brillinger for computing an estimator L is different from the one in ( ref: NW ) in that it is based on a rectangular spectral window applied to the residuals from a simple moving average. A stochastic trend will not generally show a monotonic increase or decrease over time. However, if the trend is assumed to be a random walk with drift, as in ( ref: rwd ), a test of significance of the slope may be carried out by looking at the t-statistic of the filtered estimator of * at time T and treating it as asymptotically normal. A semi-parametric test would estimate *

10 as the average of the first differences, which is equal to (y T " y 1 /ŸT " 1, and divide it by T " 1 multiplied by an estimate of the long-run standard deviation for the differences. More generally, when a stochastic trend is fitted, the relevant question may be whether there has been a change in the underlying level. This requires estimating the temporal (level) contrast, 6T " 61, and its root mean square error (RMSE). To see how this is done, suppose a model contains a general stochastic trend of the form ( ref: ullt ) and consider estimating the more general contrast, 6r " 6s where r s. The smoothed estimators, 6 r3t and 6 s3t, are easily obtained. To find the RMSE of 6 r3t " 6 s3t, define 6t Ÿs,r as 6t " 6s for s 1 t t t r and as 6r " 6s for t r and to add it to the state vector. By including the transition equation 6t Ÿs,r Ÿs,r 6 t"1 *t"1 -t Ÿs,r 1t, t s 1,.., T with 6s Ÿs,r 0 and -t Ÿs,r 1 for s 1 t t t r and zero for t r, the estimator of 6t Ÿs,r is tracked by the Kalman filter. The estimator of 6r " 6s is given by 6 Ÿs,r T together with the RMSE. A test of significance, possibly one-sided, may be based on the standard normal distribution. The temporal contrast test statistic is 6 r3t " 6 s3t RMSEŸ6 r3t " 6 s3t Ÿs,r 6T RMSEŸ6 T Ÿs,r The above algorithm is basically a fixed-point smoother as in Harvey (1989, sub-section 3.6.1). Using a fixed-interval smoother requires that the covariance between the estimation errors at times s and r be computed. This may be done as outlined in de Jong (1989), but adding the extra code may be tedious. However, if t " s is relatively large, and the trend evolves fairly rapidly, the covariance is negligible and the test statistic can be computed by noting that MSEŸ6 r3t " 6 s3t S MSEŸ6 r3t MSEŸ6 s3t, s r. In the case where the two end points are to be compared, the above MSE is given by MSEŸ6 T3T and so it may be obtained from the Kalman filter output at the end of the series. But in making such a comparison it should be noted that the estimates of the trend at the end and beginning have a larger MSE than those within the sample. A more accurate constrast is given between estimates a few periods after the beginning and before the end. Example: Global warming- Figure 4 shows the global annual surface land and marine air anomolies with respect to the average. Bloomfield (199) gives references to the construction of the data and analyses trends; see also Parker et al (1995). The smooth trend model gives a good fit with no evidence of residual serial correlation. The STAMP output shows that the level at the end of the series is with a RMSE of The level at the beginning, in 1880, is and the difference is clearly significantly different from zero as the RMSE for the temporal contrast test statistic, ( ref: testsr ), is Example: Nile - Fitting a local level to the Nile data introduced earlier / The estimated underlying level is at the beginning of the series and at the end. The temporal contrast test statistic is -.64 which is clearly significant. Example: Mink - In the mink series the trend is deterministic, so the approximate formula ( ref: mse ) cannot be used. However, there is no need since a direct test can be carried out on the t-statistic obtained by dividing the estimate of the slope by its RMSE. Since this is based on a full model, the effect of the cyclical component is taken into account. On the other hand, as was demonstrated at the end of the previous section, a nonparametric estimator of the long-run variance needs to be constructed with some care. In a model with a stochastic trend, the lack of interaction between estimators of the level at the beginning and end of the series also enables us to immediately obtain the implicit weighting function used to construct 6 T'. At the end of the series it is simply the weights used to form the filtered estimator. At the beginning the weights are the same except negative. Thus

11 6 T' %w t y t, where, for the local level model, w t 5Ÿ1 " 5 T"t " 5Ÿ1 " 5 t"1, t 1,..., T. As in ( ref: R10 ), the rate of decay, 5, depends on the signal-noise ratio, q.ifq., there is no irregular component, and 5 1; in this case w T 1 and w 1 "1 with RMSEŸ6 T 0. The weighting pattern for the local linear trend model will show a slower initial decay, as it has a shape which derives from figure. It is interesting to contrast these weighting patterns with the nonparametric weights used in Brillinger s test statistic, ( ref: DB ). More complicated structural time series models, including ones with nonstationary seasonal components, may be constructed and the test of a rise or fall in the level carried out in the same way. The weighting pattern for 6 T' may be obtained from the algorithm of Koopman and Harvey (000). OUTLIERS AND STRUCTURAL BREAKS Detection A stochastic trend allows for small breaks in every time period. However, there may sometimes be large structural breaks which arise because of some external event or policy change. These can sometimes be detected by looking at trend. A better way, which leads to formal tests, is to look at what Harvey and Koopman (199) call the auxiliary residuals. These are estimates of the various disturbances in the model. Thus the level auxiliary residual is an estimate of 1t in ( ref: rwte ) or ( ref: ullt ) and a large value indicates a shift in the level. If a break can be identified with a particular event it is reasonable to model it by including a dummy variable in the model. Example: Nile - Fitting a local level model to the Nile data of figure 3 shows the estimated trend falling around A plot of the level auxiliary residuals, as in Koopman (000) points to 1899 as being the likely date for a break and in fact this was when the first Aswan dam was built. If the local level is fitted with a dummy variable which is zero up to 1898 and one 1 is estimated to be zero and so the level is fixed. Outlying observations can be similarly detected by the irregular auxiliary residuals. Robust estimation Robust models offer an alternative way of dealing with outliers and breaks. Thus allowing / t to have a heavy-tailed distribution, such as Student s t, provides a way of dealing with outliers, while a heavy-tailed distribution in the transition equation allows for the possibility of structural breaks. Simulation techniques, such as those described in Durbin and Koopman (000a,b) are used to estimate the model. Example : Gas consumption in the UK - Estimating a Gaussian basic structural model for gas consumption produces a rather unappealing wobble in the seasonal component at the time North Sea gas was introduced in Durbin and Koopman (000a) allow the irregular to follow a t-distribution and estimate its degrees of freedom to be 13. The robust treatment of the atypical observations in 1970 produces a seasonal component which evolves more smoothly around that time. MULTIVARIATE MODELS The local linear trend model can be generalized to the multivariate case straightforwardly simply by writing

12 y t 6 t /t, /t L NIDŸ0, % /, t 1,T,T, 6 t 6 t"1 * t"1 1 t, 1 t L NIDŸ0, % 1, * t * t"1 0 t, 0 t L NIDŸ0, % 0, where y t is an N 1 vector and % /, % 1 and % 0 are N N covariance matrices. Seasonals and cycles can be added. The covariance matrices could reflect all types of influences. For example, if the series are for different regions, they may have some kind of spatial structure which captures geographical proximity. As well as having an interesting interpretation, multivariate models may provide more efficient inferences and forecast. Common factor models, in which some of the covariance matrices are less than full rank, are of particular importance. The basic ideas can be illustrated with a bivariate local level model y 1t 61t / 1t, 61t 61t"1 11t, t 1,..., T, y t 6t / t, 6t 6t"1 1t The covariance matrix of Ÿ11t, 1t U may be written % > where > 1 is the correlation. The model can be transformed as follows: y 1t 61t / 1t, y t =61t 6 t / t, where = > and the covariance matrix of the disturbances driving the new multivariate random walk Ÿ61t, 6 t U is Var 11t " >1@1. In other words by setting 1t =11t 1 t, two uncorrelated levels, 61t and 6 t, based respectively on 11t and 1 t, are obtained. If > 1 o1, then 6 t is constant and there is only one common trend which will now be denoted as 6t rather than 61t. Hence y 1t 6t / 1t, t 1,..., T, y t =6t 6 / t If = 1, the trend in the second series is always at a constant distance, 6, from the trend in the first series, that is 6t 6 61t. This is known as balanced growth. Pre-multiplying the observation vector in ( ref: bicom ) by the matrix gives "= 1 0 1

13 y t =y 1t 6 / t y 1t 6t / 1t, where / t / t " =/ 1t and = This is equivalent to the original common trends model and can be estimated directly. Since the linear combination y t " =y 1t is stationary, the nonstationary series y t and y 1t are said to be co-integrated. There is a vast amount of econometric literature on co-integration and it is important to note the connection with common trends. A recent review can be found in Maddala and Im (1998). When there are N series there may be K common trends, where 0 K N. A test of the null hypothesis that there are K common trends against the alternative that there are more can be found in Nyblom and Harvey (000). Multivariate models can be used for estimating an intervention effect with a control group. Common trends can lead to significant gains in efficiency as illustrated by the analysis of the British seat belt law in Harvey (1996). NON-GAUSSIAN OBSERVATIONS Data are sometimes intrinsically non-gaussian. For example point processes give rise to count data or duration data, while a qualitative reponses, such as yes or no may be coded as a binary series. Most of the distributions used to model such data are from the exponential family. There are two main ways of dealing with stochastic trends. The first is to combine a nonlinear measurement equation with a linear transition equation. Thus for a Poisson distribution pÿy t 5te "5t /y t!, the mean, 5t, may be connected to trend, cycle and seasonal components by an exponential link function, that is log 5t 6t Et +t. The statistical treatment is by simulation methods. For example, Carter and Kohn (1996) and Shephard and Pitt (1997) base their approach on Markov chain Monte Carlo methods such as Gibbs and Metropolis sampling, while Durbin and Koopman (000a, 000b) use importance sampling. All of these techniques can be applied within classical and Bayesian frameworks. An alternative class of models, which can be handled by conjugate filters, avoids the need for simulation techniques. Thus for Poisson observations a gamma prior combines naturally to produce a gamma posterior and a likelihood function. This implies a nonlinear transition equation. The disadvantage is that only the level can be stochastic; see Smith and Miller (1986) and Harvey and Fernandes (1989). Nevertheless, although the slope and seasonals have to be fixed, this may not be a major drawback for many non-gaussian data sets As regards testing, Nyblom (1989) shows that the 1 test against a random walk component can be employed with non-gaussian observation, while Brillinger (1995) generalises the nonparametric test statistic in ( ref: DB1 ) so as to detect an increase or decrease in the level of binary data. ************** * Headings for figures Figure 1 Trend, cycle and irregular components for mink Figure Implied weighting patterns for smoothed estimator of trend in integrated random walk plus noise models Figure 3 Flow of the Nile and its estimated level. Figure 4 Global annual surface land and marine air anomolies with respect to the average.

14 REFERENCES Bloomfield, P. (199). Trends in global temperature. Climatic Change, 1, Brillinger, D.R. (1989). Consistent detection of a monotonic trend superposed on a stationary time series. Biometrika 76, Brillinger, D.R. (1994). Trend analysis. Environmetrics 5,1-19. Brillinger, D.R. (1995). Trend analysis: binary-valued and point cases. Stochastic Hydrology and Hydraulics 9, Carter, C. K., and R. Kohn (1996). Markov chain Monte Carlo in conditionally Gaussian state space models. Biometrika 83, de Jong, P (1989). Smoothing and interpolation with the state-space model. Journal of the American Statistical Association 84: Durbin, J., and S.J. Koopman (000a). Time series analysis of non-gaussian observations based on state-space models from both classical and Bayesian perspectives (with discussion). Journal of Royal Statistical Society, Series B 6, Durbin, J. and S.J. Koopman (000b). Time Series Analysis by State Space Methods. Oxford University Press, Oxford. Green, P.G. and B.W. Silverman (1994). Nonparametric regression and generalized linear models. London: Chapman and Hall. Grenander, U. (1954). On the estimation of regression coefficients in the case of an autocorrelated disturbance, Ann. Math. Statist. 5, 5-7. Härdle, W. (1990) Applied Nonparametric Regression. Cambridge: Cambridge University Press. Hart, J.D. (1994) Automated kernel smoothing of dependent data by using time series cross-validation. Journal of the Royal Statistical Society, Series B 56, Harvey, A.C. (1989) Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge: Cambridge University Press. Harvey, A.C. (001) A unified approach to testing for stationarity and unit roots. Mimeo. Harvey A.C. and C. Fernandes (1989), Time series models for count data or qualitative observations, Journal of Business and Economic Statistics 7, Harvey, A.C., and S.J.Koopman (199). Diagnostic checking of unobserved components time series models. Journal of Business and Economic Statistics 10, Harvey, A. C., and M. Streibel (1998). Tests for deterministic versus indeterministic cycles. Journal of Time Series Analysis 19, Jones, R.H. (1993). Longitudinal Data with Serial Correlation: A State-space Approach. London: Chapman and Hall. Kitagawa, G. and W Gersch (1996). Smoothness Priors Analysis of Time Series. Berlin: Springer-Verlag. Koopman, S.J., Harvey, A.C., Doornik, J.A. and Shephard, N. (000) STAMP 6: Structural Time Series Analysis Modeller and Predictor, London: Timberlake Consultants Ltd. Koopman, S.J. and Harvey, A.C. (1999). Computing observation weights for signal extraction and filtering, mimeo. Kwiatkowski, D., Phillips, P.C.B, Schmidt, P. and Y.Shin (199). Testing the null hypothesis of stationarity against the alternative of a unit root: How sure are we that economic time series have a unit root? Journal of Econometrics 44, Leybourne, S.J. and B.P.M. McCabe (1994). A consistent test for a unit root. Journal of Business and Economic Statistics 1, Maddala, G.S. and I-M. Kim (1998). Unit Roots, Co-integration, and Structural Change. Cambridge: Cambridge University Press. Muth J.F. (1960). Optimal properties of exponentially weighted forecasts, Journal of the

15 American Statistical Association, 55, Nyblom, J. and T. Mäkeläinen (1983). Comparison of tests for the presence of random walk coefficients in a simple linear model, Journal of the American Statistical Association, 78, Nyblom, J.(1989). Testing for the constancy of parameters over time, Journal of the American Statistical Association, 84, Nyblom, J., and A.C.Harvey (000). Tests of common stochastic trends, Econometric Theory, 16, Parker, D.E., Folland, C.K. and M. Jackson (1995). Marine surface temperature: observed variations and data requirements. Climatic Change 31, Shephard, N., and M. K. Pitt (1997). Likelihood analysis of non-gaussian measurement time series. Biometrika 84: Smith, R.L. and J.E.Miller (1986). A non-gaussian state space model and application to the prediction of records. Journal of the Royal Statistical Society, Series B 48, Wecker, W.E. and C.F. Ansley (1983). The signal extraction approach to nonlinear regression and spline smoothing. Journal of the American Statistical Association, 78, West, M. and P.J.Harrison (1989). Bayesian Forecasting and Dynamic Models. New York: Springer-Verlag. Whittle, P. (1983). Prediction and Regulation, nd ed. Oxford: Blackwell. Young, P. (1984). Recursive Estimation and Time Series Analysis. Berlin: Springer-Verlag.