data lam=36.9 lam=6.69 lam=4.18 lam=2.92 lam=2.21 time max wavelength modulus of max wavelength cycle

Transcription

1 AUTOREGRESSIVE LINEAR MODELS AR(1) MODELS The zero-mean AR(1) model x t = x t,1 + t is a linear regression of the current value of the time series on the previous value. For > 0 it generates positively auto-correlated time series, = 1 is a random walk, < 1 represents stationary time series. A stationary series varies around its mean (which here is zero), randomly wandering o away from the mean in response to the \input" values of the random t series, but always returning to near zero, and never \exploding" away for more than a short time. In standard regressions, the innovations are called errors and represent the unexplainable variation in the response; here the time series is actually \driven" by the innovations. AR(1) series with 0 < < 1 represent short-term, positive correlations that would damp out exponentially if t were zero (for then x t+k = k x t ; for example). Negative values of represent short-term, negative correlations. In fact, in stationary models with jj < 1; the parameter is precisely the population correlation between any two consecutive values of the series, say x t ;x t,1 or x t ;x t+1 for any t: Correspondingly, in a reference analysis, the reference posterior mean (also the least squares estimate) of is the sample autocorrelation at lag one, just the sample correlation computed from pairs x t ;x t,1 : Finally, values of jj > 1 imply non-stationary, \explosive" models. AR(2) MODELS Models x t = 1 x t,1 + 2 x t,2 + t are most interesting in the case when the characteristic equation 0=1, 1 u, 2 u 2 has two complex conjugate roots whose reciprocals are r exp (i!) for some r in (0; 1) and!>0: In this case, x t is a stationary process { it varies around its zero-mean, randomly wandering o but always returning, never \exploding" { just as in the AR(1) model with jj < 1: In these cases, the model x t = 1 x t,1 + 2 x t,2 + t represents a quasi-cyclical process { behaving as a damped sine wave of xed frequency!; and so wavelength or period =2=!: At time t the amplitude and phase characteristics depend on t and the past innovations { they vary randomly over time in response to the innovations t : A large innovations variance v induces greater degrees of variation in this dynamic, quasi-cyclical process. If the innovation variance is very small, or were to become zero at some point, the process would decay to zero in amplitude due to the damping factor, oscillating as it does so. The damping arises from the modulus r of the model. If r is small there is a lot of damping, if r is near 1 the damping is low, and if r =1we are on the boundary between stationary and non-stationary processes; here x t would be a time-varying sinusoidal form but with no damping. It helps to note that the model is a stochastic dierence equation, and recall the deterministic version: if t = 0 for all t; then x t = 1 x t,1 + 2 x t,2 denes a damped sine wave of the form x t = cr t sin(!t + a) for some amplitude c and phase a that depend on initial values x 1 ;x 2 : Adding the stochastic innovation t at each time point randomly distorts the sine waveform after damping by r; producing the quasi-periodic result. Interpretation of the damping factor r may be guided by thinking in terms of the implied half-life,, log(2)= log(r); i.e., the number of time intervals it takes for the process to decay to half its \current" value if the innovations remain at zero. For example, the half-lives at r =0:99; 0.95, 0.90 and 0.75 are about 69, 13.5, 6.6 and 2.4 time units, respectively. 1

2 AR(p) MODELS AR(p) models x t = Pp j=1 jx t,j + t are capable of adequately representing a wide range of observed behaviours in time series { purely empirically in terms of model t { for large enough p: The mathematical structure can often be exploited to connect with key structural features of the time series, such as important periodicities, by exploiting a very important time series decomposition result. This result says, simply, that If x t follows an AR(p) model, then x t can be written as the sum of a number of AR(1) and AR(2) underlying, latent (or \hidden") processes. So, if we know all about AR(1) and AR(2) processes, and have the computer software to implement the theory and nd or estimate the latent components, we have all we need to understand AR(p) models. Here's an example. An AR(10) model was tted to the Southern Oscillation Index (SOI) time series. This gives a decent empirical t to the time series with estimated AR vector = ( 1 ;:::; 10 ) 0 given in the coecent column of the S-Plus output below. Parameter Estimate S.E. t value P r(> jtj) X X X X X X X X X X Though some of the higher order coecients may appear to be of low signicance, the higher order is needed to adequately capture high frequency noise components in the series, and the nal coecient is clearly signicant { this is a general rule, and generally we would not remove the intermediate lagged x values from the model. The estimated innovations { the tted residuals { appear roughly normal, from a qqplot, so we explore this tted model further. The latent component decomposition is implemented in a couple of S-Plus functions I have written { see the le arfit.s. The theory tells is that the parameters of the underlying AR(1) and AR(2) components arise from the reciprocals of the roots of the characteristic polynomial 0=1, 1 u, 2 u 2, p u p : For example, plug-in the point estimates j from the tted model of the SOI series, and we nd that the polynomial, of degree 10, has roots that are all complex { ve pairs of complex conjugates. This means that the tted AR(10) model corresponds to the sum of ve AR(2) models, x t 5X j=1 2 z jt

3 where z jt is a quasi-cyclical AR(2) process with modulus r j and frequency! j given by the j th pair of roots, r j exp(i! j ) for j =1;:::;5: S-plus solves polynomials and delivers complex roots. Here they are in this model, together with the wavelengths j =2=! j : j : r j : ! j : j : The estimate of the largest wavelength here, at j = 5; is 36.9 months, almost exactly 3 years; this corresponds to an underlying climatological periodicity related to El Ni~no. The modulus is estimated at 0.9, indicating an AR(2) component that is only moderately persistent; a damping factor of 0.9 corresponds to a half-life of about 6 months. The next component, j = 4; has an estimated period of about 6months, a semi-annual cycle, and is much less persistent. Components 1-3 all have lower wavelengths, so represent higher frequency uctutations that may not relate to physical periodicities but that are needed to adequately capture the high frequency correlation structure evident in the data. The theory of decomposition also tells us how toevaluate these underlying components, at each time point t; given any value of : The gure below graphs the estimated z jt in this analysis by plugging-in the above estimate of : All the components are graphed on the same scale as the series itself, so you can clearly see the relative amplitudes and the dominance of the main 3year quasi-periodic component { a \smoothed" version of the series. The semi-annual uctuations are of relatively minor amplitude, but still a real feature of the series. We can also obtain posterior distributions for the wavelengths and modulii of components to explore full posterior uncertainty about their values. This is most easily done by simulation: draw samples from the reference posterior (multivariate T) distribution of given the data, and compute the polynomial roots for each sampled vector, etc. Ihave another S-Plus function that does this and summarises the posterior for the component with maximum wavelength, as illustrated for the SOI analysis below. Evidently, there is a high degree of uncertainty about the period { the estimated value of 36 months may be an underestimate, as the posterior suggests a high probability onvalues as high as 50months. 3

4 data lam=36.9 lam=6.69 lam=4.18 lam=2.92 lam= time Decomposition of SOI series from tted AR(10) model max wavelength modulus of max wavelength cycle Simulated posteriors for wavelength and modulus of main component in SOI series 4

5 PREDICTION Looking at how a model forecasts out-of-sample { which for time series means future data { is relevant in examining model t as well as in direct forecasting. In connection with model t and adequacy, one nice subjective study is to sample a series of future x values from the predictive distribution from the model analysis, and then plot the resulting sample future following the past data used to t the model. If it is easy to see dierences in patterns in the forecasts relative to the data, model t may be questioned. A good model will produce sample futures that are hard to distinguish from the data. This can be repeated several or many times to generate a set of repeat sample futures that usefully represent the predictive distribution in terms of providing insight into the kinds of potential outcomes predicted by the model, rather than simply in terms of traditional statistical summaries such as mean outcomes. In economic and business applications, forecasting (or \prediction") is a key goal of time series modelling and analysis. For example, in the GDP series study, purely empirical AR models may adequately capture the key features of the very short time series. An AR(3) or AR(4), for example, may exhibit one key quasi-cyclical component whose period is that of the apparent \business cycle", often 3-6 years in length, though often a rather subtle pattern. One interest here is in the use of such models to anticipate possible up or down-turns by forecasting ahead and asking about the (posterior predictive) probabilities of turning points. To do this, we need to compute predictive distributions, but obviously that involves forecasting future x values to use as regressors for following values! Fortunately, this is a trivial issue if we adopt the view that forecasting is about simulating possible futures, i.e., sampling the future, as mentioned above in connection with model adequacy assessments, rather than evaluating posterior predictive distributions. Suppose we have tted an AR(p) model to observations giving us data X n = fx 1 ;:::;x n g for some n: This gives us a joint posterior for (; 2 jx n ) where is the p,vector autoregressive parameter and 2 the innovations variance. The prescription to simulate the predictive distribution for, say, the next k =8values is: sample one parameter (; 2 ) from p(; 2 jx n ); set x n+1 = Pp j=1 jx n+1,j + n+1 where the j are at their sampled values and n+1 is itself a random draw from N(0; 2 ) at the sampled 2 value; repeat the above step for x n+i = Pp j=1 jx n+i,j + n+i for i =2; 3;:::;k;at each step inserting the recently sampled value of the last x n+i,1 as predictors. The following graph demonstrates this with four repeat simulations of sample futures over the next 350 months for the SOI series. Can you see where the data ends and the forecasts begin? 5

6 Four sampled futures for the SOI series, preceded by the actual data From such analyses we can easily compute inferences about complicated events and quantities, such as turning points in an economic time series. Suppose we have sampled x n+1 ;:::;x n+8 a reasonably large number of times, say 5000; and interest lies in a possible \downturn" next year. That corresponds to x n+1 < x n ; so we simply count the number of times this occurs out of our 5000 predictive samples, and that gives us a simulation-based estimate of the predictive probability of a downturn next year. Similarly, we might explore the predictive distribution of the number of years until the next downturn by drawing a histogram of the number of years it takes in each of the 5000 predictive samples. An approximation to this analysis can be explored just by plugging-in the posterior estimates of (; 2 ); rather than simulating them. This is often done, though of course it leads to a further approximation: it ignores the posterior uncertainty about the parameters, and that may be considerable in some applications. 6