Introduction to univariate Nonstationary time series models

Transcription

1 Introduction to univariate Nonstationary time series models Laura Mayoral Winter 2012, BGSE 1 Introduction Most economic and business time series are nonstationary and, therefore, the type of models that we have studied cannot(directly) be used.

2 Nonstationary can occur in many ways: non constant means, non-constant variances, seasonal patterns, etc. Thefollowinggraphs(fromWei,p. 68,69)presenttwo economic time series that show strong patterns of nonstationarity. The first plot corresponds to a process whose mean is non constant while the second displays a time series where both the mean and the variance seem to increase over time.

3 This chapter introduces several approaches for modeling nonstationary time series. Consequences of nonstationarity for statistical inference will also be analyzed.

4 2 Trends Most economic series are trended. A process presenting a non-constant mean might present serious problems for estimation since multiple realizations of the time mean dependent function are not available. The usual approach is to consider models that after some transformations become stationary. Two popular cases: Trend stationary model, X t =α+βt+ψ(l)ε t. whereψ(l)ε t isastationaryprocess. Thisprocess is often called trend-stationary because if one subtracts the trend component βt the result is a stationary process.

5 Unit root process X t =X t 1 +β+ψ(l)ε t whereψ(1) 0.X t canalsobewrittenas(1 L)X t = β+ψ(l)ε t and is said to be a unit root process becausel=1isarootoftheautoregressivepolynomial. [Somenotation: (1 L)=.] Thetransformedprocess(1 L)X t = X t =X t X t 1 isstationaryanddescribesthechanges(orthe growthrateifx t isinlogs)intheseriesx t. 2.1 Deterministic trend functions: Trend Stationary models Anaturalwayofmodellingatrendcomponentisby usingasth-degreepolynomialinpowersoft: τ(t)=β 0 +β 1 t+...+β s t s,

6 sothattheprocess{x t }canbewrittenas X t =τ(t)+u t, (1) whereu t isacovariance-stationaryprocess. Polynomial trend functions are particularly useful when the mathematical form of the true trend is unknown since any analytic mathematical function can be approximated by a polynomial. Typically, linear functions(β 0 +β 1 t) are employed to represent the trend. The reason for considering lineartrendsisthatinsomecasestheoriginalvariable presents proportional growth that can be capturedbyanexponentialtrend, thatis, τ(t)=e βt. Then, dx t /dt = βe βt = βx t. These series are usually modelled in logarithms and then the trend becomes linear since log(τ(t))=βt.

7 Estimation. (see Hamilton, Ch. 16). Consider the model: X t =α+βt+u t, whereu t isastationaryprocess. Then,1)thepolynomial function can be consistently estimated by OLS and 2) standard t or F tests can be employed totestwhetherβand/orαaredifferentfromzero. 2.2 Stochastic trend models: ARIMA models Itispossiblethatwhatisperceivedasatrendistheresult of the accumulation of small stochastic fluctuations. The simplest model embodying a stochastic trend is the randomwalkmodel. Let{X t }bethe randomwalksequence, then X t =X t 1 +ε t, (2)

8 where {ε t } is a white noise sequence. Assuming that X t =0forallt<0andthatX 0 isafixedfiniteinitial condition then, by back-substitution, X t =X 0 + t i=1 and E(X t ) = X 0 and var(x t ) = tσ 2. This process has no trend. ε i. To introduce a trend component it is only needed to include a constant in (2). The random walk with drift model is and by back-substitution X t =β+x t 1 +ε t (3) X t =β+(β+x t 2 +ε t 1 )+ε t =...=X 0 +βt+ t i=1 Noticethatifthestartingpointisintheindefinitepastratherthan att=0,thenthemeanandthevarianceareundefined. ε i.

9 More general models can be found by allowing the stochastic component in(3) to be a stationary sequence (1 L)X t = β+u t, u t = ψ(l)ε t, where ψ(1) 0. If u t admits an ARMA(p,q) representation, such that φ(l)u t = θ(l)ε t, then X t is an ARIMA (Autoregressive Integrated Moving Average) process φ p (L)(1 L) d X t =β+θ q (L)ε t, (4) where{ε t }isawhitenoiseprocesswithvarianceσ 2 <,φ p (L)=1 φ 1 L... φ p L p andθ(l)=1+θ 1 L+...+θ q L q are the autoregressive and moving average polynomials, respectively, sharing no common factors and with all their roots outside the unit circle. ThisassumptionisnecessarytoguaranteethattheMAcomponent does not contain a unit root that would cancel out with the unit rootofthearpolynomial,inwhichcasey t wouldbeastationary process.

10 Inthiscased=1but,moregenerally,dwillbeapositive integer number and represents the number of times X t mustbedifferencedtoachieveastationarytransformation. Typically, d {0, 1, 2}. The case d = 0 corresponds to the ARMA case, studied in Chapter 2. The term β is a deterministic component and plays differentrolesfordifferentvaluesofd.ifd=0,β represents aconstanttermsuchthatthemeanof{x t }isgivenby µ=β/(1 φ 1... φ p ).Ifd=1,βisthecoefficient associated with a linear trend and if d = 2, X t is the coefficientassociatedtoaquadraticterm,t 2. Ifthevariableisinlogs,aunitrootimpliesthatthe rate of growth of the variables is stationary. Toseethisnoticethat (1 L)logX t =u t, (1 L)logX t = log(x t /X t 1 ) = log(1+(x t X t 1 )/X t 1 )

11 and if the change is small, using the approximation log(1+x) xifxisclosetozero,then. (1 L)logX t (X t X t 1 )/X t 1. Therandomwalkmodelcanbeseenasthelimitof anar(1)modelwhenφ 1.SincetheACFofthe AR(1)isρ(h)=φ h,thenthecloserφisto1,the slowerthedecayofthisfunctiontozero. Inparticular,thesampleACFofarandomwalkischaracterized by large slowly vanishing spikes and insignificant zeroacfforthedifferencedseries(1 L)y t. Thefollowinggraphsrepresentarandomwalkanda random walk with drift, respectively, computed with simulated data.

12 Simulatedseries: a)(1 L)y t =ε t,b)(1 L)y t =4+ε t.

13 NoticethatthebehavioroftheACFandthePACF for the random walk with or without drift is fairly similar although the ACF tends to decay more slowly whenthereisadrift. Oneshouldlookattheplotof the original data. If there is a drift, the deterministic component tends to dominate the stochastic one and the data looks clearly trended. Remark1 AnARIMA(p,d,q)processwithdriftisnonstationarybothinthemeanandinthevariance,whichis unbounded as t and the autocovariance function is also time dependent.

14 Remark2 ThesampleACFofatrendstationarymodel is very similar to that in (1), it also decays very slowly tozero. Thusitisdifficulttoidentifythecorrectmodel for the trend(stochastic or deterministic) by only looking at this property. Unit root tests are usually employed to determine which model is more suitable. The following graphs illustrate this fact. The first one, presents two process,a.y t =0.5t+ε t andb.y t =0.5+y t 1 +ε t while the second plots the corresponding sample ACF. 3 Comparing Trend-stationarity and Unit root Processes. Althoughbothtypeofmodelsareabletocaptureasimilar behavior in the data (trends), deciding on which model touseisnotanobvioustask. Herewediscussthreemain differences between them, related to 1) the persistence of shocks, 2) forecasting, 3) type of transformation that is needed to make the data stationary.

15 70 Trend stationary model 60 random walk Figure 1:

16 1 Sample ACF, Trend stationary 1 sample ACF, random walk Figure 2:

17 3.1 Comparison of persistence properties An important difference between trend-stationary and unit rootprocessesisthepersistenceofinnovations. IfX t is a trend-stationary process, the effect of a shock h periods ahead is measured by the impulse response function (IRF)thatinthiscaseisequalto X t+h ε t =ψ s, andthenass, X t+h / ε t 0. TheIRFofX t ifitisaunitrootprocessis X t+h ε t =1+ψ ψ s and then, as s, X t+h / ε t ψ(1). That is, anyinnovationε t hasapermanent effectonthelevel ofythatdoesnotwearsoff.

18 3.2 Comparison of forecasts WhenaTSmodelisusedtoproduceforecastsforavariable, the forecast error converges to a constant when the forecast horizon tends to infinity though. In contrast, when a unit root model is employed instead, this error goes to infinity. Consider the unit root and trend-stationary models given, respectively, by X t = X t 1 +ψ(l)ε t X t = α+βt+ψ(l)ε t whereε t isam.d.s. inbothcases. The forecast of a trend-stationary process is obtained by adding the deterministic trend to the forecast of the stationary component: ˆX t+s t =α+β(t+s)+ψ s ε t +ψ s+1 ε t

19 The corresponding forecast error is X t+s ˆX t+s t =ε t+s +ψ 1 ε t+s ψ s 1 ε t+1, and the mean-square error, MSE=E ( ) 2= ( X t+s ˆX t+s t 1+ψ ψ 2 s 1) σ 2. Then, the MSE increases with the forecasting horizon s, but since i=0 ψ 2 i <, the added uncertainty from forecasting farther into the future becomes negligible. To forecast the unit root process, recall that X t is a stationary sequence and then ˆX t+s t =α+ψ s ε t +ψ s+1 ε t , andsince ˆX t+s t = ˆX t+s t ˆX t+s 1 t and ˆX t+s 1 t = ˆX t+s 2 t +α+ψ s 1 ε t +ψ s ε t 1 +ψ s+1 ε t 2...,etc., then ˆX t+s t =αs+x t +(ψ s +ψ s ψ 1 )ε t +(ψ s+1 +ψ s +...+ψ

20 Tocomputetheforecasterror,noticethatX t+s canbe written as X t+s = (X t+s X t+s 1 )+(X t+s 1 X t+s 2 )+...+ = X t+s + X t+s X t and then ( Xt+s ˆX t+s t ) = ( Xt+s + X t+s X t ) ( ˆX t+s t + ˆX t+s 1 t +...+X t ) which yields, ( Xt+s ˆX t+s t ) = εt+s +(1+ψ 1 )ε t+s 1 +(1+ψ 1 +ψ 2 )ε t+s andthemse + ( 1+ψ ψ s 1 ) εt+1 E ( X t+s ˆX t+s t ) 2 = (1+(1+ψ1 ) 2 + (1+ψ 1 +ψ 2 ) ( 1+ψ ψ s 1 ) 2)σ 2.

21 Then the MSE increases with the forecast horizon though, in contrast with the trend-stationary case, it tends to infinity. 3.3 Transformations to achieve stationarity Wehaveseenthatifaprocessisnonstationary,thecorrect treatment to achieve a stationarity transformation is to subtract a linear trend or to take differences depending on whether the process is trend stationary or integrated (=contains a number of unit roots), respectively. However, in practice the type of nonstationarity is unknown. ConsiderthecasewhereX t isreallyarandomwalkwith drift process but a linear trend is subtracted. Then X t (α+βt)=(x 0 α)+ t i=1 ε t.

22 Then, it is clear that E(X t (α+βt)) is constant. However the transformation does not yield a stationary processbecausethevarianceisnotconstant,sincevar ( ti=1 ε t tσ 2. If X t is trend stationary and first differences are taken then (1 L)X t = (1 L)(α+βt)+(1 L)ε t = β+(1 L)ε t. The resulting process is stationary. However, the MA polynomial(1 L)ε t containsaunitrootandthen,the representation is noninvertible. 4 Other sources of nonstationarity Other common sources of nonstationarity are heterokedasticity and seasonality.

23 4.1 Processes with nonconstant means: Variance stabilizing transformations Some processes display nonconstant variances. In some cases, a variance stabilizing transformation can be implemented to overcome this problem. Acommoncaseiswhenthevarianceofsomenonstationary variables changes as its level changes. Thus Var(y t )=cf(µ t ) In order tofind a function g such that g(y t ) is a series with constant variance, we consider the first order Taylor expansionofthetransformationaroundthepointµ t g(y t ) g(µ t )+g (µ t )(y t µ t )

24 and then Var(g(y t )) = ( g (µ t ) ) 2 Var(yt ) = c ( g (µ t ) ) 2 f(µt ) then,g(y t )willhaveaconstantvarianceif g (µ t )= 1 f(µ t ) g(µ t)= 1 f(µ t ) Then, if the standard deviation of y t is proportional to thelevel,i.e.,var(y t )=cµ 2 t,then g(µ t )=log(µ t ) and then a logarithmic transformation will stabilize the variance. If the variance is proportional to the level, then g(µ t )= 1 µt =2 µ t, andtakingthesquarerootoftheprocessy t willstabilize the variance.

25 More generally, one can apply the Box-Cox transformation g(y t )=y λ t = yλ t 1 λ Some commonly used values of λ and their associated transformation are g(y t )= λ g(y t ) y t 1 yt 0 log(y t ) 0.5 T 1 y t The following graphs illustrate how by taking logarithms, the variance of the process tends to stabilize over the sample.

26

27 Some remarks: 1. The transformations above can only be applied to positive series. This is not restrictive because a constantcanbealwaysaddedtotheseriesandtheautocovariance structure would no be modified 2. These transformations, if needed, should be performed before any other analysis such as differencing

28 3. Theparameterλcanbeestimatedasanyotherparameter of the process by maximum likelihood. Usually, a grid of values of λ is considered and then a model is fitted to each of the transformed processes. The maximum likelihood estimator of λ is the one that minimizes the residual sum of squares of the estimated models. 5 Seasonal ARIMA models Seasonal series are characterized by a strong serial correlation at the seasonal lag. Consider the following plots corresponding to artificially generated monthly data.

29 PlotandACFofsomemonthlytimeseries. X t =X t 12 +ε t Fromtheplotofthedata,wecanseeaseasonalbehaviour with a period s = 12. Likewise, when looking at the ACF, we observe that correlations at the seasonal lag (=12) are highly significant. Suppose that we have r years of monthly data and we consider separately the observations corresponding to January, February, etc:

30 Eachcolumninthistablecouldbeviewedasarealization ofatimeseries. Then,wewouldexpecttofindastrong correlation between the values of observations taken in the same month of successive years. In the simplest of circumstances, we might find that the difference between X t andx t 12 isacompletelyrandomquantity: X t =X t 12 +u t. Thiscanalsobewrittenas ( 1 L 12) X t =u t or 12 X t = u t,whereu t isawhitenoiseprocess. It is clear that in many cases, the relation between the months of the calendar can be more complicated. This can be accomplish by assuming that u t is not a white noise process but rather an stationary process that admits the decomposition: sothatx t becomes: Φ P ( L 12 ) u t =Θ Q ( L 12 ) ε t, Φ P ( L 12 ) 12 X t =Θ Q ( L 12 ) ε t,

31 where Φ P ( L 12 ) = 1 Φ 1 L Φ P L 12P and Θ Q ( L 12 ) =1+Θ 1 L Θ Q L 12Q,arethe(seasonal) AR and MA polynomials, respectively. Hence, the same seasonal ARIMA model is applied to eachofthe12separatetimeseries,oneforeachmonth of the year, whose observations are separated 12 lags. If ε t isaniidsequence,thenthese12seriesarecompletly unrelated. However, it is reasonable to think that in most cases, in addition to the seasonal structure, consecutive observations are also related: if this is the case, there should be a pattern of serial correlation amongst the elementsofthedisturbanceprocessε t.assumingthatε t is stationary and that it admits an ARMA representation φ p (L)ε t =θ q (L)ν t where ν t is a white noise process. Assembling all the components, it can be obtained that: φ p (L)Φ P ( L 12 ) 12 X t =Θ Q ( L 12 ) θ q (L)ν t.

32 Finally,X t mightbethefirst(orsecond)differenceofa non-stationaryprocessy t, thatis, X t = d y t,withd= {0,1,2}inmostcases. Thus, φ p (L)Φ P ( L 12 ) 12 d Y t =Θ Q ( L 12 ) θ q (L)ν t. This model is called Seasonal ARIMA(p,d,q) (P, D, Q) process. The process X t is causal provided the roots of φ p (L) andφ P ( L 12 ) lieoutsidetheunitcircle. Because of the interaction between the seasonal and nonseasonal part of the model, the covariance function can be quite complicated. Later on in the course we will provide some general guidelines for identifying SARIMA models from the sample correlation function of the data. However, it should be emphasized that a process should be detrended before determining the seasonal pattern. Otherwise the trend component can hide the seasonal part.

33 6 Other approaches to model nonstationary processes: structural breaks and long memory models. See Banerjee and Urga, "Modelling structural breaks, long memory and stock market volatility: an overview", Journal of Econometrics, 129, References Brockwell and Davis, Chapter 9. Hamilton, Chapter 15. Wei, Chapter 4.