Univariate time-series analysis

Transcription

1 Univariae ime-series analsis. Basic conceps A ime series conains observaions of he random variable Y a cerain poins of ime. This is ofen denoed as :,..., T T):,,...,. T. We have observaions onl for a finie number of periods ( o A special feaure of ime-series is ha we can observe onl a single realizaion of he underling process. In oher words, we canno jus redraw samples from he populaion, as our reali iself is he onl sample available o us. The use of sandard saisical mehods needs a fundamenal assumpion referred o as ficional populaion. Please do no make he misake of no aking his concep seriousl. Le us assume ou have daa on all OECD counries for he period 96-, and ou would like o esimae a model explaining heir economic performance. The quesion is if wha ou have is he populaion iself, or jus a sample. Some would sa ha i is he populaion, since he are ineresed in he OECD counries iself. The consequence is ha no saisical inference is needed, because if we observe he populaion, we do no need an saisical ess. Ye, even in ime-series analsis we rel heavil on saisical inference heor. The explanaion is ha wha we observe is jus one of he infinie possible scenarios. Under differen circumsances, he effec of, sa, invesmens could have been differen on per capia income. In oher words, we canno consider he daa we observe as he populaion iself. Reali is a realizaion (a draw from) an unknown process (Daa Generaing Process - DGP) ha we wish o undersand, uncover based on he observed realizaion. Anoher imporan feaure of ime series is ha he observaions have a fixed order, ou canno reorder he observaions as ou like such as in cross-secional daa. As a resul, serial correlaion mus be given proper aenion. We can characerize a ime series ( ) b hree fundamenal saisics: Expeced value ( E ( )), variance ( Var( ) E( E( )) ) and auo-covariance: Cov(, ) E ( E( ))( E( )) s s s s The informaion on auocorrelaion is summarized in he Correlogram, which is a useful graphical diagnosic ool. I consiss of wo measures: auocorrelaion funcion (ACF) and parial auocorrelaion funcion (PACF). Auocorrelaion a lag k, or he value of he Auocorrelaion funcion a lag k: ACF k, is defined as: ACF k k Cov(, k ) k Var( ) I is worhwhile o compare his wih he sandard formula for linear correlaion: Cor(, ) k Cov(, ) k Var( ) Var( ) k The are no he same, bu he should be reasonabl close provided is homoscedasic.

2 Useful o undersand ACF(k) is o see i as a regression coefficien. Le us ake he following regression: k wih IID(, ) The OLS esimaor for he slope coefficien is: ˆ Cov(, k ) ACF( k) Var( ) How o es is auocorrelaion is presen? We use wo pes of pormaneau ess (pormaneau es are hpohesis ess, where he H is exacl specified while he alernaive hpohesis is jus loosel defined.) These are referred o as Q ess. k k Box-Pierce: QBP ( k) Tq and Ljung-Box: qi i QLB( k) T( T ) T k i In case of no auocorrelaion (H: up o lag k all ACFs are null), boh saisics should follow a chisquared disribuion wih k degrees-of-freedom. The Ljung-Box saisics should be preferred in small samples. Parial Auocorrelaion Funcion (PACF): he auocorrelaion beween and -k wih he correlaion beween and all lags - -k- removed. This is he easies o see as a coefficien of a mulivariae regression:... k k Then β k is he PACF(k). Tha is, he relaionship beween and -k wih all lower order auocorrelaion capured. Le us see an empirical example from Greene 5-.gd in Grel. We esimae he correlogram of he inflaion rae: i

3 ACF for infl +-.96/T^ lag PACF for infl +-.96/T^ lag The ACF indicaes ha values as old as quarers (5 ears) sill have saisicall significan effec on he inflaion observed oda. Such series ofen referred o as having long-memor. This behaviour of he inflaion rae ma be a hin ha i is no saionar (see nex secion on his). The PACF is significan a he firs hree lags, which ma indicae an AR(3) srucure behind he observed series (ha is DGP ma be like his: 3 3 ) We also obain an alphanumerical version of he correlogram wih es saisics: I is visible ha he Q-sa rejecs he H a all lags.

4 . Saionari and non-saionari (pa exra aenion fundamenal conceps will be inroduced) When dealing wih ime-series, unlike wih cross-secional daa, we canno assume ha all characerisics (saisical momens) of he underling DGP are unique, in he sense ha he do no depend on ime. Ye, sandard mehods such as correlaion or linear regression were all based on he assumpion ha he DGPs probabili disribuion remain he same for he whole sample. Le us ake wo pracical examples. Cross-secional example: We wish o esimae he effec of childhood nuriion on he heigh of oung aduls. For his reason we draw a sample of. We have all reason o assume ha if we were o draw a sample of, he expeced value of he heighs would no change, he onl hing ha would change is he accurac of our esimaes. The reason is ha he underling process, he DGP for heighs remains he same for all individuals: is mean, and variance will be he same independenl of how large sample we choose. Lengh is hence saionar. The same applies o he explanaor variable, nuriion. Sandard saisical echniques can be used. Time-series example: We wish o esimae he effec of educaion on he per capia income of Hungar for he period Obviousl, since per capia income and educaion have rends, he expeced value would depend on he sample lengh chosen. The average educaion has a mean of 9.48 for he 97-6 period, while.9 for he 99-6 period. This is because i has a rend. The same applies o GDP per capia: is expeced value depends on he lengh of he sample, hence is saisical means are no unique. We canno argue hence ha all observaions are drawn from he same probabili disribuion, and sandard saisical echniques would lead o differen esimaes depending on he sample size or which period we ake. Obviousl, hese series are nonsaionar. Le us define saionari now. Saionari in he sric sense: A process is called sricl saionar if is probabili disribuion is independen of ime. In oher words: if we have a ime series observed beween ime and T, hen he join probabili disribuion of hese observaions should be he same as hose observed beween an +k and T+k. Wha we acuall sa here is ha he main characerisics of he ime series should remain he same whenever we observe i. This is however a heoreical concep because i can be esed hrough he saisical momens. I is more useful o formulae he definiion in erms of momens, which leads o he weak version of saionari. Saionari in he weak sense or covariance saionari defines saionar processes in erm of heir momens. This makes his definiion easier o work wih and esable. Time series is called covariance saionar if:. I has a finie mean: E ( ). (This means ha he expeced value should be independen of ime and finie). I has a finie variance: Var( )

5 3. Is auocovariance depends on he difference of he observaions (k) onl and is independen of ime: Cov(, ) Cov(, ) s k sk Bu how can we find ou if a series is saionar? Firs, here is some arihmeical wa o find his ou, provided we know he DGP. If his is no he case (usuall) we can rel on differen saisical esing mehods, called uni-roo ess. Box and Jenkins idenifies hree basic pe of saionar ime-series. Whie-noise (WN): Whie noise has he properies of he ideal disurbance (or error) erm. I has zero mean, i is homoscedasic and exhibis no serial correlaion whasoever. Time series ε is a whienoise if: E( ), for all k, and Cov(, ) if k. Var( ) k The whie noise is purel random, i conains no useful informaion whasoever. B above definiion, a whie noise is saionar. The correlogram of a random sandard normal variable (a whie-noise) k Moving average (MA): A moving average process of order q equals a (an opional) consan, whie noise erm, and he linear combinaion of he lagged errors from - o -q. q, WN(, ) an MA(q) process. j j j The expeced value of an MA(q) process is: q j j E E E, since his does no depend on, his condiion fulfils he j requiremens of saionari. The variance of an MA(q) process is: process has a finie variance. q q j j j j, ha is, as long as q is finie or j for all j, his j

6 The firs-order auocovariance of an MA() process is Cov(, ) E E E E and zero for higher orders. The ACF is:, k k, k The PACF can be obained b invering he MA() o an AR( ) process: L ( L) ( L) ko k or or Where he coefficiens are he respecive PACF coefficiens Correlogram of he MA() process:.8 Auoregressive (AR) model: An auoregressive process of order p equals a (an opional) consan, whie noise erm, and he linear combinaion of he lagged dependen variable from - o -p. p i i WN(, ) an AR(p) process. i The expeced value of an AR(p) process is: p E( ) ie( i ), provided p i i is finie. The variance of an AR(p) process is: i p i, he expeced value of an AR(p) process i

7 p i i p i i i p, ha is, as long as i, his process has a finie posiive variance. The auocovariance of an AR() process is easil obained b invering i o an infinie MA process: i ( L ) 3 3 i i ( ) (...) i i i L L L L k From his i is obvious ha he dnamic mulipliers are for all lags k. The covariance is: k i i k The ACF a lag k is: ACFk The PACF a lag k is obvious: q, if k PACF( k), if k Correlogram of he AR() process:.7 Be careful ha AR processes can also be non-saionar. 3. A general soluion of difference equaions (and he characerizaion of AR models) You should recognize ha auoregressive ime-series models are sochasic difference equaions. As a resul he can be reaed and solved so. Le us define an AR(p) model, where w is a sochasic forcing funcion. I ma or ma no include a consan erm, bu i does alwas include a random, IID disurbance erm (ε ) or innovaion which is assumed o be a whie noise process.

8 p w where w (.) ~ iid(, ) i i Even hough he behaviour of he above model can be described wih a recursive approach, hence b expressing as a funcion of he saring value and he pas innovaions, his ma become quie difficul as p is increasing. There is a beer wa o arrive a a general soluion. All difference equaion as in (.) can be wrien in a marix form: F (.) where, p p w, p p F, where F is called he companion marix and has dimension pxp. In case of p= we have: and w (.3) which is he same as a a ssem of equaions: w (.4) So basicall wha we do here is o ransform an p-order differenial equaion ino a ssem of p equaions. The grea advanage of doing so is ha we acuall ransformed a p order problem ino a firs order problem. For example a a P order equaion can be rewrien recursivel as follows: j F F F F F v j j (.5) Or ou can sar ou from period as well: F F F F F F 3 3 3

9 i F F vi i (.6) which is equivalen wih (.5), jus we changed he numbering of he forcing process from backward o forward. So when we wish o esimae he effec of an innovaion in period i on in period (i ), hen can simpl express i from (.6) as follows: F i i (.7) The quesion of course, wha is F -i? Forunael, since F is a quadraic marix, we can make use of he diagonalizaion. Hence F E E (.8), where E is he marix of eigenvecors, and Λ is a diagonal marix of eigenvalues. Since Λ is diagonal:. Hence, lim F onl if lim which is onl possible if all p eigenvalues of F are less han one. This is he condiion for saionari. If his is no he case, and if onl one of he roos (eigenvalues) equals one in absolue erms, he effec of an innovaion will never deca. Wha is more, i can even be explosive (increasing as we ge farher awa from he innovaion), if a leas one of he eigenvalues exceeds one. We can use he above form o simulae he behaviour of he process.6. (.9). Cop his small code ino R o find ou how: F<-marix(NA,,) F<-cbind( c(.6,),c(.,)) Y<-marix(NA,,) Y[,]<-c(,) ef<-eigen(f) for (i in :) { Y[i,]<-F%*%Y[i-,] [i]=i} plo(,y[,],pe="l",main="()=.6*(-)+.*(-)",lab="effec of uni innovaion on ()") You should obain he following graph:

10 Wha we did was simpl o assign some saring value in he vecor Y ( =, =), and we simpl used (.6) o esimae he value of a an period. Alernaivel we can solve he equaion (.9) so ha and.the general soluion o an p-order homogenous difference equaion is p i i i A (.). So if.6. hen he roos of he companion marix are.84 and -.4 (rounded o digis). The eigenvalues can be obained from he companion marix as follows: F.6. since Fv v F I v.6. FI F I.6. which is he characerisic polnomial. he roos (eigenvalues) are:.838 and Hence A.84 A(.4) is a general soluion. No we can go for a paricular soluion. if hen A A A(.84 (.4) ) If, hen A(.84 (.4)) A he paricular soluion is hen:.4 (.84 (.4) ) which can be also ploed in R:.4

11 x<-(/.4)*(.84^-(-.4)^) plo(,x,pe="l",main="()=.6*(-)+.*(-)",lab="effec of uni innovaion on ()") No surprisingl, his is he same as wha we simulaed from he marix form. Le us look a some oher cases. Example : non-saionar series wih real roos.8. F.8. The eigenvalues are and -., he general soluion is hence: A A (.) Now we look for he paricular soluion so ha and. A A A A A( (.) ) and A( (.)) A so he paricular soluion is:. ( (.) ). You can observe ha one of he eigenvalues (or characerisic roos) of he companion marix was one (uni-roo). and his assures ha he effec of he uni impulse in = will no fade awa.

12 effec of uni innovaion on () ()=.8*(-)+.*(-) 5 5 Example : Explosive series This is b no means usual in economics or social sciences. Ye, for he example s sake we can r his one:.9. The eigenvalues of F are.8, -.8, hence his process is explosive. The soluion if and is (.8 (.8) ).6 The graph reflecs he explosive naure of his model:

13 Example 3: Saionar process wih complex roo Le us have he following AR() model:.8.6 The eigenvalues of F are i and i. The process is saionar since he real pars of he roos are all smaller he one. Ye, complex roos has an addiional meaning: he process oscillaes. Forunael, all complex numbers z can be rewrien in a rigonomeric form: b z a bi r cos isin where an a and r z a bi Hence r.774 and.79 The general soluion when he roos are complex is: Ar cos( ) A r sin( ) Hence: A.774 cos(.79 ) A (.774) sin(.79 ) If and hen he paricular soluion: A.774Asin(.79) A.774sin(.79) he paricular soluion is hen: (.774) sin(.79 ).774sin(.79)

14 4. Lag operaors Lag operaors can make our life easier, when used properl. Le L be an operaor wih he following characerisics: L L( L ) L ha is mulipling b L creaes a ime series lagged b one period. Since. Generall: k L k The lag operaor is commuaive, disribuive and associaive: For example: ( L L ), 3 L L L 3, L The operaion: ( L) is called firs-differencing. An imporan operaion is inversion: 3 3 L L L L... ( L) provided (he series a he righ-hand side is absolue summable). Proof: mulipl boh sides b L : n n L L L L L L L L L L L L lim if, he equali holds. n n If L n Bear in mind ha he inversion was rue onl if he effec of an observaion in he disan pas on curren value was negligible. This is wha we expec from saionar series. Onl saionar series are inverible. The above finding is known as Wold s heorem. An saionar ime-series can be expressed as an infinie MA series. This is a fundamenal resul in saisics. A pracical version is he inversion of a saionar AR process o an infinie MA process. wih lag operaors his is equivalen wih: ( L ). Mulipling Le boh sides b he inverse ( L) ields: ( L) L L L i i i i i i i i, if We can use he lag operaor form as well o find he characerisic roos of a difference equaion. A p-order auoregressive process is saionar if all he roos (soluions) of he following polnomial:

15 p z z pz lie ouside he uni circle (if he roos are real (no complex) i means ha all roos should be larger han one in absolue value). Some examples: Ex.:.5 he polnomial form is: (.5 L ) The characerisic equaion is:.5z, where he roo is: z so his AR() model is saionar. Ex.:.5.3 he polnomial form is: (.5.3 ) L L The characerisic equaion is:.5z.3z, where he roos are: z, z.84, z.7 Boh roos exceed in absolue erms, he above AR() model is saionar. Ex.3:.4. he inverse polnomial form is: (.4. ) L L The characerisic equaion is: , where he roos are: z,..4.4z.z There is a problem now because he expression under he square roo is negaive. The roos are complex in his case. No worr: i.8i( i ) z So:,.4.8i ithe absolue value of hese complex numbers is is modulus:.4 i i 5.36, which is larger han one. The process is saionar.

16 This means ha he process will reurn o is mean in an oscillaing wa. I is because of poeniall complex roos ha we menion uni circle in he definiion. Simpl, he modulus looks like an equaion of a circle. Wha he definiion required was ha if he complex roo was a+bi or a-bi hen a +b >. Tha is he modulus lies ouside of a circle wih uni radius. Do no ge confused. In secion 3 we were looking for he characerisic roos of he polnomial of an AR form, and we required ha all characerisic roos are wihin he uni circle. Wha we are looking for in secion 4, are he roos of he inverse polnomial, hence roos ouside he uni-circle are desired for saionar series. Le us look a he behaviour of he following series:.5.3 u B he firs mehod (using he eigenvalues of he companion marix) we obain wo complex eigenvalues (roos):.5±.487i. The modulus ( =.3) is less han one, hence he process is saionar. Le us rewrie he process wih lag operaors now! L L u The characerisic equaion is:.5z.3z, and he roos of he (.5.3 ) inverse polnomial are.83±.6i. The modulus ( =3.3) exceeds one, so he process is saionar. Observe ha he modulus from his equaion is over he modulus of he oher (deviaions are due o rounding errors).

17 .c Tesing for he order of inegraion B simpl looking a a series ou will no be able o decide if i is saionar or no or if i needs o be differenced. Le us ake he following wo series: a random walk wih drif (also called sochasic rend model) and a deerminisic rend model (also called rend saionar) The expeced value of boh series is going o be: E, provided he iniial value is zero. 6 Y 4 X Would ou be able o ell which of he above series are produced b a deerminisic or a sochasic rend model if ou had no been old he ruh? The hones is answer is no. You can guess bu ha is no enough. Acuall he disincion beween rend saionar and random walk wih drif series came ino he focus of debaes in macroeconomics during he 7s and remained here as a crucial poin of research unil he 9s. The radiional view on business ccles was moivaed b Kenesian models, where he long-run oupu was deermined b echnolog (suppl side) leading o some smooh growh of oupu, and business ccles were aribued o shor-run deviaions from equilibrium. This idea can be modeled b a rend saionar process: ln u, where denoes per capia income and he parameer δ is he growh rae of per capia income per period. The residual u has all he shocks ha would capure he deviaions from he long-run growh pah. In his model one would expec ha he shocks have jus an immediae effec so he econom should reurn o is long-run growh pah. If ou do no like his much, ou can assume ha i akes some ime o reurn o equilibrium, b making he model auoregressive:

18 ln ln u (< α <), bu he model is sill rend saionar. Kdland and Presco, followed b man, argued ha ccles ma acuall arise even in equilibrium. Tha is, he equilibrium oupu of he econom, ha had been believed o follow a smooh pah, ma also exhibi cclical behaviour. Because his means ha here are shocks in real (no nominal) variables, his school is called he Real Business Ccle School or RBC. This idea is bes capured b a sochasic rend model: ln ln u The econom will have a endenc o grow a a rae δ, bu an shocks (in his conex i is called echnological or producivi shock) could have a lasing effec. Remember: having a uni-roo means exacl he same. So ccles are no alwas signs of disequilibrium. This is he reason wh he aricle of Granger and Newbold in 974 became so influenial. People sared esing if macroeconomic series plaing a role in RBC heor (GDP, emplomen, TFP) were rend saionar or no. Dicke Fuller es: The firs and mos fundamenal uni-roo es (979). There are hree possible null-hpoheses:. The DGP is a random-walk: u wih u IID. (, u) In his case he es equaion is u, where if hen he null hpohesis is acceped, if hen i is rejeced. The es equaion is esimaed b OLS, bu he disribuion of he coefficien is no going o converge o normali so ou should no use Suden s -disribuion o find criical values. Insead criical values are supplied in a differen able or buil in ino some packages.. The DGP is eiher a random-walk wih a consan or a random-walk wih drif: u wih u IID(, ). In his case he es equaion is u u, where if hen he null hpohesis is acceped, if hen i is rejeced. 3. The DGP is a random-walk wih a consan and a deerminisic rend: u wih u IID(, u). In his case he es equaion is u, where if hen he null hpohesis is acceped, if hen i is rejeced. If we rejec he nullhpohesis bu find ha β, hen he series are rend-saionar. So ha he DF es works properl he residual should exhibi no serial correlaion. Bu his assumpion is oo bold. Le us see wh: You can observe ha he DF es uses an AR() model o es is a series is non-saionar. Bu wha if he real DGP is a higher-order AR model?

19 .7.3, his model is also non-saionar since he sum of he wo coefficiens equals one (alernaivel ou can esimae he roos ha are -.43 and ). Afer differencing he real DGP becomes:.3.3 If ou approximae his equaion b an AR() model ou will ge a coefficien close o zero bu he omied lags will cause an auocorrelaion in u. There are wo soluions:. Correcing he es saisics of he DF es for serial correlaion and possible heeroscedasici: his is he Phillips-Perron (PP) es.. Adding lags of he dependen variable in he es equaion of he DF es o capure auocorrelaion: Augmened Dicke-Fuller es (ADF es). The augmened Dicke Fuller es has he following es equaions: i i q q u, i u, i u Choosing he q can be done based on model selecion saisics, like he AIC or BIC. Smarer sofwares selec q for ou, based on some crieria. i Oher modificaions for he DF es are also available: he DF-GLS es for example, derend he series before running a sandard ADF procedure. The main problem of DF pe es is he low power. This means ha he probabili of Tpe II error is high. For he DF pe ess his means ha ou have a high chance ha when ou have an auoregressive DGP wih a high degree of posiive auocorrelaion, close o bu less han one, he es will no rejec he null hpohesis. Kwiakowski, Phillips, Schmid and Shin suggesed a es wih saionari as null hpohesis (KPSSes). The null hpoheses can be ha he series is eiher saionar or rend saionar. Depending on his he es equaions are: q i uˆ or uˆ The saisics is based on he residual: S uˆ which is he recursive sum of he residual. s s The es saisics is: T KPSS T S where is he esimaed long-run variance ofû. Using he uni roo ess, we can deermine he order of inegraion as follows:

20 . We carr ou a uni-roo es on he level of : if i is saionar hen he process is I(). If no, we ake he difference of he series.. We carr ou a uni-roo es on Δ. If i is saionar hen is I(), if no, we difference Δ furher. 3. The process is followed as long as i needed o achieve saionari. Be aware ha he uni-roo es suffer from a power problem: he end o accep he null hpohesis of non-saionari even when i is no rue, especiall if he auoregressive coefficien is close o one..d Overdifferencing A word of cauion: i is possible o overdifference he series, ha is, o ake he difference of a saionar series, which will lead o a special auoregressive paern. Overdifferencing will also increase he noise (measuremen errors and sampling error) relaive o signal, leading o a decrease in efficienc. Le us assume ha is saionar: wih IID(, ) u aking firs difference ields: x The firs consequence is obvious: So he differenced series is going o have higher variance. x The second consequence is ha x is going o be seriall correlaed: Cov( x, x ) E E Cov( x, x ) j.5, j x x Cov( x, x ) for all j> Tha is, if we obain a variable afer differencing ha has a firs order auocorrelaion around -.5 hen we have reason o believe ha we overdifferenced i. ACF and PACF of he firs difference of a sandard normal variable.

21 Noe: observe ha he ACF() is close o -.5.