Time Series, Par Coe - Saioariy, auocorrelaio, parial auocorrelaio, removal of osaioary compoes, idepedece es for ime series - Liear Sochasic Processes: auoregressive (AR), movig average (MA), auoregressive movig average (ARMA) - Fi of models AR, MA ad ARMA o saioary ime series - Liear models for o-saioary ime series - Predicio of ime series - Noliear aalysis of ime series wih sochasic models - Noliear aalysis of ime series ad dyamical sysems Lieraure - The Aalysis of Time Series, A Iroducio, Chafield C., Sixh ediio, Chapma & Hall, 4 - Iroducio o ime series ad forecasig, Brockwell P.J. ad Davis R.A., Secod ediio, Spriger, - No-Liear Time Series, A Dyamical Sysem Approach, Tog H., Oxford Uiversiy Press, 993 - Noliear Time Series Aalysis, Kaz H. ad Schreiber T., Cambridge Uiversiy Press, 4
physiology Real world ime series uivariae ime series mechaics elecroics oly oe ime series limied legh geophysics ecoomy o-saioariy oise
observed quaiy variable Χ Defiiios / oaios The values of he observed quaiy chage wih radomess (sochasiciy) a some small or larger degree radom variable (r.v.) Χ The observaios ake place mos ofe a fixed ime seps samplig ime For each ime poi we cosider he value x of he r.v. Χ The se of he values of x over a ime period (give i uis of he samplig ime) (uivariae) ime series x { x, x,, x } If here are simulaeous observaios of more ha oe variable mulivariae ime series We apply mehods ad echiques o he give uivariae or mulivariae ime series i order o ge isigh for he sysem ha geeraes i ime series aalysis The ime series ca be cosidered as realizaio of a sochasic or deermiisic process (dyamical sysem) X
close idex volume close idex close idex Exchage idex ad volume of he Ahes Sock Exchage (ASE) 7 ASE idex, period 985-6 ASE idex, period 7-6 5 5 4 4 3 3 86 88 9 9 94 96 98 4 6 8 ASE idex, period 8 7 8 9 5 x 5 ASE volume, period 998-8 6 4 5 8 6 3 4 5 6 7 8 9 mohs 98 99 3 4 5 6 7 8 Predicio? Wha is he idex value omorrow? The day afer? Dyamical sysem? sochasic process? Wha is he mechaism of he Greek sock marke?
Geeral Idex of Comsumer Prices Geeral Idex of Cosumer Prices (GICP) Geeral Idex of Comsumer Prices, period Ja - Aug 5 5 5 5 3 4 5 6 Tred? Seasoaliy / periodiciy? Auocorrelaio? Auoregressio? Predicio?
umber of suspos umber of suspos umber of suspos Aual suspo umbers 5 5 5 Aual suspos, period 7-7 75 8 85 9 95 5 Aual suspos, period 96-995 8 6 4 8 6 4 Aual suspos, period 9-9 9 94 96 98 Wha is he mechaism / sysem / process ha geeraes suspos? Is i a periodic sysem + oise? Is i a sochasic sysem? Is i a chaoic sysem? 96 97 98 99 Give he suspo umber for up o 995, wha is he suspo umber i 996? ad he afer? Wha will be he suspo umber i 3, 4?
suspo umber Model compariso Geuie predicio Geuie predicios of suspo daa 5 5 995 5 5 year
Wha is he geeraig sysem of a real ime series? x(i) preical EEG Real ime series ical EEG Cadidae sochasic models sochasic 3 4 5 ime i secods 3 4 5 ime i secods 4 ime idex i Cadidae deermiisic models periodic + oise low dimesioal chaos high dimesioal chaos 3 4 5 ime i secods 3 4 5 ime i secods 3 4 5 ime i secods
Drippig waer fauce (origial experime a UC Saa Cruz). x x x 3 ( x3, x) ( x, x ) The observaio of he drippig fauce shows ha for some flow velociy he drops do o ru a cosa ime iervals. Cruchfield e al, Scieific America, 986 The scaer diagram of he daa showed ha he drop flow is o radom. scaer diagram i i ( x, x ) ( xi, xi, xi ) Héo map s.4s.3s i i i observed variable x s w w i oise i i i chaos
AE idex Geeral Idex of Comsumer Prices Geeral Idex of Comsumer Prices, period Ja - Aug 5 5 No-saioariy 5 Tred? Seasoaliy / periodiciy? 5 3 4 5 6 4 Auroral Elecroje Idex 8 6 Volailiy? 4 Auocorrelaio? 5 5 5 3 35 4 45 5 ime [ mi]
AE idex AE idex 4 8 y, y,, y Auroral Elecroje Idex Variace sabilizig rasformaio Trasform Χ =T(Υ ) ha sabilizes he variace of Υ? Var[ X ] cos simple soluio: X log( Y)? 6 4 5 5 5 3 35 4 45 5 ime [ mi] 8 7 6 5 4 x, x,, x 3 5 5 5 3 35 4 45 5 ime [ mi] Logarihm rasform of Auroral Elecroje Idex Power rasform (Box-Cox): X Y Assumpio: Var[Υ ] chages as a fucio of he red μ λ Χ Var[y ] - -.5.5 Y Y log( Y ) Y Oher rasforms? 4 c 3 c c c?
AE idex f Z (z) AE idex f Y (y) AE idex f X (x) 4 Auroral Elecroje Idex 5 x -3 8 6 4 3 y ormal y, y,, y 4 5 5 5 3 35 4 45 5 ime [ mi] Logarihm rasform of Auroral Elecroje Idex 8 - -5 5 5 x.5 7.4 6 5.3. x=log(y) ormal X log( Y) 4. 3 5 5 5 3 35 4 45 5 ime [ mi] Gaussia rasform of Auroral Elecroje Idex 4 3 - - -3-4 5 5 5 3 35 4 45 5 ime [ mi] 4 6 8 y.5.4.3.. x= - (F Y (y)) ormal -4-4 z X F ( Y ) Y?
idex ime series y, y,, y Saioariy - red Tred: slow chage of he successive values of y deermiisic red: a fucio of ime μ = f() Plasic deformaio 6 S&P5 sochasic red: radom slow chage μ 4 8 6 4 8 85 87 9 9 95 97 5
Removal of red Y X. Deermiisic red : kow or esimaed fucio of ime μ = f() Example: polyomial of degree p p f () a a a p Plasic deformaio μ : mea value as fucio of (slowly varyig mea level) X Y {X } saioary Fi wih firs degree polyomial Fi wih fifh degree polyomial
close idex close idex close idex. Sochasic red 7 6 5 5 4 4 3 3 Idex of he Ahes Sock Exchage (ASE) ASE idex, period 985 -- orig orig local liear, breakpois polyomial,p= - 86 88 9 9 94 96 98 4 6 8 α. Smoohig wih movig average filer Simple filer: movig average 7 6 5 4 3 orig MA(3) MA(5) ˆ ASE idex, period 985-6 5 4 3 ASE idex, period 7-7 8 9 q y q 3 ˆ y y y q 3 3 3 j jq 86 88 9 9 94 96 98 4 6 8 "q " 4? More geeral filer: movig weighed average q ˆ ay a j j jq q jq Simple movig average: a j, j q,, q q j
close idex close idex b. Tred removal wih differecig Oe lag differece or firs differece Y Y Y ( B) Y B: lag operaor BY Y Secod order lag differece Y ( Y ) ( B)( B) Y ( B B ) Y Y Y Y If he red is locally liear, i is removed by firs differeces: Y Y Y X X aa a a a a ( ) a cosa! If he red is locally polyomial or degree p, i is removed by usig p Y? p Y p! c X [show firs: ] 4 ASE idex: firs differeces, period 7-4 ASE idex: firs differeces, period 7 - - - -4-4 -6 86 88 9 9 94 96 98 4 6 8-6 8
close idex close idex close idex close idex close idex Which mehod for red removal is bes? 6 5 4 orig local liear, breakpois polyomial,p= ASE idex, period 7-5 ASE idex dereded, period 7 - local liear, breakpois polyomial,p= 3 5-5 - 8 8 6 5 4 orig MA(3) MA(5) ASE idex, period 7-5 ASE idex dereded, period 7 - MA(3) MA(5) 3 5-5 - 8 8 Esimaio of red 5 ASE idex: firs differeces, period 7-5 -5-8
differece of logs idex relaive chage firs differece more o differecig rasform 6 4 8 6 4 y, y,, y - ime series 8 85 87 9 9 95 97 5 S&P5 8 85 87 9 9 95 97 5 5-5..5 -.5 -. -.5 -. S&P5, firs differeces S&P5, relaive chages -.5 8 85 87 9 9 95 97 5 S&P5, differece of logs..5 -.5 -. -.5 -. -.5 8 85 87 9 9 95 97 5 chage of he value x y y relaive chage of he value y y x y chage of he logarihm of he value x l y l y
umber of suspos umber of suspos Removal of seasoaliy or periodiciy Y s X s : periodic fucio of wih period d 5 5 Aual suspos, period 7 - Aual suspos 7 75 8 85 9 95 Period d ad appropriae fucio s?. kow or esimaed periodic fucio s = f() 8 6 4 8 6 4 Aual suspos, period 9-9 9 94 96 98 X Y s {X } saioary a. Esimaio of s i i=,,d from he averages for each compoe Period d is kow k / d sˆ k y i i jd k j b. Removal of periodiciy usig lag differeces of order d (d-differecig) Y Y Y ( B d ) Y d d
year cycle of GICP residual GICP GICP dereded GICP Removal of red ad periodiciy Y s X Y Y s X. Removal of red X Y s Y s. Removal of periodiciy {Χ }: ime series of residuals Firs remove red ad he periodiciy or vice versa? 5 Geeral Idex of Comsumer GICP: Liear Prices, fiperiod /-8/5 3 GICP: Residual of liear fi 5-5 - 3 4 5 6 year GICP: Year cycle esimae 3-3 3 4 5 6 3 GICP: dereded ad deseasoed - - - - -3 3 4 5 6 year o-saioary y y y saioary,,, -3 3 4 5 6 year x, x,, x Is here iformaio i he residuals?
f Y (y) f X (x) idex firs differece Y : he value of he quaiy y, y,, y ime series Time correlaio Sochasic process Y X 6 S&P5 S&P5, firs differeces 4 8 6 5 chage of he value 4-5 x y y 8 85 87 9 9 95 97 5-8 85 87 9 9 95 97 5 3 fy ( y) 3.5 x -3 Gaussia pdf superimposed o S&P5 6 5 fx ( x) Gaussia pdf superimposed o S&P5 reurs Saic descripio margial disribuio.5.5 4 3 Dyamic descripio? Time correlaio.5 5 5 -.5.5 Y X
Disribuio ad momes of a sochasic process A sochasic process ca be fully described i erms of he margial ad joi probabiliy disribuios Z f ( y) f ( y, ) Y Y margial disribuio, Z f ( y, y ) f ( y, y,, ) Y, Y Y joi disribuio of r.v.,, 3 Z f ( y, y, y ) f ( y, y, y,,, ) Y, Y, Y 3 Y 3 3 3 joi disribuio of 3 r.v. Y yf y y Firs order mome (mea) (, )d Secod order mome Ceral secod order mome Higher order momes Y Y y y f ( y, y,, )dy d y (, ) Y Y ( Y )( Y ) (, ) (, ) auocovariace The probabiliy disribuio ad momes may chage i ime
Sric-sese saioariy Saioariy The disribuios do o chage wih ime (equivalely, all momes are cosa) Z,,, 3 Z Z f ( y) f ( y, ) f ( y) Y Y Y f ( y, y ) f ( y, y ) Y, Y Y, Y f ( y, y, y ) f ( y, y, y ) Y, Y, Y 3 Y, Y, Y 3 3 cosa Z Wide-sese saioariy The firs wo momes are cosa i ime Y YY, (, ) ( ) Y Y (, ) (, ) ( ) cosa Z cosa - mea - variace - auocovariace for τ= Y () cosa variace () Y () Y
Auocorrelaio Saioary ime series X Auocovariace X X X X X ( )( ) ( ) () Variace X X Auocorrelaio () () () ( ) ( ) a a lag τ. () Noaio: () () Commes: k k ad k k ad k k Auocovariace marix Time correlaio of variables of Measures he memory of X Auocorrelaio marix X
X Basic sochasic processes X idepede ad ideically disribued r.v. (iid) P( X x, X x,, X x ) P( X x ) P( X x) P( X x ) E X whie oise (WN), o-correlaed r.v. E XX i j ij 3 Y radom walk (RW) Y Y X X X X E Y Y, Y,, Y Y X iid E Y E Y E X E X? Variace icreases liearly wih ime!
Ucorrelaed (whie oise) ad idepede (iid) observaios Chafield C., The Aalysis of Time Series, A Iroducio, 6 h ediio, p. 38 (Chaper 3): Some auhors prefer o make he weaker assumpio ha he z s are muually ucorrelaed, raher ha idepede. This is adequae for liear, ormal processes, bu he sroger idepedece assumpio is eeded whe cosiderig o-liear models (Chaper ). Noe ha a purely radom process is someimes called whie oise, paricularly by egieers. p. (Chaper ): Whe examiig he properies of o-liear models, i ca be very impora o disiguish bewee idepede ad ucorrelaed radom variables. I Secio 3.4., whie oise (or a purely radom process) was defied o be a sequece of idepede ad ideically disribued (i.i.d.) radom variables. This is someimes called sric whie oise (SWN), ad he phrase ucorrelaed whie oise (UWN) is used whe successive values are merely ucorrelaed, raher ha idepede. Of course if successive values follow a ormal (Gaussia) disribuio, he zero correlaio implies idepedece so ha Gaussia UWN is SWN. However, wih o-liear models, disribuios are geerally o-ormal ad zero correlaio eed o imply idepedece. Wei W.W.C., Time Series Aalysis, Uivariae ad Mulivariae Mehods, p. 5:.4 Whie Noise Processes A process {a } is called a whie oise process if i is a sequece of ucorrelaed radom variables from a fixed disribuio wih cosa mea (usually assumed ), cosa variace ad zero auocovariace for lags differe from.
4 X For each order p: Gaussia (ormal) sochasic process f ( x, x,, x ) X, X,, X p p is p-dimesioal Gaussia disribuio Gaussia disribuio is compleely defied by he firs wo momes sric saioariy weak saioariy Example Sochasic process: Is he process weak saioary? E[ X ] E[ A]E[si( )] X Asi( ) A r.v. E[ A] Var[ A] ~ U[, ] E[ XX ] E A si( )si( ( ) )... cos( ) The firs ad secod order momes do o deped o ime.? θ ad A idepede
ime series Sample auocovariace / auocorrelaio Sample mea x, x,, x x x ubiased esimae of he mea μ of he ime series? Sample auocovariace Aoher esimae of auocovariace Biased esimaes: ( ) ( x ) () ( ) c() ( xx x ) c x x E[ c ] ( )Var[ x] E[ c] Var[ x] c x x,,, c( ) Sample auocorrelaio r( ) r() c() r ~ N(,Var[ r ]) For large : Var[ r ] ( m m m m 4 mm ) m Var[ r ] m very large m Noaio c() c bias icreases wih he lag τ Noaio r() r Barle formula
Auocorrelaio for whie oise x, x,, x whie oise ime series, r ~ N(, )? Tes for idepedece observed saioary ime series residual ime series afer red or periodiciy removal Η : x, x,, x Hypoheses Is i iid? Η Are here correlaios? x x x is iid Η : is whie oise,,, x, x,, x Saisical Sigificace es for auocorrelaio H : H : r R r z / Rejecio regio: / z Bad of isigifica auocorrelaio: a/ x, x,, x whie oise r N(, ) for sigificace level for =.5 Η
r() Numerical Example For a ime series of observaios, he auocorrelaio for τ=,, are: 3 4 5 6 7 8 9 -.38 -.8. -.8....7 -.8.5 Assume ha he ime series is purely radom (Η :ρ=): Var[ r ].5 for =.5, we expec 95% of auocorrelaios o be i he ierval.96.96.7.39 ρ, ρ και ρ τ για τ=3,4, Example of GICP.3.. GICP residual: auocorrelaio Sigificace es Η : for each idepedely -. -. -.3 -.4 5 5 A sigificace level =.5, Η is rejeced for τ= Is here ay correlaio i he GICP ime series?
r() The Pormaeau sigificace es Q(k) A es for each lag,,k Oe es for all lags ogeher? H :,,,k Tes saisic Q: ~ k Q k r ( k ) / ( ) Box-Pierce Q r j Q rejecio regio R Q k; a Ljug-Box.3. GICP residual: auocorrelaio k 35 3 GICP residual: Pormaeau (Ljug-Box). -. Q 4.6 k ; a 8.3 5 5 -. -.3 -.4 5 5 H for τ= is rejeced 5 5 5 k
x() r() x() r() radom ime series. radom ime series: auocorrelaio.8..6.4 -.. -. 3 4 5 6 7 8 9 logisic ime series -.3 4 6 8 logisic ime series: auocorrelaio..8.5..6.5.4 -.5. -. -.5 3 4 5 6 7 8 9 A appropriae sigificace es? -. 4 6 8
close idex r() Q(k) close idex r() Q(k) close idex r() Q(k) close idex 6 5 4 3 ASE idex, period 7-7 8 9 4 3 - - ASE idex: firs differeces, period 7 - Is here correlaio i he reurs ime series of he ASE idex (ime period 7-)? Wha is he appropriae saioary ime series: firs differeces or reurs? Is here correlaio? firs differeces x y y.8.6.4. -. -.4 ASE firs differeces: auocorrelaio oliear? 5 5 X X E ASE firs differeces: Pormaeau (Ljug-Box) sample Q X (k,) -3 7 8 9 ASE idex: reurs, period 7 -.5..5 -.5 -. -.5 -. 7 8 9 ASE idex: square reurs, period 7 -..5..5 7 8 9 reurs x l y l y square of reurs x l y l y x ( x) -.6 4 6 8 ASE reurs: auocorrelaio.8.6.4. -. -.4 -.6 -.8 4 6 8.3.5..5..5 -.5 ASE square reurs: auocorrelaio -. 4 6 8 4 6 8 k ASE reurs: Pormaeau (Ljug-Box) 5 5 sample Q X (k,) 4 6 8 k 35 3 5 5 5 ASE square reurs: Pormaeau (Ljug-Box) sample Q X (k,) 4 6 8 k