Chapter 2 Fundamental Concepts

Transcription

1 Chpter 2 Fundmentl Concepts This chpter describes the fundmentl concepts in the theory of time series models In prticulr we introduce the concepts of stochstic process, men nd covrince function, sttionry process, nd utocorreltion function 2 Time Series nd Stochstic Processes The sequence of rndom vribles {Z t : t 0, ±, ±2, ±3,} is clled stochstic process It is known tht the complete probbilistic structure of such process is determined by the set of distributions of ll finite collections of the Z s Fortuntely, we will not hve to del explicitly with these multivrite distributions Much of the informtion in these joint distributions cn be described in terms of mens, vrinces, nd covrinces Consequently, we concentrte our efforts on these first nd second moments (If the joint distributions of the Z s re multivrite norml distributions, then the first nd second moments completely determine ll the joint distributions 22 Mens, Vrinces, nd Covrinces For stochstic process {Z t : t 0, ±, ±2, ±3,} the men function is defined by µ t EZ ( t for t 0, ±, ± 2, (22 tht is, µ t is just the expected vlue of the process t time t In generl µ t cn be different t ech time point t The utocovrince function, γ t,s, is defined s γ ts, Cov( Z t, Z s for t 0, ±, ± 2, (222 where Cov(Z t, Z s E[(Z t µ t (Z s µ s ] E(Z t Z s µ t µ s The utocorreltion function, ρ t,s, is given by ρ t, s Corr( Z t, Z s for t 0, ±, ± 2, where (223

2 Chpter 2 Fundmentl Concepts Corr( Z t, Z s Cov( Z t, Z s Vr( Z t Vr( Z s γ ts, γ t, t γ ss, (224 We review the bsic properties of expecttion, vrince, covrince, nd correltion in Appendix: Expecttion, Vrince, Covrince, nd Correltion on pge 6 Recll tht both covrince nd correltion re mesures of the (liner dependence between rndom vribles but tht the unitless correltion is somewht esier to interpret The following importnt properties follow from known results nd our definitions: γ t, t Vr( Z t γ t, s γ s, t γ ts, γ tt, γ s, s ρ tt, ρ ts, ρ s, t ρ ts, (225 Vlues of ρ t,s ner ± indicte strong (liner dependence, wheres vlues ner zero indicte wek (liner dependence If ρ t,s 0, we sy tht Z t nd Z s re uncorrelted To investigte the covrince properties of vrious time series models, the following result will be used repetedly: If c, c 2,, c m nd d, d 2,, d n re constnts nd t, t 2,, t m nd s, s 2,, s n re time points, then Cov m n m n c i Z ti, d j Z sj c i d j Cov( Z ti, Z sj i j i j (226 The proof of Eqution (226, though tedious, is strightforwrd ppliction of the liner properties of expecttion As specil cse, we obtin the following well-known result Vr n i c i Z ti The Rndom Wlk n n i c 2 i Vr ( Z ti + 2 c i c j Cov( Z ti, Z tj i i 2 j (227 Let, 2, be sequence of independent, identiclly distributed rndom vribles ech with zero men nd vrince σ2 The observed time series, {Z t : t, 2,}, is constructed s follows: 2

3 22 Mens, Vrinces, nd Covrinces Z Z Z t t (228 Alterntively, we cn write Z t Z t + t (229 If the s re interpreted s the sizes of the steps tken (forwrd or bckwrd long number line, then Z t is the position of the rndom wlker t time t From Eqution (228 we obtin the men function: so tht We lso hve µ t EZ ( t E ( t E ( + E ( E ( t µ t 0 for ll t (220 Vr( Z t Vr( t Vr( + Vr( Vr ( t σ2 + σ2 + + σ 2 t terms tσ2 so tht Vr( Z t tσ2 (22 Notice tht the process vrince increses linerly with time To investigte the covrince function, suppose tht t s Then we hve γ t, s Cov( Z t, Z s Cov( t, t + t s From Eqution (226 we hve γ t, s Cov( i, j i j However, these covrinces re zero unless i j in which cse they equl Vr( i σ2 There re exctly t of these so tht γ t,s t σ2 s t 3

4 Chpter 2 Fundmentl Concepts Since γ t,s γ s,t, this specifies the utocovrince function for ll time points t nd s nd we cn write γ t, s tσ2 for t s (222 The utocorreltion function for the rndom wlk is now esily obtined s γ ts, t ρ ts, γ tt, γ s ss, for t s (223 The following numericl vlues help us understnd the behvior of the rndom wlk 8 ρ 2, ρ 2 89, ρ 24, ρ 25 25, The vlues of Z t neighboring time points re more nd more strongly nd positively correlted s time goes by On the other hnd, the vlues of Z t distnt time points re less nd less correlted A simulted rndom wlk is shown in Exhibit (2 where the s were selected from stndrd norml distribution Note tht even though the theoreticl men function is zero for ll time points, the fcts tht the vrince increses over time nd tht the correltion between process vlues nerby in time is nerly indicte tht we should expect long excursions of the process wy from the men level of zero The simple rndom wlk process provides good model (t lest to first pproximtion for phenomen s diverse s the movement of common stock prices, nd the position of smll prticles suspended in fluid (so-clled Brownin motion 4

5 22 Mens, Vrinces, nd Covrinces Exhibit 2 A Rndom Wlk 8 RndWlk 3-2 Index A Moving Averge As second exmple, suppose tht {Z t } is constructed s t + t Z t (224 2 where (s lwys throughout this book, the s re ssumed to be independent nd identiclly distributed with zero men nd vrince σ2 Here nd µ t EZ ( t E t + t E ( t + E ( t Vr( Z t Vr t + t Vr( t + Vr( t σ2 Also 5

6 Chpter 2 Fundmentl Concepts Cov( Z t, Z t Cov t + t t + t 2, Cov t, t + (, t t + ( 2, t Cov( t, t Cov( t, t (s ll the other covrinces re zero 025σ2 or Furthermore, γ t, t 025σ2 for ll t Cov( Z t, Z t 2 Cov t + t t 2 + t 3, since the s re independent Similrly, Cov(Z t, Z t k 0 for k >, so we my write (225 For the utocorreltion function we hve γ t, s 025σ2 for t s 05σ2 for t s 0 0 for t s > for t s 0 ρ ts, 05 for t s 0 for t s > (226 since 025 σ2 /05 σ2 05 Notice tht ρ 2, ρ 3,2 ρ 4,3 ρ 9,8 05 Vlues of Z precisely one time unit prt hve exctly the sme correltion no mtter where they occur in time Furthermore, ρ 3, ρ 4,2 ρ t,t 2 nd, more generlly, ρ t,t k is the sme for ll vlues of t This leds us to the importnt concept of sttionrity 6

7 23 Sttionrity 23 Sttionrity To mke sttisticl inferences bout the structure of stochstic process on the bsis of n observed record of tht process, we must usully mke some simplifying (nd presumbly resonble ssumptions bout tht structure The most importnt such ssumption is tht of sttionrity The bsic ide of sttionrity is tht the probbility lws tht govern the behvior of the process do not chnge over time In sense, the process is in sttisticl equilibrium Specificlly, process {Z t } is sid to be strictly sttionry if the joint distribution of Z t, Z t2,, Z tn is the sme s the joint distribution of Z t k, Z t2 k,, Z tn k for ll choices of time points t, t 2,, t n nd ll choices of time lg k Thus, when n the (univrite distribution of Z t is the sme s tht of Z t k for ll t nd k; in other words, the Z s re (mrginlly identiclly distributed It then follows tht E(Z t E(Z t k for ll t nd k so tht the men function is constnt for ll time Additionlly, Vr(Z t Vr(Z t k for ll t nd k so tht the vrince is lso constnt in time Setting n 2 in the sttionrity definition we see tht the bivrite distribution of Z t nd Z s must be the sme s tht of Z t k nd Z s k from which it follows tht Cov(Z t, Z s Cov(Z t k, Z s k for ll t, s, nd k Putting k s nd then k t, we obtin γ ts, Cov( Z t s, Z 0 Cov( Z 0, Z s t Cov( Z 0, Z t s γ 0, t s tht is, the covrince between Z t nd Z s depends on time only through the time difference t s nd not otherwise on the ctul times t nd s Thus for sttionry process we cn simplify our nottion nd write γ k Cov( Z t, Z t k nd ρ k Corr( Z t, Z t k (23 Note lso tht γ k ρ k ---- γ 0 The generl properties given in Eqution (225 now become γ 0 Vr( Z t ρ 0 γ k γ k ρ k ρ k γ γ 0 ρ k (232 7

8 Chpter 2 Fundmentl Concepts If process is strictly sttionry nd hs finite vrince, then the covrince function must depend only on the time lg A definition tht is similr to tht of strict sttionrity but is mthemticlly weker is the following: A stochstic process {Z t } is sid to be wekly (or second-order sttionry if The men function is constnt over time, nd 2 γ t, t k γ 0, k for ll time t nd lg k In this book the term sttionry when used lone will lwys refer to this weker form of sttionrity However, if the joint distributions for the process re ll multivrite norml distributions, it cn be shown tht the two definitions coincide White Noise A very importnt exmple of sttionry process is the so-clled white noise process which is defined s sequence of independent, identiclly distributed rndom vribles { t } Its importnce stems not from the fct tht it is n interesting model itself but from the fct tht mny useful processes cn be constructed from white noise The fct tht { t } is strictly sttionry is esy to see since Pr( t x, t2 x 2,, tn x n Pr( t x Pr( t2 x 2 Pr( tn x n (by independence Pr( t k x Pr( t2 k x 2 Pr( tn k x n (identicl distributions Pr( t k x, t2 k x 2,, tn k x n (by independence s required Also µ t E( t is constnt nd γ k Vr( t 0 for k 0 for k 0 Alterntively, we cn write ρ k 0 for k 0 for k 0 (233 The term white noise rises from the fct tht frequency nlysis of the model (not considered in this book shows tht, in nlogy with white light, ll frequencies enter eqully We usully ssume tht the white noise process hs men zero nd denote Vr( t by σ2 8

9 23 Sttionrity The moving verge exmple, pge 8, where Z t ( t + t /2, is nother exmple of sttionry process constructed from white noise In our new nottion, we hve for the moving verge process tht ρ k 05 0 for k 0 for k ± for k 2 Rndom Cosine Wve As somewht different exmple, consider the process defined s follows: where Φ is selected (once from uniform distribution on the intervl from 0 to A smple from such process will pper highly deterministic, since Z t will repet itself identiclly every 2 time units nd look like perfect (discrete time cosine curve However, its mximum will not occur t t 0 but will be determined by the rndom phse Φ Still, the sttisticl properties of this process cn be computed s follows: So µ t 0 for ll t Also Z t cos 2π t + Φ for t 0, ±, ± 2, 2 EZ ( t E 2π t + Φ cos 2 2π t + φ cos sin 2π t + φ 2π 2 dφ sin 2π----- t + 2π 2π 2 φ 0 sin 2π----- t 2 0 (since the sines must gree This exmple contins optionl mteril tht is not needed in order to understnd the reminder of this book 9

10 Chpter 2 Fundmentl Concepts γ ts, E 2π t + Φ 2π s + Φ cos cos 2 2 cos 2π t + φ cos 2π s + φ So the process is sttionry with utocorreltion function dφ -- cos 2π t s 2π t s + 2φ 2 + cos cos 2π t s cos 2π t s 2 2 ρ k sin 2π t s + 2φ 4π 2 cos 2π----- k 2 for k 0, ±, ± 2, dφ φ 0 (234 This exmple suggests tht it will be difficult to ssess whether or not sttionrity is resonble ssumption for given time series on the bsis of the time plot of the observed dt The rndom wlk of pge 2, where Z t t, is lso constructed from white noise but is not sttionry For exmple, Vr(Z t t σ2 is not constnt; furthermore, the covrince function γ t, s tσ2 for 0 t s does not depend only on time lg However, suppose tht insted of nlyzing {Z t } directly, we consider the differences of successive Z-vlues, denoted Z t Then Z t Z t Z t t, so the differenced series, { Z t }, is sttionry This represents simple exmple of technique found to be extremely useful in mny pplictions Clerly, mny rel time series cnnot be resonbly modeled by sttionry processes, since they re not in sttisticl equilibrium but re evolving over time However, we cn frequently trnsform nonsttionry series into sttionry series by simple techniques such s differencing Such techniques will be vigorously pursued in the remining chpters 0

11 24 Exercises 24 Exercises 2 Suppose E(X 2, Vr(X 9, E(Y 0, Vr(Y 4, nd Corr(X,Y 025 Find: ( Vr(X + Y (b Cov(X, X + Y (c Corr(X + Y, X Y 22 If X nd Y re dependent but Vr(X Vr(Y, find Cov(X + Y, X Y 23 Let X hve distribution with men µ nd vrince σ 2 nd let Z t X for ll t ( Show tht {Z t } is strictly nd wekly sttionry (b Find the utocovrince function for {Z t } (c Sketch typicl time plot of Z t 24 Let { t } be zero men white noise processes Suppose tht the observed process is Z t t + θ t where θ is either 3 or /3 ( Find the utocorreltion function for {Z t } both when θ 3 nd when θ /3 (b You should hve discovered tht the time series is sttionry regrdless of the vlue of θ nd tht the utocorreltion functions re the sme for θ 3 nd θ /3 For simplicity, suppose tht the process men is known to be zero nd the vrince of Z t is known to be You observe the series {Z t } for t, 2,, n nd suppose tht you cn produce good estimtes of the utocorreltions ρ k Do you think tht you could determine which vlue of θ is correct (3 or /3 bsed on the estimte of ρ k? Why or why not? 25 Suppose Z t 5 + 2t + X t where {X t } is zero men sttionry series with utocovrince function γ k ( Find the men function for {Z t } (b Find the utocovrince function for {Z t } (c Is {Z t } sttionry? (Why or why not? X t 26 Let {X t } be sttionry time series nd define Z t X t + 3 ( Show tht Cov( Z t, Z t k is free of t for ll lgs k (b Is {Z t } sttionry? for t odd for t even

12 Chpter 2 Fundmentl Concepts 27 Suppose tht {Z t } is sttionry with utocovrince function γ k ( Show tht W t Z t Z t Z t is sttionry by finding the men nd utocovrince function for {W t } (b Show tht U t 2 Z t [Z t Z t ] Z t 2Z t + Z t 2 is sttionry (You need not find the men nd utocovrince function for {U t } 28 Suppose tht {Z t } is sttionry with utocovrince function γ k Show tht for ny fixed positive integer n nd ny constnts c, c 2,, c n, the process {W t } defined by W t c Z t + c 2 Z t + + c n Z t n is sttionry (Note tht Exercise (27 is specil cse of this result 29 Suppose Z t β 0 + β t + X t where {X t } is zero men sttionry series with utocovrince function γ k nd β 0 nd β re constnts ( Show tht {Z t } is not sttionry but tht W t Z t Z t Z t is sttionry (b In generl, show tht if Z t µ t + X t where {X t } is zero men sttionry series nd µ t is polynomil in t of degree d, then m Z t ( m Z t is sttionry for m d nd nonsttionry for 0 m < d 20 Let {X t } be zero-men, unit-vrince sttionry process with utocorreltion function ρ k Suppose tht µ t is nonconstnt function nd tht σ t is positive-vlued nonconstnt function The observed series is formed s Z t µ t + σ t X t ( Find the men nd covrince function for the {Z t } process (b Show tht utocorreltion function for the {Z t } process depends only on time lg Is the {Z t } process sttionry? (c Is it possible to hve time series with constnt men nd with Corr(Z t,z t k free of t but with {Z t } not sttionry? 2 Suppose Cov(X t,x t k γ k is free of t but tht E(Z t 3t ( Is {X t } sttionry? (b Let Z t 7 3t + X t Is {Z t } sttionry? 22 Suppose tht Z t t t 2 Show tht {Z t } is sttionry nd tht, for k > 0, its utocorreltion function is nonzero only for lg k 2 23 Let Z t t θ( t 2 ( Find the utocorreltion function for {Z t } (b Is {Z t } sttionry? 2

13 24 Exercises 24 Evlute the men nd covrince function for ech of the following processes In ech cse determine whether or not the process is sttionry ( Z t θ 0 + t t (b W t Z t where Z t is s given in prt ( (c Z t t t 25 Suppose tht X is rndom vrible with zero men Define time series by Z t ( t X ( Find the men function for {Z t } (b Find the covrince function for {Z t } (c Is {Z t } sttionry? 26 Suppose Z t A + X t where {X t } is sttionry nd A is rndom but independent of {X t } Find the men nd covrince function for {Z t } in terms of the men nd utocovrince function for {X t } nd the men nd vrince of A n 27 Let {Z t } be sttionry with utocovrince function γ k Let Z -- Z Show n t t tht γ n 0 2 k Vr( Z γk n n n k n n -- k ---- γk n k n + 28 Let {Z t } be sttionry with utocovrince function γ k Define the smple vrince n s S Z n ( t Z 2 t n n ( First show tht ( Z t µ 2 ( Z t Z 2 + nz ( µ 2 t t (b Use prt ( to show tht n ES ( 2 n n 2 k γ n Vr( Z γ n γk n n k (Use the results of Exercise (27 for the lst expression (c If {Z t } is white noise with vrince γ 0, show tht E(S 2 γ 0 3

14 Chpter 2 Fundmentl Concepts 29 Let Z θ 0 + nd then for t > define Z t recursively by Z t θ 0 + Z t + t Here θ 0 is constnt The process {Z t } is clled rndom wlk with drift ( Show tht Z t my be rewritten s Z t tθ 0 + t + t ++ (b Find the men function for Z t (c Find the utocovrince function for Z t 220 Consider the stndrd rndom wlk model where Z t Z t + t with Z ( Use the bove representtion of Z t to show tht µ t µ t for t > with initil condition µ E( 0 Hence show tht µ t 0 for ll t (b Similrly, show tht Vr(Z t Vr(Z t + σ2, for t > with Vr(Z σ2, nd, hence Vr(Z t t σ2 (c For 0 t s, use Z s Z t + t+ + t+2 ++ s to show tht Cov(Z t, Z s Vr(Z t nd, hence, tht Cov(Z t, Z s min(t, s σ2 22 A rndom wlk with rndom strting vlue Let Z t Z 0 + t + t ++ for t > 0 where Z 0 hs distribution with men µ 0 nd vrince σ2 0 Suppose further tht Z 0,,, t re independent ( Show tht E(Z t µ 0 for ll t (b Show tht Vr(Z t t σ 2 + σ2 0 (c Show tht Cov(Z t, Z s min(t, s σ2 + σ2 0 tσ2 + σ2 0 (d Show tht Corr( Z t, Z s for 0 t s sσ2 + σ Let { t } be zero-men white noise process nd let c be constnt with c < Define Z t recursively by Z t cz t + t with Z ( Show tht E(Z t 0 (b Show tht Vr(Z t σ2 ( + c 2 +c 4 ++ c 2t 2 Is {Z t } sttionry? (c Show tht nd, in generl, Corr( Z t, Z t c Vr ( Z t Vr( Z t Corr( Z t, Z t k c k Vr ( Z t k for k > 0 Vr( Z t 4

15 24 Exercises (d For lrge t, rgue tht σ2 Vr( Z t c 2 nd Corr( Z t, Z t k c k for k > 0 so tht {Z t } could be clled symptoticlly sttionry (e Suppose now tht we lter the initil condition nd put Z Show c 2 tht now {Z t } is sttionry 223 Two processes {Z t } nd {Y t } re sid to be independent if for ny time points t, t 2,, t m nd s, s 2,, s n, the rndom vribles { Z t, Z t2,, Z tm } re independent of the rndom vribles { Z s, Z s2,, Z sn } Show tht if {Z t } nd {Y t } re independent sttionry processes, then W t Z t + Y t is sttionry 224 Let {X t } be time series in which we re interested However, becuse the mesurement process itself is not perfect, we ctully observe Z t X t + t We ssume tht {X t } nd { t } re independent processes We cll X t the signl nd t the mesurement noise or error process If {X t } is sttionry with utocorreltion function ρ k, show tht {Z t } is lso sttionry with ρ k Corr( Z t, Z t k σ 2 for k + σ2 X We cll σ 2 X σ2 the signl-to-noise rtio, or SNR Note tht the lrger the SNR, the closer the utocorreltion function of the observed process {Z t } is to the utocorreltion function of the desired signl {X t } k 225 Suppose Z t β 0 + [ A i cos( 2πf i t + B i sin( 2πf i t ] where β 0, f, f 2,, f k re i constnts nd A, A 2,, A k, B, B 2,, B k re independent rndom vribles with zero mens nd vrinces Vr(A i Vr(B i σ2 i Show tht {Z t } is sttionry nd find its covrince function 5

16 Chpter 2 Fundmentl Concepts 226 Define the function Γ t, s --E[( Z In geosttistics, Γ t,s is clled the 2 t Z s 2 ] semivriogrm ( Show tht for sttionry process Γ t, s γ 0 γ t s (b A process is sid to be intrinsiclly sttionry if Γ t,s depends only on the time difference t s Show tht the rndom wlk process is intrinsiclly sttionry 227 For fixed, positive integer r nd constnt φ, consider the time series defined by Z t t + φ t + φ 2 t 2 ++ φ r t r ( Show tht this process is sttionry for ny vlue of φ (b Find the utocorreltion function 228 (Mthemticl sttistics required Suppose tht Z t Acos[ 2π( ft + Φ ] for t 0, ±, ± 2, where A nd Φ re independent rndom vribles nd f is constnt the frequency The phse Φ is ssumed to be uniformly distributed on (0,, nd the mplitude A hs Ryleigh distribution with pdf f ( e 2 2 for > 0 ( Show tht {Z t } is sttionry (b Show tht for ech time point t, Z t hs norml distribution (It cn lso be shown tht ll of the finite dimensionl distributions re multivrite norml so tht the process is strictly sttionry (Hint: Let X Acos[ 2π( ft + Φ ] nd Y Asin[ 2π( ft + Φ ] nd find the joint distribution of X nd Y 25 Appendix: Expecttion, Vrince, Covrince, nd Correltion In this ppendix we define expecttion for continuous rndom vribles However, ll of the properties described hold for ll types of rndom vribles Let X hve probbility density function f(x nd let the pir (X,Y hve joint probbility density function f(x,y The expected vlue of X is defined s EX ( xf( x dx (If xf( x dx < ; otherwise E(X is undefined E(X is lso clled the expecttion of X or the men of X nd is often denoted µ or µ X 6

17 25 Appendix: Expecttion, Vrince, Covrince, nd Correltion Properties of Expecttion If h(x is function such tht If, then As corollry to Eqution ( we hve the importnt result EX ( + by+ c E( X + be( Y + c Also EXY ( xyf( x, y dxdy The vrince of rndom vrible X is defined s EhX ( ( hx ( fx ( dx hx ( fx ( dx < hxy (,fxy (, dxdy <, then EhXY [ (, ] hxy (, fxy (, dxdy Vr( X E{ [ X E( X ] 2 } (2A (2A2 (2A3 (2A4 (2A5 (provided E(X 2 exists The vrince of X is often denoted σ 2 or σ2 X Properties of Vrince Vr( X 0 Vr( + bx b 2 Vr( X If X nd Y re independent, then Vr( X + Y Vr( X + Vr( Y (2A6 (2A7 (2A8 Vr( X EX ( 2 [ EX ( ] 2 (2A9 The positive squre root of the vrince of X is clled the stndrd devition of X nd is often denoted σ or σ X The rndom vrible (X µ X /σ X is clled the stndrdized version of X The men nd stndrd devition of stndrdized vrible re lwys zero nd one, respectively The covrince of X nd Y is defined s Cov( X, Y E[ ( X µ X ( Y µ Y ] 7

18 Chpter 2 Fundmentl Concepts Properties of Covrince If X nd Y re independent Cov( X, Y 0 (2A0 Cov( + bx, c + dy bdcov( X, Y Vr( X + Y Vr( X + Vr( Y + 2Cov( X, Y Cov( X + Y, Z Cov( X, Z + Cov( Y, Z Cov( X, X Vr( X (2A (2A2 (2A3 (2A4 The correltion coefficient of X nd Y, denoted Corr(X,Y or ρ, is defined s Cov( X, Y Corr( X, Y Vr( XVr( Y (2A5 Alterntively, if X* is stndrdized X nd Y* is stndrdized Y, then ρ E(X*Y* Properties of Correltion Cov( X, Y Cov( Y, X Corr( X, Y (2A6 Corr( + bx, c + dy sign( bdcorr( X, Y where sign( bd Corr(X,Y ± if bd > 0 0 if bd 0 if bd < 0 if, nd only if there re constnts nd b such tht Pr(Y + bx (2A7 (2A8 8