Linear Stochastic Models

Transcription

1 LECTURE 5 Lnear Stochastc Models Autcovarances of a Statonary Process A temporal stochastc process s smply a sequence of random varables ndexed by a tme subscrpt Such a process can be denoted by x(t) The element of the sequence at the pont t = τ s x τ = x(τ) Let {x τ+1,x τ+,,x τ+n }denote n consecutve elements of the sequence Then the process s sad to be strctly statonary f the ont probablty dstrbuton of the elements does not depend on τ regardless of the sze of n Ths means that any two segments of the sequence of equal length have dentcal probablty densty functons In consequence, the decson on where to place the tme orgn s arbtrary; and the argument τ can be omtted Some further mplcatons of statonarty are that (51) E(x t )=µ< for all t and C(x τ+t,x τ+s )=γ t s The latter condton means that the covarance of any two elements depends only on ther temporal separaton t s Notce that, f the elements of the sequence are normally dstrbuted, then the two condtons are suffcent to establsh strct statonarty On ther own, they consttute the condtons of weak or nd-order statonarty The condton on the covarances mples that the dsperson matrx of the vector [x 1,x,,x n ] s a bsymmetrc Laurent matrx of the form (5) Γ = γ γ 1 γ γ n 1 γ 1 γ γ 1 γ n γ γ 1 γ γ n 3 γ n 1 γ n γ n 3 γ wheren the generc element n the (, )th poston s γ = C(x,x ) Gven that a sequence of observatons of a tme seres represents only a segment of a sngle realsaton of a stochastc process, one mght magne that there s lttle chance of makng vald nferences about the parameters of the process 66,

2 DSG POLLOCK : LINEAR STOCHASTIC MODELS However, provded that the process x(t) s statonary and provded that the statstcal dependences between wdely separated elements of the sequence are weak, t s possble to estmate consstently those parameters of the process whch express the dependence of proxmate elements of the sequence If one s prepared to make suffcently strong assumptons about the nature of the process, then a knowledge of such parameters may be all that s needed for a complete charactersaton of the process Movng-Average Processes The qth-order movng average process, or MA(q) process, s defned by the equaton (53) y(t) =µ ε(t)+µ 1 ε(t 1) + +µ q ε(t q), where ε(t), whch has E{ε(t)} =, s a whte-nose process consstng of a sequence of ndependently and dentcally dstrbuted random varables wth zero expectatons The equaton s normalsed ether by settng µ =1orby settng V {ε(t)} = σ ε = 1 The equaton can be wrtten n summary notaton as y(t) =µ(l)ε(t), where µ(l) =µ +µ 1 L+ +µ q L q s a polynomal n the lag operator A movng-average process s clearly statonary snce any two elements y t and y s represent the same functon of the vectors [ε t,ε t 1,,ε t q ] and [ε s,ε s 1,,ε s q ] whch are dentcally dstrbuted In addton to the condton of statonarty, t s usually requred that a movng-average process should be nvertble such that t can be expressed n the form of µ 1 (L)y(t) =ε(t) where the LHS embodes a convergent sum of past values of y(t) Ths s an nfnte-order autoregressve representaton of the process The representaton s avalable only f all the roots of the equaton µ(z) =µ +µ 1 z+ +µ q z q = le outsde the unt crcle Ths concluson follows from our dscusson of partal fractons As an example, let us consder the frst-order movng-average process whch s defned by (54) y(t) =ε(t) θε(t 1)=(1 θl)ε(t) Provded that θ < 1, ths can be wrtten n autoregressve form as (55) ε(t) =(1 θl) 1 y(t) = { y(t)+θy(t 1) + θ y(t ) + } Imagne that θ > 1 nstead Then, to obtan a convergent seres, we have to wrte (56) y(t +1)=ε(t+1) θε(t) = θ(1 L 1 /θ)ε(t), 67

3 DSG POLLOCK : TIME SERIES AND FORECASTING where L 1 ε(t) =ε(t+ 1) Ths gves (57) ε(t) = θ 1 (1 L 1 /θ) 1 y(t +1) = θ 1{ y(t+1)/θ + y(t +)/θ + y(t 3)/θ 3 + } Normally, an expresson such as ths, whch embodes future values of y(t), would have no reasonable meanng It s straghtforward to generate the sequence of autocovarances from a knowledge of the parameters of the movng-average process and of the varance of the whte-nose process Consder (58) γ τ = E(y t y t τ ) { } = E µ ε t µ ε t τ = µ µ E(ε t ε t τ ) Snce ε(t) s a sequence of ndependently and dentcally dstrbuted random varables wth zero expectatons, t follows that {, f τ + ; (59) E(ε t ε t τ )= σε, f = τ + Therefore (51) γ τ = σε µ µ +τ Now let τ =,1,,q Ths gves (511) γ = σε(µ + µ 1 + +µ q), γ 1 =σε(µ µ 1 +µ 1 µ + +µ q 1 µ q ), γ q =σ εµ µ q Also, γ τ = for all τ>q The frst-order movng-average process y(t) = ε(t) θε(t 1) has the followng autocovarances: (51) γ = σε(1 + θ ), γ 1 = σεθ, γ τ = f τ>1 68

4 DSG POLLOCK : LINEAR STOCHASTIC MODELS Thus, for a vector y =[y 1,y,,y T ] of T consecutve elements from a frstorder movng-average process, the dsperson matrx s 1+θ θ θ 1+θ θ (513) D(y) =σε θ 1+θ 1+θ In general, the dsperson matrx of a qth-order movng-average process has q subdagonal and q supradagonal bands of nonzero elements and zero elements elsewhere It s also helpful to defne an autocovarance generatng functon whch s a power seres whose coeffcents are the autocovarances γ τ for successve values of τ Ths s denoted by (514) γ(z) = τ γ τ z τ ; wth τ = {, ±1, ±,} and γ τ = γ τ The generatng functon s also called the z-transform of the autocovarance functon The autocovarance generatng functon of the qth-order movng-average process can be found qute readly Consder the convoluton µ(z)µ(z 1 )= µ z µ z (515) = = τ µ µ z ( µ µ +τ )z τ, τ = By referrng to the expresson for the autocovarance of lag τ of a movngaverage process gven under (1), t can be seen that the autocovarance generatng functon s ust (516) γ(z) =σ εµ(z)µ(z 1 ) Autoregressve Processes The pth-order autoregressve process, or AR(p) process, s defned by the equaton (517) α y(t)+α 1 y(t 1) + +α p y(t p)=ε(t) 69

5 DSG POLLOCK : TIME SERIES AND FORECASTING Ths equaton s nvarably normalsed by settng α = 1, although t would be possble to set σ ε = 1 nstead The equaton can be wrtten n summary notaton as α(l)y(t) =ε(t), where α(l) =α +α 1 L+ +α p L p For the process to be statonary, the roots of the equaton α(z) =α +α 1 z+ + α p z p = must le outsde the unt crcle Ths condton enables us to wrte the autoregressve process as an nfnte-order movng-average process n the form of y(t) =α 1 (L)ε(t) As an example, let us consder the frst-order autoregressve process whch s defned by (518) ε(t) =y(t) φy(t 1) =(1 φl)y(t) Provded that the process s statonary wth φ < 1, t can be represented n movng-average form as (519) y(t) =(1 φl) 1 ε(t) = { ε(t)+φε(t 1) + φ ε(t ) + } The autocovarances of the process can be found by usng the formula of (1) whch s applcable to movng-average process of fnte or nfnte order Thus (5) γ τ = E(y t y t τ ) { } = E φ ε t φ ε t τ = φ φ E(ε t ε t τ ); and the result under (9) ndcates that γ τ = σε φ φ +τ (51) = σ εφ τ 1 φ For a vector y =[y 1,y,,y T ] of T consecutve elements from a frst-order autoregressve process, the dsperson matrx has the form (5) D(y) = σ ε 1 φ 1 φ φ φ T 1 φ 1 φ φ T φ φ 1 φ T 3 φ T 1 φ T φ T 3 1 7

6 DSG POLLOCK : LINEAR STOCHASTIC MODELS To fnd the autocovarance generatng functon for the general pth-order autoregressve process, we may consder agan the functon α(z) = α z Snce an autoregressve process may be treated as an nfnte-order movngaverage process, t follows that σε (53) γ(z) = α(z)α(z 1 ) For an alternatve way of fndng the autocovarances of the pth-order process, consder multplyng α y t = ε t by y t τ and takng expectatons to gve (54) α E(y t y t τ )=E(ε t y t τ ) Takng account of the normalsaton α = 1, we fnd that { σ ε, f τ =; (55) E(ε t y t τ )=, f τ> Therefore, on settng E(y t y t τ )=γ τ, equaton (4) gves { σ ε, f τ =; (56) α γ τ =, f τ> The second of these s a homogeneous dfference equaton whch enables us to generate the sequence {γ p,γ p+1,} once p startng values γ,γ 1,,γ p 1 are known By lettng τ =,1,,p n (6), we generate a set of p + 1 equatons whch can be arrayed n matrx form as follows: γ γ 1 γ γ p 1 σ ε γ 1 γ γ 1 γ p 1 α 1 (57) γ γ 1 γ γ p α = γ p γ p 1 γ p γ α p These are called the Yule Walker equatons, and they can be used ether for generatng the values γ,γ 1,,γ p from the values α 1,,α p,σε or vce versa For an example of the two uses of the Yule Walker equatons, let us consder the second-order autoregressve process In that case, we have (58) γ γ 1 γ γ 1 γ γ 1 γ γ 1 γ = α α 1 α = α α 1 α α 1 α +α α α 1 α α α 1 α α α 1 α α α 1 α γ γ 1 γ 71 = σ ε γ γ 1 γ γ 1 γ

7 DSG POLLOCK : TIME SERIES AND FORECASTING Gven α = 1 and the values for γ,γ 1,γ, we can fnd σ ε and α 1,α Conversely, gven α,α 1,α and σ ε, we can fnd γ,γ 1,γ It s worth recallng at ths uncture that the normalsaton σ ε = 1 mght have been chosen nstead of α = 1 Ths would have rendered the equatons more easly ntellgble Notce also how the matrx followng the frst equalty s folded across the axs whch dvdes t vertcally to gve the matrx whch follows the second equalty Pleasng effects of ths sort often arse n tme-seres analyss The Partal Autocorrelaton Functon Let α r(r) be the coeffcent assocated wth y(t r) n an autoregressve process of order r whose parameters correspond to the autocovarances γ,γ 1,,γ r Then the sequence {α r(r) ; r =1,,}of such coeffcents, whose ndex corresponds to models of ncreasng orders, consttutes the partal autocorrelaton functon In effect, α r(r) ndcates the role n explanng the varance of y(t) whch s due to y(t r) when y(t 1),,y(t r+ 1) are also taken nto account Much of the theoretcal mportance of the partal autocorrelaton functon s due to the fact that, when γ s added, t represents an alternatve way of conveyng the nformaton whch s present n the sequence of autocorrelatons Its role n dentfyng the order of an autoregressve process s evdent; for, f α r(r) and f α p(p) = for all p>r, then t s clearly mpled that the process has an order of r The sequence of partal autocorrelatons may be computed effcently va the recursve Durbn Levnson Algorthm whch uses the coeffcents of the AR model of order r as the bass for calculatng the coeffcents of the model of order r +1 To derve the algorthm, let us magne that we already have the values α (r) =1,α 1(r),,α r(r) Then, by extendng the set of rth-order Yule Walker equatons to whch these values correspond, we can derve the system (59) wheren (53) g = γ γ 1 γ r γ r+1 γ 1 γ γ r 1 γ r γ r γ r 1 γ γ 1 γ r+1 γ r γ 1 γ 1 α 1(r) α r(r) = r α (r) γ r+1 wth α (r) =1 = 7 σ (r) g,

8 DSG POLLOCK : LINEAR STOCHASTIC MODELS The system can also be wrtten as (531) (53) γ γ 1 γ r γ r+1 γ 1 γ γ r 1 γ r γ r γ r 1 γ γ 1 γ r+1 γ r γ 1 γ α r(r) α 1(r) 1 = g σ (r) The two systems of equatons (9) and (31) can be combned to gve γ γ 1 γ r γ r+1 1 σ(r) γ 1 γ γ r 1 γ r α 1(r) + cα r(r) + cg = γ r γ r 1 γ γ 1 α r(r) + cα 1(r) γ r+1 γ r γ 1 γ c g + cσ(r) If we take the coeffcent of the combnaton to be (533) c = g σ(r), then the fnal element n the vector on the RHS becomes zero and the system becomes the set of Yule Walker equatons of order r + 1 The soluton of the equatons, from the last element α r+1(r+1) = c through to the varance term σ(r+1) s gven by (534) α r+1(r+1) = 1 { r } σ(r) α (r) γ r+1 = α 1(r+1) α r(r+1) = α 1(r) α r(r) + α r+1(r+1) α r(r) α 1(r) σ (r+1) = σ (r){ 1 (αr+1(r+1) ) } Thus the soluton of the Yule Walker system of order r + 1 s easly derved from the soluton of the system of order r, and there s scope for devsng a recursve procedure The startng values for the recurson are (535) α 1(1) = γ 1 /γ and σ (1) = γ { 1 (α1(1) ) } 73

9 DSG POLLOCK : TIME SERIES AND FORECASTING Autoregressve Movng Average Processes The autoregressve movng-average process of orders p and q, whch s referred to as the ARMA(p, q) process, s defned by the equaton (536) α y(t)+α 1 y(t 1) + +α p y(t p) =µ ε(t)+µ 1 ε(t 1) + +µ q ε(t q) The equaton s normalsed by settng α = 1 and by settng ether µ =1 or σ ε = 1 A more summary expresson for the equaton s α(l)y(t) =µ(l)ε(t) Provded that the roots of the equaton α(z) = le outsde the unt crcle, the process can be represented by the equaton y(t) =α 1 (L)µ(L)ε(t) whch corresponds to an nfnte-order movng-average process Conversely, provded the roots of the equaton µ(z) = le outsde the unt crcle, the process can be represented by the equaton µ 1 (L)α(L)y(t) =ε(t) whch corresponds to an nfnte-order autoregressve process By consderng the movng-average form of the process, and by notng the form of the autocovarance generatng functon for such a process whch s gven by equaton (16), t can be seen that the autocovarance generatng functon for the autoregressve movng-average process s (537) γ(z) =σ ε µ(z)µ(z 1 ) α(z)α(z 1 ) Ths generatng functon, whch s of some theoretcal nterest, does not provde a practcal means of fndng the autocovarances To fnd these, let us consder multplyng the equaton α y t = µ ε t by y t τ and takng expectatons Ths gves (538) α γ τ = µ δ τ, where γ τ = E(y t τ y t ) and δ τ = E(y t τ ε t ) Snce ε t s uncorrelated wth y t τ whenever t s subsequent to the latter, t follows that δ τ =f τ> Snce the ndex n the RHS of the equaton (38) runs from to q, t follows that (539) α γ τ = f τ>q Gven the q +1 nonzero values δ,δ 1,,δ q, and p ntal values γ,γ 1,,γ p 1 for the autocovarances, the equatons can be solved recursvely to obtan the subsequent values {γ p,γ p+1,} 74

10 DSG POLLOCK : LINEAR STOCHASTIC MODELS To fnd the requste values δ,δ 1,,δ q, consder multplyng the equaton α y t = µ ε t by ε t τ and takng expectatons Ths gves (54) α δ τ = µ τ σε, where δ τ = E(y t ε t τ ) The equaton may be rewrtten as (541) δ τ = 1 ( µ τ σε δ τ ), α =1 and, by settng τ =,1,,q, we can generate recursvely the requred values δ,δ 1,,δ q Example Consder the ARMA(, ) model whch gves the equaton (54) α y t + α 1 y t 1 + α y t = µ ε t + µ 1 ε t 1 + µ ε t Multplyng by y t, y t 1 and y t and takng expectatons gves (543) γ γ 1 γ γ 1 γ γ 1 α α 1 = δ δ 1 δ δ δ 1 µ µ 1 γ γ 1 γ α δ µ Multplyng by ε t, ε t 1 and ε t and takng expectatons gves (544) δ δ 1 δ δ δ 1 δ α α 1 α = σ ε σε σε µ µ 1 µ When the latter equatons are wrtten as (545) α α 1 α δ δ 1 = σ µ ε µ 1, α α 1 α δ µ they can be solved recursvely for δ, δ 1 and δ on the assumpton that that the values of α, α 1, α and σε are known Notce that, when we adopt the normalsaton α = µ = 1, we get δ = σε When the equatons (43) are rewrtten as (546) α α 1 α α 1 α + α γ γ 1 = µ µ 1 µ µ 1 µ δ δ 1, α α 1 α γ µ δ they can be solved for γ, γ 1 and γ Thus the startng values are obtaned whch enable the equaton (547) α γ τ + α 1 γ τ 1 + α γ τ =; τ> to be solved recursvely to generate the succeedng values {γ 3, γ 4,} of the autocovarances 75