Xidian University Liu Congfeng Page 1 of 49

dom Sgl Processg Cher4 dom Processes Cher 4 dom Processes Coes 4 dom Processes... 4. Deo o dom Process... 4. Chrcerzo o dom Process...4 4.. ol Chrcerzo o dom Process...4 4.. Frs-Order Deses o dom Process...5 4..3 Me o dom Process...5 4..4 Vrce o dom Process...5 4..5 Secod-Order Deses o dom Process...6 4..6 Auocorrelo d Auocovrce Fucos o dom Process...6 4..7 Power Secrl Desy o dom Process...7 4..8 Hgher-order Momes...7 4..9 Hgher-order Secr...8 4.. Nh-Order Deses...8 4.3 Sory o dom Process...8 4.3. Wde sese Sory dom Process...8 4.3. Proeres or Wde Sese Sory dom Processes... 4.4 mles o dom Processes... 4.4. Srgh Le Process... 4.4. Semrdom Bry rsmsso rocess...3 4.4.3 dom Bry rsmsso Process...6 4.4.4 Semrdom elegrgh Wves Process...8 4.4.5 dom elegrh Process... 4.4.6 dom Susodl Sgls...3 4.4.7 dom Wlk Process...5 4.5 Dee I egre o dom Processes...9 4.6 Jo Chrcerzos o dom Process...3 4.6. Frs-Order Jo Deses...3 4.6. Cross-correlo Fuco...3 4.6.3 Cross-covrce Fuco...3 4.6.4 Jo Sory...3 4.6.5 Cross-secrl Desy...33 4.7 Guss dom Processes...33 4.7. Frs-Order Deses or Guss dom Processes...33 4.7. Secod-Order Deses or Guss dom Processes...34 4.8 Whe dom Processes...34 4.9 AMA dom Processes...35 4.9. Movg Averge Process, MA(q)...36 4.9. Auoregressve Processes, A()...39 4.9.3 Auoregressve Movg Averge Process, AMA (, q)...43 4. Perodc dom Processes...47 4. Smlg o Couous dom Processes...47 4. rgodc dom Processes...48 4.3 Summry...49 d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge o 49

dom Sgl Processg Cher4 dom Processes 4 dom Processes 4. Deo o dom Process From cher d 3 we leed h eerme s seced by he hree ule S, BF, P,where S s couble or ocouble se rereseg he oucomes o he eerme, BF s Borel eld secyg he se o eves or whch cosse robbles es, d P s robbly mesure h llows he clculo o he robbly o ll he eves he Borel eld. A rel rdom vrble ws deed s rel-vlued uco o S subjec o () he eve seced by e ( e) e : s member o he BF or ll e, hus gureeg he esece o he cumulve dsrbuo, d (b) P e : ( e) or P e : ( e), or boh. I, sed o ssgg rel vlue o ech e, me uco e, s deed or ech e, we sy rdom rocess s seced. oughly sekg, rdom rocess s mly o me ucos ogeher wh robbly mesure. For e smle sce S we c vsulze he rdom rocess s Fgure 4., h o mg rom he smle sce o sce o me wveorms. e (,e ) S Wveorm Sce Fgure 4. A dom rocess vewed s ucol mg A rdom rocess c lso be vewed s show Fgure 4.. For rculr oucome e, wh robbly me wveorms. P, he me wveorm show, occurs. he me sgls rerese esemble o e I we evlue hese me wveorms, he vlues o he secod colum rom he rgh re obed. Comg dow h colum o vlues, we hve mg rom he oucomes o he r le, d s hs mg descrbes rculr rdom vrble. Smlrly rdom vrble deed or he mg show me s he ls colum. A smle emle o rdom rocess wll ow be gve. d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge o 49

dom Sgl Processg Cher4 dom Processes Probbly Oucome P(e) e S dom Process (,e) dom Vrble (,e) dom Vrble (,e) P e -.5 -.6 P e - -.5 -.8. P 3 e 3 - -.4. Fgure4. Illusro o rdom rocess s esemble o me wveorms d reled smle sce d robbly mesure. I should be oed h rdom rocess e, s uco o wo vrbles, d e, where s he me vrble d e s he oucome vrble. For rculr me rdom vrble. I e, or he oucome s ed, he e e,, e seces, s rculr me uco clled smle uco or relzo o he rdom rocess. Wh boh d e ed, sy d, e becomes rel umber or rel rdom rocess or comle vlue or comle rdom rocess. orml As he oo or rdom vrble s usully cl leer,y or Z rher h he more e, Y e, Z e, rdom rocess s usully deoed s rher h e commo o use smll or me uco o rdom rocess uco or relzo o he rocess. Also s e e,,. Also, s, o dce rculr vlue o he rdom vrble or rculr, resecvely. Such, or,, s clled smle, s rculr rdom vrble or ech, commo lerve deo o rdom rocess s deed se o rdom vrbles where e he deg se. I he de se γ c be u o oe o oe corresodece wh he se o egers, he rdom rocess s clled rdom sequece or dscree me rdom rocess s me. Whle, he deg se kes o couum o vlues, we wll cll he resulg rocess couous rdom rocess or couous me rdom rocess. d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 3 o 49

dom Sgl Processg Cher4 dom Processes 4. Chrcerzo o dom Process We sw Cher h rl chrcerzos or rdom vrble clude s me, vrce, momes, d he lke, d smlrly, useul rl chrcerzos or rdom rocess clude he me, vrce, d momes. However, ll wll, geerl, be ucos o me. For rdom rocesses ew yes o chrcerzos lke he h-order deses, uocorrelo uco, d secrl desy wll be deed h wll be useul lyzg rdom rocesses. hese chrcerzos gve dere sscl roeres o he rocess d dere mous o ormo cocerg he rocess. A herrchcl relosh bewee he vrous yes o chrcerzos ess d s show Fgure 4.3. ch o he chrcerzos wll ow be deed d her reloshs eled. ol Chrcerzo Prl Chrcerzos N h -Order Deses Auocorrelo Fuco d -Order Deses Hgher-Order Momes Power Secrl Desy s -Order Deses Hgher-Order Secrum Me Vrce Hgher-Order Mome Fgure 4.3 eloshs bewee vrous chrcerzos or rdom rocess 4.. ol Chrcerzo o dom Process emember h or every, s rdom vrble. hs gves us coubly e or e umber o rdom vrbles descrbed or he rdom rocess. A rdom rocess s deed o be comleely or olly chrcerzed he jo deses or he rdom vrbles,...,, re kow or ll mes,,, d ll., d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 4 o 49

dom Sgl Processg Cher4 dom Processes 4.. Frs-Order Deses o dom Process he rs-order robbly desy ucos o he rdom vrbles wll be deoed by or ;. I hey re ll he sme, he ( ) deed or ll me ; does o deed o d we cll he resulg desy he rs-order desy o he rocess; oherwse, we hve mly o rs-order deses. A emle o such mly o rs-order deses s show Fg. 4.4. he rs-order deses re oly rl chrcerzo s hey do o co ormo h seces he jo deses o he rdom vrbles deed wo or more dere mes. ( )= (; )=( ) F(;) ( ) ( ) ( ) ( - ) ( 3 ) ( 4 ) - 3 4-3 4 Fgure 4.4 Fmly o rs order deses or dere mes. 4..3 Me o dom Process he me o rdom rocess, For he cse where he me o s cos., s hus uco o me seced by ( ) d (4.) does o deed o, we use he oo 4..4 Vrce o dom Process he Vrce o dom Process () s smlrly deed by, where (4.) d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 5 o 49

dom Sgl Processg Cher4 dom Processes 4..5 Secod-Order Deses o dom Process For y wo mes d, he rdom vrbles d re deed. he secod-order deses o he rdom rocess () re he mly o deses seced or ll d where d re o equl. hese deses c be deoed s or, ;,., 4..6 Auocorrelo d Auocovrce Fucos o dom Process Gve he wo rdom vrbles ) d ) we kow h mesure o ler reloshs bewee hem s seced by he correlo ( (.As d go hrough ll ossble vlues,hs correlo c chge d s hus uco o d. he uocorrelo uco o rel rocess s, geerl, wo-dmesol o he vrbles d deed by Sce, (4.3),, s see o be symmercl d s,, (4.4) I cer cses, wh we wll dee ler s wde sese sory, he uocorrelo uco s see o deed oly o he me derece. For hs cse, we dee oe-dmesol uocorrelo uco by Sce he,whch s wre s c be erchged wh () (4.5), we see h s eve uco o (4.6) A uocovrce uco, C,, s deed by C, (4.7), I C, s ormlzed, we ob he ormlzed uocovrce uco deed by C,, C, C, (4.8) I geerl, he uocorrelo uco or comle rdom rocess, * Where * sds or comle cojuge. By wrg uocorrelo uco o hve he roeres s deed by (4.9) s j I, we c esly show he d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 6 o 49

dom Sgl Processg Cher4 dom Processes *,, * A uocovrce uco C, or comle rocess C s deed by * *, (4.), (4.) I C, or comle rocess s ormlzed, he ormlzed uocovrce uco s deed by C, *, C,, (4.) C 4..7 Power Secrl Desy o dom Process A very mor clss o rdom rocesses re hose or whch he uocovrce uco, c be wre s uco o he me derece s.hs ye o rocess wll be descrbed ler s beg sory uocorrelo. hs chrcerzo s clled ower secrl desy S d deed s he Fourer rsorm o he uocorrelo uco s ollows: S j ( ) e ( ) d (4.3) hs ormulo roughly descrbes he rego he requecy dom where ower o he rocess ess d he relve rooro o ower ech requecy. he ower secrl desy o gve rel rsom rocess hs he ollowg roeres: S w S w ve uco o w S w w F F S w S el d oegve Fourer rsorm Prs wdw S ol verge ower he rocess (4.4) w w w S wdw S wdw S wdw w w w Averge ower requecy bd w,w 4..8 Hgher-order Momes For he rdom vrbles,,, he hgher-order momes clude 3 d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 7 o 49

dom Sgl Processg Cher4 dom Processes,, 3 4, d so o. I ler sysems we wll d h hese hgher-order chrcerzo re o ecessry d h he me d uocorrelo re suce o dscuss u-ouu chrcerscs. O he oher hd, or oler sysems he order o deses d momes requred deeds erely o he ye o sysem cosdered. 4..9 Hgher-order Secr For cer rdom rocess, eseclly hose geered s he ouu o oler sysem, become coveos o dee hgher-order secr. Precse deo d roeres o hgher-order secr re reseed Cher 6. 4.. Nh-Order Deses he h order desy uco or he rdom rocess mes,,,, re gve by,,, or,,, ;,,, o emhsze he mes. ch o he chrcerzos o rdom rocess cos sscl ormo cocerg he rocess, d he vrous reloshs show Fgure 4.3 rovdes wy o vew he my dere sscl chrcerzos or rdom rocesses. eloshs bewee orms o sscl sory es d wll be oud useul or descrbg urher sscl roeres o rdom vrbles. 4.3 Sory o dom Process s clled srog sese or src sese sory he ses o rdom vrbles, d,,..., hve he sme robbly desy A rdom rocess,..., uco or ll, ll d ll. h s or ll,,...,, d, we hve,..., ;,,...,,,..., ;,,..., (4.5), A slghly weker orm o sory s sory o order N where he codos re rue or ll N or N ed eger. I he codos re rue or he N, he jo desy my be egred o gve oe less vrble, mkg he codos rue or ll less h N. Sory o order N does o reclude h he rocess could be sory o order greer h N. However, s o sory o order N, wll be o sory o y order greer h N. 4.3. Wde sese Sory dom Process A rocess s wek sese or wde sese sory d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 8 o 49

dom Sgl Processg Cher4 dom Processes () he eeced vlue o he rocess () he uocorrelo uco, Whe, s cos, hus deede o he me vrble, d s uco o me derece oly. c be wre s uco o me derece, oly, we wll deoe he uocorrelo o he rocess s or somemes s jus. I codo () bove s ssed, he rocess hs he lowes orm o sory uocorrelo.i codo () s ssed, he rocess hs he lowes orm o sory clled sory me. A rdom rocess s sory me, cos or ll me. I rocess s sory o order oe, he mus be sory me. However, rdom rocess c be sory me whou beg sory o order oe or sory uocorrelo. I Fgure 4.6 he reloshs bewee he vrous orms o sory re show. A sold rrow mes mlco, o rrow mes o mlco, whle doed rrow wh he urher ormo gve by he rrow mes mlco wh h ormo. Src Sese Sory Sory o Order(>) I Guss Sory o Order Wde Sese Sory Sory o Order Sory Auocorrelo Sory he Me Fgure 4.6 Coceul relosh bewee some commo orms o sory. A rdom rocess s clled symoclly sory o rculr sese codos or h sese re ssed he lm s ll mes roch e. We sy h rocess s cyclo sory o rculr sese he sese s ssed or mes h re erodclly dslced by mou. I hs bee show h rdom rocess s Guss d wek sese sory, s src sese sory. hs s ruly remrkble resul s mes h he uocorrelo uco d he me o wek sese sory rocess mus co ll he ormo ecessry o deerme he h-order desy uco. hs roery wll be emed urher Seco 4.7 o Guss rdom rocess. d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 9 o 49

dom Sgl Processg Cher4 dom Processes 4.3. Proeres or Wde Sese Sory dom Processes he uocorrelo uco or wde sese sory rocess s uco o he me derece d c hus be wre erms o h vrble s show o be rue:. For rel rocess he ollowg roeres c be eve uco o τ bouded uco o τ F S w, w F or ll S Fourer rsorm rs F rel d osve (4.6) S wdw verge ower he rocess C relosh o uocovrce uco o rocess 4.4 mles o dom Processes 4.4. Srgh Le Process Gve rdom rocess deed s hs eresso s bbrevo or For every P Q (4.7) e Pe Qe, (4.8) e,, s srgh le wh erce P d sloe Q o he rocess re reseed Fgure 4.7. e e e. Some relzos mle 4. We re sked o d he ollowg or he srgh le rocess: () Frs-order deses o he rocess., ; o he rocess. (b) Secod-order deses, (c) Me o he rocess. (d) Auocorrelo uco, o he rocess. (e) he dere ye o sory or hs rocess. d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge o 49

dom Sgl Processg Cher4 dom Processes (,e) (,e ) P(e 3 ) P(e ) P(e ) P(e ) (,e ) (,e ) (,e 3 ) Fgure 4.7 Four elzos o he srgh le rocess Soluo () he rs-order desy o he rocess c be obed rom he rsormol heorem s c be wre s ulry rdom vrble Where d q re he roos o he Jcob q P Q, he sum o he wo rdom vrbles P d Q. By deg Y P, he jo desy or d Y becomes Y J, s esly clculed s, q, q, q PQ, y (4.9), q J q d y. he oly soluo o hese equos s y y, q (4.) J, (4.) q Sce he desred mrgl desy s obed by egrg ou he ulry vrble, we hve PQ y, y / Noced h P d Q re ssclly deede, q dy (4.), c be wre s roduc d c be deermed by he covoluo o he wo deses s deede rdom vrbles. s sum o wo (b) he secod-order deses o he rocess c be obed by usg he rsormol heorem or wo ucos o wo rdom vrbles, sce d P Q, P Q re gve by (4.3) d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge o 49

dom Sgl Processg Cher4 dom Processes he se o equos hs oly oe soluo gve by q, q (4.4), (4.5) he Jocb s esly clculed s, so he jo robbly desy uco or d c be show o be, (c) he me o he rocess, use he rs-order desy o q /, / PQ (4.6), s comued by dg deermed r (), h s A lerve roch s o oce h he rdom vrble vrbles P d Q. I hs wy. Oe wy o ge he me s o d (4.7) c be wre s P Q P Q s relly uco o he wo rdom (4.8) hereore, he me o he rocess s srgh le h hs erce o o ecessry h he me be oe o he relzos o smle ucos o he rocess. P d sloe o Q (d) o ob he uocorrelo uco or he rdom rocess, s ecessry o deerme,. I s (4.9) Ag, hs uco c be oud severl dere wys. Frs, c be oud by usg he jo desy deermed r (b) s ollows: o he rdom vrbles d (4.3), dd I you do o hve he secod-order desy, you mus deerme beore he egrl bove c be s uco o he wo evlued. A lerve roch s o oce h he roduc rdom vrbles P d Q d mes d, rom whch he eceed vlue c be cculed s P Q hereore, oce q d, c be deermed. ( P Q P PQ Q, (4.3) ) PQ, s seced, he momes d jo momes gve bove c be clculed (e) he me o he rocess s gve by uco o me d he rdom rocess P Q. I he Q, he me s o s sory he me. Oherwse, wll o be sory he me. he rs-order desy gve s herely uco o me. hereore he rocess d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge o 49

dom Sgl Processg Cher4 dom Processes s o sory o order, d cosequely o sory o y hgher-order deses. For he rocess o be sory uocorrelo, mus be uco o me derece oly. ce or he degeerve cse o PQ d Q uocorrelo, d hus co be wde sese sory well., he rocess wll o be sory 4.4. Semrdom Bry rsmsso rocess he h d could rerese he oucome o lg hoes co. I wll be ssumed h he robbly o geg hed o sgle rl s / d h osses dere mes re deede. he smle sce cosss o ll ossble doubly e sequeces o heds d ls : S e : e (..., q, q, q, q,...) or q { h, } (4.3) { Wh he bove smle sce, robbly mesure, d he Borel eld equl o he ower se, he semrdom bry rsmsso rocess, o he me s s, usg d - s vlues, s descrbed o ech subervl q h (, e), or ( ) (4.33) q A relzo o he rocess, e s show Fgure 4.8 or he rculr oucome eerme descrbed by, h,,, h, h,, h,,, h, h, h,, h, h, e. (,e ) e o he 6-4 - 4 6 8 - e =(, h,,, h, h, h,, h,,, h, h, h,, h, h, ) Fgure 4.8 A elzo o he Semrdom Bry rsmsso Process. he deed rocess s colleco or esemble o me wveorms h chge rom o -, chge rom - o, or rem he sme ossble rso me os h re eqully sced dsce r log he me s. We re uble or he rocess. We c, however, chrcerze he rocess by secyg cer sscl roeres. I he ollowg rgrhs he rs-order desy, he me, he secod-order desy uco, d he uocorrelo uco o he rocess wll be oud. d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 3 o 49

dom Sgl Processg Cher4 dom Processes Frs-Order Desy. By cug cross he rocess rbrry me, rdom vrble s deed. Sce he vlued o or ll re eher or -, hs rdom vrbles re he sme. From he hoes co ssumo he robbly o geg eher hed or l ( or - or he rocess durg ech ervl) s /. hereore he rs-order desy s s ollows or ll : Me. he me, (4.34), o he rocess c be oud drecly rom he rs-order desy by Secod-order Desy. d rereseg d d d (4.35) or rbrry mes d re dscree rdom vrbles whch ke o vlues o + d -. her jo desy uco s gve by, P,, P,, P,, P,, (4.36) I d re dere ervls, d re deede rdom vrbles s he osses were deede. I d re he sme ervls, hey re he sme rdom vrble. he secod-order desy hus deeds o wheher d re he sme or dere ervls. For d dere ervls, P, P P (4.37) 4 Smlrly P,, P,, d, /4. hus he secod-order desy rom equo (4.36) becomes 4 P re see o be equl o,,,, (4.38), For d sme ervls, he robbles eeded c be obed by usg he codol robbles s ollows:, P P P (4.39) I he sme ervl P. hereore d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 4 o 49

dom Sgl Processg Cher4 dom Processes Smlrly, / P, P (3.4) P. Sce P, d, d he sme ervl co chgg sg, P re boh zero. hus, P P (4.4) Usg hese resuls q. (4.36) gves he secod-order desy o he rdom rocess or mes d he sme ervl s,, (4.4), Auocorrelo Fuco. he uocorrelo uco, s oud by comug he eeced vlue o he roduc o d s (4.43),, dd As he desy, s dere or he cses o d he sme d dere ervls, he uocorrelo uco s comued s ollows: For d dere ervls,,,,, 4 4, dd (4.44) For d he sme ervl,,,, dd (4.45) hereore, s he wo-dmesol ske o blocks uco log he le show Fgure 4.9. Sory. he semrdom bry rsmsso rocess s sory he me sce, whch s o uco o me, d sory o order sce s rs-order deses re o uco o me. However, he semrdom rocess s o sory o order, d hus ll hgher orders, becuse he secod-order deses re o he sme or shed versos o me derg by se mou. Smlrly he rocess s see o be o sory uocorrelo d hus o wde sese sory lso. d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 5 o 49

dom Sgl Processg Cher4 dom Processes (, ) 3 - - - 3 3 Fgure 4.9 he uocorrelo uco (, ) or he semdom bry rsmsso rocess. 4.4.3 dom Bry rsmsso Process he rdom bry rsmsso rocess, Y (), s deed by Y ( ) ( D) (4.46) Where () s he semrdom bry rsmsso rocess d D s uorm rdom vrble o, h s deede o (). Frs Order Deses o dom Bry rsmsso Proccess. he rs-order desy or hs rocess c be obed by rs dg he rs-order desy uco or ( d) codoed o rculr vlue o D d he egrg hs codol desy wh he robbly desy uco or D. he rs-order desy uco or ( d) s obed smlrly o (4.34), d sce d lys o r,he codol desy s he sme. Sce he resul s o uco o d, he egro gves he rs-order desy s he sme s h or he semrdom bry rocess: (4.47) Me o dom Bry rsmsso Process. Sce he rs-order desy s he sme or he rdom bry rsmsso rocess, he eeced vlue wll lso be he sme, h s, zero. Alervely, we c d he me by usg he ered eeced vlue ormul s ollows: Y Y d (4.48) Secod Order Deses o dom Bry rsmsso Process. o ob he secod-order deses, wll be covee o use he codol robbly desy uco or ( d), where d s y vlue bewee d. hus he secod-order desy codoed o d s s ollows: For vlues o d D d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 6 o 49

dom Sgl Processg Cher4 dom Processes d sde d ousde he doubly shded re show Fgure 4., he desy becomes: sde: ousde: 4 (4.49) D d,,, 4 D d,,,,, 4 4 (4.49b) d d Fgure 4. egos or he codol deses For mullyg (4.49) d (4.49b) by he rore robbles d ddg gves, 4 4,,,, (4.5) Auocorrelo Fuco o dom Bry rsmsso Process. he uocorrelo uco or he rdom bry rsmsso rocess c be oud by usg he ered eeced vlue roch s ollows: YY, Y Y D D d, d d, d D d dd where, Y, s kow o be or, h rego. So he d, d wll be oe rovded h he o d d he ervls. Assume h, h, d h wll er s Fgure 4.. I wll be roer ervl rovded h (4.5) he sme bo s show Fgure 4. d zero ousde, lls oe o. he he o d d, d d d. d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 7 o 49

dom Sgl Processg Cher4 dom Processes (, ) ( -d, -d) -=τ <τ< errgg hese codos o be u s rego or d gves roer ervl or he vlue o he d d, dd eeg he ses bove or YY d d d o be he, o be oe. hus rom (4.5) we hve (4.5), we d h YY /,, he o wll ever ll ervl becuse d s lwys. I where, so YY or hose, vlues o d. hus he uocorrelo uco or he rdom bry rsmsso rocress c be eressed s Fgure 4. egos or he rdom bry rsmsso rocess YY,, /, oherwse (4.53) Sory o he dom Bry rsmsso Process. he rdom bry rdom rsmsso wve s hus see o be sory o me, sory uocorrelo, sory o order oe, d wde sese sory. 4.4.4 Semrdom elegrgh Wves Process Cosder he eerme o rdom seleco o mes uorm re er u me such h he robbly o k os he ervl, s gve by P k k k e o s [, ] (4.54) k! he oucome o sgle rl o he eerme resuls sequece o mes d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 8 o 49

dom Sgl Processg Cher4 dom Processes Dee he semrdom elegrh rocess s ollows:, e...,,,,,,... (4.55) me e hs eve umber o o s [, ] e (4.56) me e hs odd umber o o s [, ] Frs-Order Desy. A y gve me, s dscree rdom vrble deoed by whch kes o oly he vlues or -. he robbly desy uco or Where P Sce he eves ecly c be wre s P P (4.57) Phe umber o o s[, ] s eve P{ ecly o s, or ecly o s, or ecly ko s, or} (4.58) k d j os re muully eclusve or k j, he robbly o he uo o ll he eves c be wre s he sum o robbles. Subsug he robbly o ecly k os rom q. (4.54) o (4.58) gves he ollowg or : Fcorg ou he P k k e k k k! P k o s [, ] e I smlr sho he P{ d recogzg he sum s cosh, P smles o (4.59) P{ } e cosh( ) (4.6) P c be wre s } P{k o s [, ]} sh( ) (k )! e (4.6) k k k From hese resuls he rs-order desy o he semrdom elegrh wve become For, he desy s obed by relcg by Me o he me e ( ) e cosh( ) ( ) e sh( ) ( ).. Usg he bove obed rs-order desy uco or he rocess s [ ( )] ( e ) d cosh( ) ( ) e he smlg roery o he del uco c be used o smly sh( ) ( ) d s ollows: (4.6), we clcule (4.63) d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 9 o 49

dom Sgl Processg Cher4 dom Processes, For A lo o he me goes o zero s [ ( )] becomes e cosh( ) e sh( ) e (4.64) e,d hus [ ( )] c lly be wre s e (4.65) s show Fgurg 4.3. I s oced h or osve me srs d. Close o he org s lmos.sce ech relzo o he rocess srs. [()]=e -λ Secod-order Desy. he rdom vrbles d deoed by d,resecvely, re dscree rdom vrbles kg o oly vlues d - wh jo robbly desy uco, s (, ) P{ P{ P{ P{,,,, } ( } ( } ( } (,,,, d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge o 49 ) ) ) ) (4.66) he weghs o he del ucos re oud by wre he rdom vrbles jo robbly erms o codol robbly d mrgl robbly s ollows: P {, } P{ } P{ } (4.67) he codol robbly h odd umber o os occur ervl, s or he ollowg he P ws oud erler s P e sh e (4.68) P sh (4.69) Usg he codol d mrgl robbles jus deermed gves he jo robbly s P, e cosh sh Fgure 4.3 Me o he semrdom elegrh wve rocess (4.7) I smlr wy he oher jo robbles c be deermed, resulg he jo robbly desy uco

dom Sgl Processg Cher4 dom Processes, e cosh cosh, e cosh cosh, e sh sh, e sh cosh, Auocorrelo Fuco. From he, bove he uocorrelo uco, dd (4.7) (4.7) Crryg ou he egro d usg he smlg roery o he wo-dmesol del uco,, c be smled o gve e By reversg he roles o d show o be, or d reeg he rocedure descrbed bove, (4.73), c be, e or ll d (4.74) A lo o, s uco o d s show Fgure 4.4 d s rough o bsolue eoels. I Fgure 4.4b,, s loed s uco o he me derece. Sce s uco o me derece oly, c be wre s. hus, where he semrdom elegrh wve rocess s sory uocorrelo. However, s o wde sese sory sce s uco o s show Fgure 4.3. (, )=e(-λ - ) (τ) e -λ τ - =τ τ= - τ () = Fgure 4.4 Auocorrelo uco or he semrdo elegrh wve rocess. () lo o (, ) s uco o d, (b) lo o (τ). (b) d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge o 49

dom Sgl Processg Cher4 dom Processes 4.4.5 dom elegrh Process Le be he semrdom elegrh rdom rocess. he rdom elegrh wve c be deed s Where A s dscree rdom vrble wh robbly desy s A s ssumed o be deede o he rdom rocess A Y( ) A ( ) (4.75) ( ) ( ) ( ) (4.76). A rdom vrble s deede o rdom rocess s grou deede o ll rdom vrbles deed cross he rocess. he eeced vlue or me o he rocess vlue o A s zero: he uocorrelo uco or YY Y A A Y c be oud s ollows:, A A A, e Y c be deermed s ollows sce he eeced (4.77) (4.78) he eeced vlue o he roduc bove c be wre s he roduc o he eeced vlues becuse s lso deede o he rocess (uco o deede rocesses re deede), d sce A, he uocorrelo uco c be wre erms o he me derece s e YY (4.79), Sce he me s e cos, (zero), d he uocorrelo uco s uco o me derece oly, we hve show h he rdom elegrh wve s wde sese sory rdom rocess. A logcl queso hs me s: Is he rdom elegrh wve sory o y oher vrey? For emle, s sory o order? I order o swer h queso he rs-order desy or he semrdom bry rsmsso mus be oud. Oe wy o obg he desy s o egre ou he A codol desy or Y : Where Y y y d A s gve by q.(4.76) d (4.8) Y y or he semrdom elegrh rs-order desy, mlude A c be deermed by modyg q.(4.6), whch A, s y e y e sh y Subsug he wo deses o he equo or Y y cosh (4.8) d egrg usg he smg roery o he del uco ves, er smlco, he desy s y.5 y.5 y (4.8) Y d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge o 49

dom Sgl Processg Cher4 dom Processes he rs-order desy s hus show o be deede o me, d hece he rdom elegrh wve s sory o order oe. he swer o he queso o sory o order wo s sved or he reder s homework. 4.4.6 dom Susodl Sgls I my suos collecos o hese sgls hve dere rd requeces, dere hses, d dere hses, d dere mludes. I hese rmeers c be modeled s rdom vrbles, he he rdom susodl sgl rdom rocess c be deed by Where A, d re rdom vrbles. A cos (4.83) mle 4.3 Dee rdom rocess s A A d vrbles wh kow robbly desy ucos cos, where A d re deede rdom d where s deermsc d kow. hs s he cse o kow susodl requecy sgl wh rdom mlude d hse. Furher s ssumed h A s ylegh rdom vrble d s uorm rdom vrble wh he ollowg robbly desy ucos: A, (4.84), elsewhere e For hs deed rdom rocess deerme () he me o he rocess, (b) he uocorrelo uco o he rocess, (c) he rs-order deses o he rocess, (d) he secod order desy ucos, d (e) he yes o sory o he rocess. Soluo (, s () he me o he rocess, Acos AA cos ( (4.85) he secod se c be mde sce he eeced vlue o roduc o deede rdom vrbles s he roduc o he eeced vlues. Sce s uorm o,, he secod eeced vlue bove s zero, hus gvg or ll, (b) he uocorrelo uco, s Acos Acos (4.86) A cos, cos (4.87) d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 3 o 49

dom Sgl Processg Cher4 dom Processes Ag he secod se c be doe sce ucos o deede rdom vrbles re deede. he rs erm s esly obed rom he desy Usg he roduc dey or coses he secod erm becomes A, wheres he secod erm kes slghly more work. cos cos cos ( ) cos( ( ) (4.88) he rs eeced vlue o he rgh sde o he equo bove s zero sce he egro o cose over wo erods s zero, whle he secod eeced vlue gves he brckeed erm sce s ordom. hereore he uo correlo uco becomes (, ) [ A ]cos( ( ) (4.89) (c) he rs-order deses ; c be obed severl dere wys or he rdom vrble deed by A cos (4.9) From he equo bove, rdom vrble B by roduc Where Ad s oe uco o he wo rdom vrbles A d. Deg B hs bee show o be B b s kow o be cos, s see s he roduc o A d B. he desy or he b b B db (4.9), b B b b (4.9), elsewhere A s gve he roblem seme. Srge s my seem, he resul o erormg he egro gves e (4.93) hereore he rs order desy or he rocess s Guss wh vrce gve by evlug q.(4.89) or s (d) he secod-order desy or A. 46 (4.94) d deed by d s ucos o he rdom vrbles A d,, resecvely, c be oud usg he wo ucos o wo rdom vrbles rsormo heorem o he ollowg rsormo: d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 4 o 49

dom Sgl Processg Cher4 dom Processes Acos Acos Aoher wy o ob he desy uses he resul obed Cher, h d (4.95) re Guss rdom vrbles sce s ylegh dsrbued rdom vrble deede o he rdom vrble A whch s uormly dsrbued. By hs resul, he roblem o chrcerzg he wo Guss rdom vrbles d s equvle o dg he mes, vrces, d correlo or hose vrbles. I becomes eede dg hose momes h we ed ou he cose ucos rom bove o gve Acoscos As s (4.96) Acoscos As s Sce A cos d A s re deede d Guss, he rdom vrbles d re joly Guss s hey re ler ucos o Guss rdom vrbles. By kg eeced vlues, we c show he momes o be A A, (4.97) A hese ve rmeers deerme he secod-order Guss desy uco or he rdom susodl descrbed hs emle. (e) Sce he me o s zero or ll me, he rocess s sory me. As he uocorrelo uco s uco o me derece oly, he rocess s sory o uo correlo, d sce s sory me, s lso wde sese sory. Furher, sce wde sese sory, s src sese sory. s Guss rdom rocess d 4.4.7 dom Wlk Process Cosder he eerme o ossg co coubly e umber o mes. he oucome o he eerme s cres roduc o sgle osses h re eher heds or ls. he smlr sce or he rdom rocess s he he se o ll coubly e sequeces o heds d ls. he Borel eld s deed s he ower se o he smle sce h s he se o ll ossble subses o hs smle sce. Assume h he robbly o geg hed ech oss s equl o d h ech oss s deede o he ohers. d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 5 o 49

dom Sgl Processg Cher4 dom Processes he rdom wlk rocess s deed log uorm segmes o he me s or ech oucome e..., h, h,,,... s ollows: e cs,,,,,... C = umber o heds - umber o ls rs elemes o e (4.98) he d S re me duro d se sze, resecvely, d ed. I here re k heds he rs elemes, he he umber o ls s k d he umber o heds mus he umber o ls s k k or k. hereore he rocess c be deed s ollows or ech oucome:, e K S,,,, (4.99) where K s rdom vrble rereseg he umber o heds. A ew relzos o he rocess re show Fgure 4.5. (,e ) e =(h, h,, h, h, ) 3S S S 3 4 5 e =(h,,,, h, ) S -S -S (,e ) 3 4 5 e 3 =(h,, h,, h, ) S (,e 3 ) 3 4 5 Fgure 4.5 elzo o he rdom wlk rocess. he rs-order desy, me, secod-order desy, d uocorrelo uco or he rdom wlk re ow reseed. Frs Order Desy. By he deede ssumo, K hs boml desy d ollows: j j K k k j (4.) j j Where s he robbly o hed ech oss d s he robbly o l. Sce K S or rsormo o K, he robbly desy o becomes k k k k k or he resul bove ws obed by usg he rocedure or rsormo o dscree rdom vrble. I d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 6 o 49 S (4.)

dom Sgl Processg Cher4 dom Processes we he rs ervl, he desy becomes d or y, he secod ervl, we hve S S (4.) S S (4.3) ereseve robbly desy uco or severl s re show Fgure 4.6. As becomes lrge, he eveloe o he desy ucos roches Guss uco ceered d wh vrce rmeer o. ( ) 3S S S 3S S -S / /8 /4 / 3/8 / 3/8 3 3 /4 /8 or =/ 3 Fgure 4.6 ereseve rs order robbly desy ucos or he rdom wlk rocess Me. Usg he desy uco or gve q. (4.), we c wre he me o k k k d k k S d s (4.4) Usg he smlg roery o he del uco, we c wre he eeced vlue o he rdom wlk rocess or, s k k S k k (4.5) k he rs r o he sum s jus he verge vlue he boml desy mes d he secod s jus mes, sce s sum over he boml desy. hereore S S S, or (4.6) A lo o he eeced vlue s show Fgure 4.7 or /, /, d /. For ll oher h / he le he me. s. hus, uless /, he rdom wlk rocess s o sory, s d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 7 o 49

dom Sgl Processg Cher4 dom Processes d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 8 o 49 Auocorrelo Fuco. Oe roch h c be used o d he uocorrelo s o rs d he secod-order desy d he comue he eeced vlue usg h desy. A lerve roch, whch or hs rculr roblem s eser, s o comue he eeced vlue drecly s uco o oher rdom vrbles. o do hs, we hve o vew he rdom rocess s he sum o deede declly dsrbued rdom vrbles:,,,, (4.7) where re dscree deede rdom vrbles wh decl robbly desy ucos S S (4.8) emsce o he rdom elegrh wve develome, he uocorrelo uco, wll ke o dere ervls log he s. Assumg, where s he h ervls,, he uocorrelo uco becomes or m, j j m j j m j j m j j m, (4.9) Becuse he re deede d declly dsrbued, he rs eeced vlue c be wre s 3 4 5-4(-)S -(-)S (-)S 4(-)S [()] P</ Fgure 4.7 he me o he rdom wlk rocess or vrous o, he Probbly o heds.

dom Sgl Processg Cher4 dom Processes d he secod becomes m dere rdom vrbles. hereore, becomes m,, sce roducs co ll (4.) Sce he re declly dsrbued wh deses, q. (4.8), her rs- d secod-order momes re S S S S S Subsug hese mome o q.(4.) d smlyg, gves, Ierchgg m d gves us he ollowg or I we se /, he, S m S S (4.), or m, s ollows: (4.) m :, ms m S (4.3), becomes s show Fgure 4.8. 5 4 3 S S S S S S S S S S 3S 3S 4S 4S 4S 3S 3S 3S S S S S S S 3 4 5 Fgure 4.8 Auocorrelo uco or he rdom wlk whe =5 4.5 Dee I egre o dom Processes For gve rdom rocess e rculr wveorm by As e goes over he smle sce, rdom vrble I e d deoe by I :, d gve e, sy e, we hve he dee egrl o h e e I b, d (4.4) I e seces rdom vrble. For smlcy, we wll wre h b I d (4.5) I s logcl hs o o sk wh s he me d he vrce o such rdom vrble d d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 9 o 49

dom Sgl Processg Cher4 dom Processes wh ormo bou he rdom rocess do we eed o d hem. he me o I s gve s I b [ ] d (4.6) I c be show mos cses h he eeced vlue d he egrl c be erchged o gve b b I ] d I smlr sho he secod mome o I c be oud s Gve I d I [ d (4.7) b b b b I d udu u b b, u ddu, he vrce o I s esly see o be b b I I C u ddu (4.8), ddu (4.9) I hereore he me d vrce o I re deermed by he secod-order chrcerzo o he rocess, mely s me d uocovrce uco u hgher-order chrcerzos o he rocess uco or I, ol chrcerzo o he rocess I he rocess C,. Hgher-order momes requre sel. I order o ob he robbly desy o he ervl o egro s requred. s wde sese sory rocess wh me d uocorrelo uco he ormuls or I he becomes d Sce he uocorrelo uco I c be smled. he eeced vlue o I rom equo b b I ] d d b [ (4.) s uco o he me derece u, he wo-dmesol egrl gve he equo c be chged o oe-dmesol egrl. he cremel re requred hs chge s show Fgure 4.9, d he resulg egrl or he vrce becomes I or I b b b b b C b d d (4.) A emle llusrg he rocedure or dg he me d vrce o dee egrl o wde sese sory rocess s ow reseed. d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 3 o 49

dom Sgl Processg Cher4 dom Processes =u+(b-) =u+τ+ τ =u+τ =u b (b-) -(b-) b (b-)-τ b-τ =u-(b-) Are=[(b-)-τ] τ u Fgure 4.9 Icremel re or he chge o vrbles -u=τ. mle 4.4 A wde sese sory rdom rocess ollows: Deerme Soluo s chrcerzed by s me d uocorrelo uco s d 4 e. Dee rdom vrble I d 3 I d I. he eeced vlue o he rdom vrble I s esly obed rom (4.) s I b 3 4 I d. From he gve me d uocorrelo uco, he vrce o I c be deermed usg (4.) s ollows: I b b 4 e 4 b e d 4e 4.6 Jo Chrcerzos o dom Process d wo rdom rocess deed cross he rdom rocess cross he oher rocess d Y re deed o be deede y se o rdom vrbles s grou deede o y se o rdom vrbles deed Y. wo rdom rocess re clled ucorreled her cross-correlo uco c be wre s he roduc Y Y Y or ll, (4.) d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 3 o 49

dom Sgl Processg Cher4 dom Processes whch mles h y rdom vrble deed cross he rocess rdom vrble cross he rocess Y me. me s ucorreled wh y 4.6. Frs-Order Jo Deses Dee he rdom vrbles d Y, oo d Y resecvely. he secod-order jo desy s gve s he jo robbly desy uco or hese wo rdom vrbles s, y ;. he oo, jo deses rom ll choces o mes d., y s commoly used. he rs-order jo deses coss o ll 4.6. Cross-correlo Fuco he cross-correlo uco, deoed s, Y, s deed s Y, ) [ ( ) Y ( )] (4.3) ( I he rdom rocesses re comle rocess, useul deo or he cross-correlo uco c be obed by kg he comle cojuge o he secod rocess s 4.6.3 Cross-covrce Fuco * Y, ) [ ( ) Y ( )] (4.4) ( Smlr o he uo correlo uco ece ow deed cross wo rocesses he cross-covrce uco C, Y s deed by CY, ( ) ( ) Y ( ) [ Y ( )] ( ) ( ) Y ( ) ( ) I he rocess re comle, he he ollowg deo s used or he cross-correlo uco: C Y, ( ) ( ) Y ( ) ( * Y ) Y (4.5) (4.6) he cross covrce uco d cross correlo uco re reled by ( where ) d ), resecvely. * CY, Y, ( ) Y ( ) rerese he mes o rocess d Y ( (4.7) Y s ucos o d 4.6.4 Jo Sory he rdom rocesses re joly wde sese sory ech rocess d Y re wde sese d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 3 o 49

dom Sgl Processg Cher4 dom Processes sory hemselves d he cross-covrce uco s oly uco o me derece d Y s cos d o uco o or. Whe hs s rue, he cross-correlo uco d cross-covrce uco re wre erms o he me derece oly: C Y Y Y (4.8) Jo src sese sory would mly h or every me rsle o mou or y, he jo robbly deses re he sme. hs c be eressed s,,,, y, y,, y ;,,,,,,,, m,, m ;,,,,,,, m,, m (4.9) 4.6.5 Cross-secrl Desy Whe he rdom rocess d Y re joly wde sese sory, s covee o dee cross-secrl desy h gves secrl coe ormo o he jo rocess. he cross secrl desy or rocesses uco Y s d Y s deed s he Fourer rsorm o he cross-correlo S S Y Y ( ) ( ) 4.7 Guss dom Processes Y Y e e j j d d (4.3) A rdom rocess re joly Guss or ll choces o s Guss rdom rocess he rdom vrbles,...,,,,,, d or ll., here re umber o roeres d heorems reled o Guss rdom rocess. hree o he mos useul ollow. () A rdom rocess h s wde sese sory d Guss s src sese sory. () Iegrls o Guss rdom rocess over gve me ervl re Guss rdom vrbles d rdom vrbles over my dere ervls re joly Guss rdom vrbles. (3) he rocess deed by he egrl o Guss rdom rocess over he ervl o, s vres rom o, s Guss rdom rocess, whch c be wre s Y d or (4.3) 4.7. Frs-Order Deses or Guss dom Processes I Guss rdom rocess s chrcerzed by s me d uocorrelo uco d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 33 o 49

dom Sgl Processg Cher4 dom Processes,, s rs- d secod-order desy ucos c be wre erms o he me d uocovrce uco C,.he rs-order desy s ; e C C,, 4.7. Secod-Order Deses or Guss dom Processes (4.3) he secod-order desy or he rdom vrbles d deed, resecvely, s d s, ;, e m K m K Where, m C K C, C,, C,, (4.33) Whe he Guss rdom rocess s wde sese sory, he rs- d secod- order deses c be wre erms o he me d uocovrce uco Ad ; C e C s C, ;, e m K m K (4.34) Where, m, K C C C C (4.35) 4.8 Whe dom Processes For cse lyss, s covee o dee he whle ose rocess. he me comes rom he c h he secrl desy s ssumed o be l d co ll requeces. A useul deo s h whe ose rdom rocess s chrcerzed by s ower secrl desy S N or d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 34 o 49 S s ollows: (4.36) he uocorrelo uco or he whe ose rocess s he verse Fourer rsorm o he ower secrl desy, whch becomes

dom Sgl Processg Cher4 dom Processes N (4.37) he uocorrelo uco d ower secrl desy or he whle ose rocess re gve Fgure 4.. hs rculr deo mles he ollowg: () () or o equl. hs mes h he rdom vrbles deed wo o equl mes re orhogol d becuse o he zero me, hey re lso ucorreled. Furhermore, he whe ose rocess s Guss, hey re deede rdom vrbles. S () (τ) /N /N τ () Whe Nose PSD (b) Whe Nose Auocorrelo Fgure 4. Power secrl desy d uocorrelo uco or he whe ose rdom rocess. (3) he verge ower s e. hus he whe ose rocess s o hysclly relzble, bu c be used s bss or geerg relzble rocesses by ssg whe ose hrough ler. Whe ose should o be hough o s observble rocess; oly s ouu er lerg s observble. Noeheless, s covee o mke whe ose ssumos o smly lyss. No oher chrcerzos re gve he deo, so s ossble o hve y rs-order d hgher-order robbly desy ucos. I my roblems s covee o use deses h re Guss, d hus he words Guss whe ose re commoly used o dey whe ose rdom rocess wh he sme uocorrelo s gve Ad he h order deses beg Guss. I some roblems he ower s o cos bu vres wh me. A whe ose rocess s deed s osory whe rdom rocess s uocorrelo uco s wre s 4.9 AMA dom Processes, (4.38) u N u I workg wh couous ler me-vryg d me-vr sysems, useul sysem model s derel equo erms o he u d ouu. For me-vr sysems he coeces o he derel equo re cos, d he Llce rsorm s covee d eede or lyss. For dscree me sh vr ler sysems, derece equo erms o u d ouu s useul d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 35 o 49

dom Sgl Processg Cher4 dom Processes model, d he Z-rsorm lys sgc role lyzg such sysems. Furher dscussos o hese reloshs wll be reseed Cher 5, whle hs seco we vesge secl clss o rdom rocess resulg rom drvg dscree me sysem wh whe ose rocess. I he ouu, he w rereses he u o ler me-vr cusl sysem d u-ouu relosh c be wre s I s usully ssumed h I k k bk w k k k (4.39) q d h he equo bove s deed or eher ll or or. w s whe sequece wh w d w jwk jk or ll j d k, he s clled uoregressve movg verge(ama) rdom rocess o order resulg rocess, q, whch s wre s AMA, q. I ll he or ll k,,..., k, he resulg rdom rocess verge (MA) rocess o order q bbreved s MA q d govered by I b or ll k,,..., k b w k k, he resulg rdom rocess rocess o order, wre s A P d govered by k k k bw he word uoregressve comes orm he sscl commuy d mles h he k s clled movg (4.4) s clled uo regressve(a) sel, whch mes h c be wre s ler uco o s ow revous vlues. Deedg o he sscl roeres ssumed or (4.4) regresses o w k, he AMA rocesses deed by (4.39) hve vryg roeres wh resec o chrcerzos lke me, vrce, uocorrelo, rs-order deses, d so o. Also dere resuls re obed he or dees greer h or equl zero. I w d w s deed or ll me dees or jus re deed or ll, he sysem wll hve reched sedy se ouu. However, hey re deed or oly me dees, he begg smles wll co rse oro rechg sedy se oly or lrge vlues o he me de. 4.9. Movg Averge Process, MA(q) For hs dscusso le he goverg equo or movg verge rocess o order q, MA q q. (4.4), be deed s d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 36 o 49

dom Sgl Processg Cher4 dom Processes b w k, k k (4.4) Where w s whe sory rocess, o ecessrly Guss, wh zero me, w,d vrce gve by w or ll. Me o MA(q) Process. he me o he movg verge rocess o order q c be oud by kg he eeced vlue o q. (4.4) s [ ] bk w k bk w k, (4.43) k k w k or k o q, he me o he MAq rocess s gve by Sce, (4.44) Vrce o he Movg Averge Process, MA(q). he vrce o he MA q rocess s gve by he vrce, (4.45) s zero by deo or ll, From(4.44) we kow h.for ll,so he vrce s jus he secod mome. Aer subsug he o q. (4.4) o q. (4.45), he vrce becomes q q q bk w[ k] bk w[ k] bjw[ j] (4.46) k k j Mullyg ou he erms he brckes d kg he eeced vlue hrough he summos gves or, q q b b w[ k] w[ j] b, q (4.47) k j he ls se s ossble becuse he oly erms where w[ k] w[ j] whch I j k. Noce h hs vrce s hus cos or. k j q k, he resul bove s moded o clude oly hose k rgumes, d s hus uco o : he vrce k re ozero re hose or w d j w h hve b, q (4.48) k k s hus see o be uco o d r o he l rse erod ul reches q yeldg sedy se er h. Auocorrelo Fuco or he Movg Averge Process. MA(q). For k d j less h zero he uocorrelo uco j k, or he MA(q) rocess s deed o be zero,whle geerl, wll d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 37 o 49

dom Sgl Processg Cher4 dom Processes deed o wo me dees Le k m d k,where k d m re osve egers d c be wre s k mk b w[ k m ] bjw[ k j], k, m q q k (4.49) j k q o beg wh d m q. For hs rge o m, q. (4.49) c be wre s k m, k b wk m bw k m bqwk m q b w k b w k b w k q q (4.5) Lookg ll cross roduc erms, we see h he oly oes h gve ozero vlue o re hose wh equl dees, so q.(4.5) reduces o For qm k m, k brbr m w k r r qm r r b b r m m q here re o overls he roducs, so we hve k m, k, m q (4.5) (4.5) Clerly, he uocorrelo uco or he MA q rocess s deede o he srg me k d uco o he me derece m, rovded h k q. hus er he me q, he rocess becomes wde sese sory sce he me ws revously show o be cos, mely, d he uocorrelo hs bee show o be uco o me derece oly. mle 4.5 Fd he me, uocorrelo uco, d vrce or he MA(), MA(), d MA(3),rdom rocesses ssumg h sedy se hs bee reched, q w s zero me whe rdom rocess. Soluo k, or he models gve d wh he For he MA q rdom rocess he mes re ll zero becuse o he ssumo h rdom rocess; or, d deed o be zero or. w s whe MA() For hs rocess we hve he deg equo b w b w or (4.53) he uocorrelo uco evlued lgs o,, d er sedy se re gve rom (4.5) s b b b b r brbr r k, k r r b b (4.54) d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 38 o 49

dom Sgl Processg Cher4 dom Processes he sedy se vrce k s gve by k, sce he me s zero. he uocorrelo uco or egve me derece s obed by symmery k k MA(). I smlr sho we ob he uocorrelo ucos or he MA() d MA(3) s ollows: b w b w b w or (4.55) he uocorrelo uco er sedy se s esly oud rom (4.5) o be he sedy se vrce MA(3) b b b bb bb bb k, k k k k s gve by k, sce he me s zero. b w b w b w b w 3 or 3 (4.56) (4.57) he uocorrelo uco er sedy se s esly deermed rom (4.5) s he sedy se vrce b b b b bb bb bb3 bb bb 3 3 bb3 k, k 3 k k k s gve by k 3, sce he me s zero. (4.58) 4.9. Auoregressve Processes, A() A uoregressve rocess, A ssumed h A he eco rocess w j w k jk A rdom rocess., s secl cse o he AMA, q rocess. For hs dscusso s s govered or by k bw, k w, or k (4.59), s whe sory rdom rocess wh. We would lke o deerme he me, vrce, d uocorrelo or hs Me o A() Process. he me o he A rocess s obed by kg he eeced vlue o he d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 39 o 49

dom Sgl Processg Cher4 dom Processes goverg q. (4.59) o yeld he ollowg derece equo: Sce [ k ] [ k] b w[ ], w, d, or, k (4.6), he soluo o he derece equo yelds (4.6) Vrce o A() Process. he vrce o he deed A rocess c be oud drecly rom he deo o he vrce d he rocess wre vrce s gve q. (4.59). Usg he zero me ssumo, we [ ] k b w j b w k j (4.6) k j he eeced vlue o ll cross roducs o w wh he summos o he k, gve by k k w (4.63) k re ll zero, sce k re o ucos o w s hey re rom me recedg w hus he vrce reduces o k kw j j b k j (4.64) Mullyg he roduc erms ou, kg he eeced vlue d rerrgg, we d he vrce rom q, (4.64) o be jk k j, b (4.65) k j he vrce me de s hus see o be uco o he vrces he revous mes, lus uco o vrous recedg uocorrelo ucos vrous me dees, lus weghed sum o he whe sequece vrce. A lerve orm or he vrce c be obed by mullyg he d kg he eeced vlue s o q. (4.59) by k k bw (4.66) k Mullyg ou d kg he eeced vlue o he erms resuls he ollowg equo or he vrce whch, becuse o he zero me, s he uocorrelo uco d : k, k, b (4.67) k I he orm he vrce s wre erms o he uocorrelo ucos or dees, k s k d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 4 o 49

dom Sgl Processg Cher4 dom Processes goes rom o. hs sme roch wll ow be used o deerme he uocorrelo uco or he uoregressve rocess oher lgs. Auocorrelo o A() Process. he uocorrelo uco, geerl, goes hrough rse erod sce he sgl srs zero wh zero l codos. o derve he uocorrelo uco, s ssumed h. he dervo begs wh he sgle se derece by kg he eeced vlue o s ollows: [ k] b [ ] w k (4.68) Mullyg ou, kg he eeced vlue hrough he sum, d usg he c h w,sce s o uco o w d equo Smlrly mullyg d smlyg leds o k, k k w s whe rocess, yelds he, (4.69) o q. (4.59) by j or j o, kg he eeced vlue,, j k j, k, j,3,, k (4.7) hus qs. (4.67), (4.69), d (4.7) gves se o equos h deermes he uocorrelo ucos lgs o erms o vrous oher uocorrelo ucos, d hese re he key equos solvg or he sedy se. I j, q. (4.7) holds s well. I we ssume h sedy se hs bee reched, he r s oly. So we c rewre qs. (4.69) d (4.7) s, k j k k j, j,,, re ucos o me derece r s (4.7) By he symmery roery o he uocorrelo uco or sory rdom rocess, j j, hese equos c be u o he ollowg mr orm: (4.7) he equos rereseed hs mr orm re clled he Yule-Wlker equos or he uoregressve rocess deed q.(4.59). I we kow he hrough, or her esmes, hese equos my be solves or he A rmeers,,,, d hus be used or sysem deco. d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 4 o 49

dom Sgl Processg Cher4 dom Processes he Levso recurso lgorhm [] c be used o eede he soluo, sce he coece mr s oelz mr. However, we hve he,,, vlues, meg he sysem s kow, d we w o solve or k re he ukow uocorrelos he lgs hrough : 3 he uocorrelo uco, we mus use q.(4.67) or so h he d rerrge he equos o (4.7) 3 4 3 b (4.73) mles 4.6 d 4.7 llusre hs rocedure or dg he uocorrelo uco or A() d A(). mle 4.6 Fd he me, uocorrelo, d vrce or he uoregressve rocess A() rdom rocess deed by b w, (4.74) Where s such h sble sysem s rereseed. Soluo From (4.67) d (4.69) he uocorrelo uco d mus ssy he wo equos,, b,, (4.75) I we ssume h sedy se hs bee reched, he he uocorrelo uco c be wre erms o me derece oly d wh he symmery roery o he uocorrelo ucos (4.75) becomes Solvg hese wo equos or b d b gves, b o ob he uocorrelo uco lgs greer h,q. (4.7) d j, we hve d he sedy se we ob, j j, (4.76) (4.77) (4.78) d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 4 o 49

dom Sgl Processg Cher4 dom Processes b j j j j, j (4.79) 4.9.3 Auoregressve Movg Averge Process, AMA (, q) For hs dscusso wll be ssumed h he goverg equo or movg verge rocess o order q, AMA, q gve q. (4.39) s deed s [ ] k or osve me de rom o d h w q k [ k] bk w[ k], (4.89) k w s whe sory rocess, o ecessrly, d vrce gve by w or ll, We Guss, wh or ll would lke o rlly chrcerze he resulg AMA, q rocess by dg s me, vrce, d uocorrelo uco. Me o AMA (, q). he me o he AMA, q rocess s esly oud by kg he eeced vlue o q. (4.89) o gve Sce w k [ ] [ k] b w[ k], k q k k (4.9) k or ll k,,, q, q. (4.9) c be wre s derece equo or wh zero drvg uco: [ ] [ k], k Sce ll l codos re ssumed equl o zero, he me k (4.9) or. Vrce o AMA (, q). Sce he me s zero or ll, he vrce c be wre erms o he uocorrelo uco evlued me dees d s q [ ] k b w k, k [ ] (4.9) k k O kg he eeced vlue q. (4.9), we c show he vrce o be hs resul c be wre s q (4.93) k [, k] bk w k k k q (4.94) k [, k] bk W, k k k d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 43 o 49

dom Sgl Processg Cher4 dom Processes hus he vrce s wre erms o he weghed sum o he uocorrelo uco evlued dere mes d he weghed sum o vrous cross correlos dere mes. hese erms wll be deermed he ollowg seco o he deermo o he uocorrelo uco or geerl AMA rocess. Auocorrelo or AMA (, q). he uocorrelo uco, geerl, goes hough rse erod sce he sgl srs zero l codos. o derve he uocorrelo uco, s ssumed h. he dervo begs wh he eeced vlue o he roduc o d, where s rom (4.89), s ollows: k b w k k Mullyg ou d kg he eeced vlue hrough he sum yelds he equo Smlrly mullyg d smlyg leds o k q k k, k bk w k k q k (4.95), (4.96) o q. (4.89) by j or j o, kg he eeced vlue, j k j, k bk w k] [ j,, j,3,..., (4.97) k q k hus qs. (4.96) d (4.97) gve se o equos h deerme he uocorrelo uco lgs o erms o vrous oher uocorrelo ucos. I we ssume h sedy se hs bee reched, he r s, re ucos o me derece r s oly. So we c rewre qs. (4.96) d (4.97) s j k k j j (, b) j,,..., k Where he j (, b) re he secod sums show qs. (4.96) d (4.97). (4.98) hese j (, b) he smuleous equos bove re rher comle oler uco o he AAM, q model rmeer vecors d b. Usg he symmery roery o he uocorrelo uco or sory rdom rocess, j j o he ollowg mr orm: hus he soluo or he uocorrelo uco, llows he equos o (4.98) o be u, b, b, b (4.99) k ll lgs s o loger soluo o smuleous ler equos bu soluo o se smuleous oler equos. he ollowg d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 44 o 49

dom Sgl Processg Cher4 dom Processes emle llusres he overll rocedure dg he me, vrce, d uocorrelo uco o he smles AMA rocess, mely he AMA(,) rocess. mle 4.8 Fd he () me, (b) uocorrelo uco, d (c) vrce or he AMA(,) rdom rocess gve by he derece equo b w b w, (4.) Assume h w s zero me whe rdom rocess wh vrce w jwk jk. Assume ll l codos or egve me de o be zero d h sedy se hs bee obed. Soluo () I ws show q. (4.9) h he me o AMA rocess s zero. hereore or ll. (b) he vrce or s he sme s he uocorrelo uco zero lg sce he me s zero. hereore we hve usg (4.93) h, bw b w, b w b w he ls wo erms he eresso bove wll be evlued serely. Subsug he derece equo or, mullyg ou, d kg he eeced vlue gves w bw bw w W b b w b w w b b b he rs eeced vlue he eresso bove s zero becuse s o uco o w he ls erm s zero becuse he w s whe sequece. he oher eeced vlue s gve s w b w b w w b (4.) (4.), d (4.3) Subsug he resuls rom (4.) d (4.3) o (4.), we lly ob he vrce s he,, b b b b (4.4) c be clculed by subsug he deo o he AMA(,) rocess o he eeced vlue;, bw b, bw b w he erm s zero becuse s o uco o w (4.5). he hrd erm bove d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 45 o 49

dom Sgl Processg Cher4 dom Processes w s deermed by relcg w w b w b w b by s equvle rom he deg equo: (4.6) hus, er subsug (4.6) o (4.5), he sgle lg uocorrelo uco o he AMA(,) rocess becomes,, b b (4.7) o ob he sedy se soluo, we ssume h he uocorrelo uco c be wre erms o me derece oly. hus (4.4) d (4.7) re wre s Solvg hese wo equos or uocorrelo uco o : b bb d b b b b b b b bb b b bb yelds he ollowg vlues or he o d he uocorrelo uco or hgher lds, j, we hve h, w d w s deermed s, bw bw, I smlr sho s esy o show h, j, j (4.8) (4.9) wh or lgs j (4.). hs roery s geerl or AMA, q rocesses wheever j s greer h q, evlug hs resul he sedy se yelds he uocorrelo uco or lgs greer h oe or he AMA(,) rocess s ollows; (c) he sedy se vrce s jus he j j, j (4.) gve (4.9), sce he me s zero. hus (4.) b b b b I becomes cresgly more dcul o ob closed orm soluos or he uocorrelo uco d vrce s he order d q o he AMA, q rocess s cresed. d Uversy Lu Cogeg -Ml:clu@ml.d.edu.c Pge 46 o 49