Stochastic Processes Adopted From p Chapter 9 Probability, Random Variables and Stochastic Processes, 4th Edition A. Papoulis and S.

Transcription

1 Sochasic Processes Adoped From p Chaper 9 Probabiliy, adom Variables ad Sochasic Processes, 4h Ediio A. Papoulis ad S. Pillai

2 9. Sochasic Processes Iroducio Le deoe he radom oucome of a experime. To every such oucome suppose a waveform,, is assiged. The collecio of such, waveforms form a, k sochasic i process. The se of { k } ad he ime, idex ca be coiuous, or discree couably ifiie or fiie as well. Fig. 9. For fixed S i he se of all experimeal oucomes,, is a specific ime fucio. For fixed,, i is a radom variable. The esemble of all such realizaios, over ime represes he sochasic

3 process. see Fig 9.. For example a cos, where is a uiformly disribued radom variable i,, represes a sochasic i process. Sochasic i processes are everywhere: Browia moio, sock marke flucuaios, various queuig sysems all represe sochasic pheomea. If is a sochasic process, he for fixed, represes a radom variable. Is disribuio fucio is give by F x, P{ x} 9- Noice ha F x, depeds o, sice for a differe, we obai a differe radom variable. Furher df x, f x, 9- dx represes he firs-order probabiliy desiy fucio of he process. 3

4 For = ad =, represes wo differe radom variables = ad d = respecively. Their joi disribuio ib i is give by F x, x,, P{ x, x} 9-3 ad f x, x,, F x, x,, x x 9-4 represes he secod-order desiy fucio of he process. Similarly f represes he h x, x, x,,, order desiy fucio of he process. Complee specificaio of he sochasic process requires he kowledge of f x, x, x,,, for all i, i,,, ad for all. a almos impossible ask i realiy. 4

5 Mea of a Sochasic Process: E { } x f x, dx 9-5 represes he mea value of a process. I geeral, he mea of a process ca deped o he ime idex. Auocorrelaio fucio of a process is defied as, E{ } x x f x, x,, dx dx ad i represes p he ierrelaioship pbewee he radom variables = ad = geeraed from he process. Properies:.,, [ E{ }]., E{ }. 9-7 Average isaaeous power 9-6 5

6 3., represes a oegaive defiie fucio, i.e., for ay se of cosas {a } { ai i i j Eq. 9-8 follows by oicig ha The fucio C a i a j,. 9-8 i j E{ Y } for Y i a i i,, 9-9 represes he auocovariace fucio of he process. Example 9 9. Le z d. T T. The T T E [ z ] E { } d d T T T T T T, d d 9-6

7 Example 9. 9., ~, cos U a 9- This gives, } {si si } {cos cos } {cos } { E a E a ae E 9- Similarly }. {si cos } {cos sice E d E Similarly } cos {cos, a E a } cos {cos a E a cos 9-3

8 Saioary Sochasic Processes Saioary processes exhibi saisical properies ha are ivaria o shif i he ime idex. Thus, for example, secod-order saioariy implies ha he saisical properies of he pairs {, } ad { +c, +c} are he same for ay c. Similarly firs-order saioariy implies ha he saisical properies of i ad i +c are he same for ay c. I sric erms, he saisical properies are govered by he joi probabiliy desiy fucio. Hece a process is h -order Sric-Sese Saioary S.S.S if f x, x, x,,, f x, x, x, c, c, c 9-4 for ay c,, where he lef side represes he joi desiy fucio of he radom variables,,, ad he righ side correspods o he joi desiy fucio of he radom variables c, c,, c. A process is said o be sric-sese saioary if 9-4 is 8 rue for all, i,,,,,, ad ay c. i

9 For a firs-order sric sese saioary process, from 9-4 we have f x, f x, c 9-5 for ay c. I paricular c = gives f x, f x 9-6 i.e., he firs-order desiy of f is idepede d of. I ha case E[ ] x f x dx, a cosa. Similarly, for a secod-order sric-sese saioary process we have from 9-4 f x, x,, f x, x, c, for ay c. For c = we ge f c 9-7 x, x,, f x, x, 9-8 9

10 i.e., he secod order desiy fucio of a sric sese saioary process depeds oly o he differece of he ime idices I ha case he auocorrelaio fucio is give by, E{ } x x f x, x, dx dx,. 9-9 i.e., he auocorrelaio fucio of a secod order sric-sese saioary yprocess depeds oly o he differece of he ime idices. Noice ha 9-7 ad 9-9 are cosequeces of he sochasic process beig firs ad secod-order sric sese saioary. O he oher had, he basic codiios for he firs ad secod order saioariy Eqs. 9-6 ad 9-8 are usually difficul o verify. I ha case, we ofe resor o a looser defiiio ii of saioariy, i i kow as Wide-Sese Saioariy W.S.S, by makig use of

11 9-7 ad 9-9 as he ecessary codiios. Thus, a process is said o be Wide-Sese Saioary if i E{ } 9- ad ii E{ }, 9- { i.e., for wide-sese saioary processes, he mea is a cosa ad he auocorrelaio fucio depeds d oly o he differece bewee he ime idices. Noice ha does o say ayhig abou he aure of he probabiliy desiy fucios, ad isead deal wih he average behavior of he process. Sice follow from 9-6 ad 9-8, sric-sese saioariy always implies wide-sese saioariy. However, he coverse is o rue i geeral, he oly excepio beig he Gaussia process. This follows, sice if is a Gaussia process, he by defiiio,,, are jil joily Gaussia radom variables for ay,, whose joi characerisic fucio is give by

12 j k k C, / i k ik k l, k,,, e where h C i, is i as defied d d o If i is wide-sese k saioary, he usig i 9- we ge,,, e k i k ik k k j C ad hece if he se of ime idices are shifed by a cosa c o geerae a ew se of joily Gaussia radom variables c, c,, c he heir joi characerisic fucio is ideical o 9-3. Thus he se of radom variables ibl { i } i ad { i} i have he same joi probabiliy disribuio for all ad all c, esablishig he sric sese saioariy of Gaussia processes from is wide-sese saioariy. To summarize if is a Gaussia process, he wide-sese saioariy w.s.s sric-sese sese saioariy s.s.s. Noice ha sice he joi p.d.f of Gaussia radom variables depeds oly o heir secod order saisics, which is also he basis

13 for wide sese saioariy, we obai sric sese saioariy as well. From , refer o Example 9., he process a cos, i 9- is wide-sese saioary, i bu b o sric-sese saioary. Similarly if is a zero mea wide sese saioary process i Example 9., he i 9- reduces o z z T E{ z } dd. T T T As, varies from TT o +T, varies Fig. 9. from T o + T. Moreover is a cosa over he shaded regio i Fig 9., whose area is give by T T d T d ad hece he above iegral reduces o z T T d T T T T T d. 9-4 T T 3

14 Sysems wih Sochasic Ipus A deermiisic sysem rasforms each ipu waveform, io i a oupu waveform Y, i T[, i ] by operaig oly o he ime variable. Thus a se of realizaios a he ipu correspodig o a process geeraes a ew se of realizaios { Y, } a he oupu associaed wih a ew process Y., i Y, i T[] [ ] Y Fig. 9.3 Our goal is o sudy he oupu process saisics i erms of he ipu process saisics ad he sysem fucio. A sochasic sysem o he oher had operaes o boh he variables ad. 4

15 Deermiisic Sysems Memoryless Sysems Y g[ ] Sysems wih Memory Fig. 9.3 Time-varyig sysems Time-Ivaria sysems Liear sysems Y L [ ] h LTI sysem Liear-Time Ivaria LTI sysems Y h d h d. 5

16 Memoryless Sysems: The oupu Y i his case depeds oly o he prese value of he ipu. i.e., Y g{ } 9-5 Sric-sese saioary ipu Memoryless sysem Sric-sese saioary oupu. see 9-76, Tex for a proof. Wide-sese saioary ipu saioary Gaussia wih Memoryless sysem Memoryless sysem Fig. 9.4 Need o be saioary i ay sese. Y saioary,bu o Gaussia wih Y. see

17 Theorem: If is a zero mea saioary Gaussia process, ad Y = g[], where g represes a oliear memoryless device, he Y, E{ g }. 9-6 Proof: Y E{ Y } E[ g{ x g x f x, x dx dx }] 9-7 where, are joily Gaussia radom variables, ad hece A f x, x e xa x/,, x x, x T A E{ } LL T 7

18 where L is a upper riagular facor marix wih posiive diagoal eries. i.e., l l L. l Cosider he rasformaio so ha Z L Z, Z, z L x z, z E{ Z Z T } L E{ } L L AL I T ad dhece Z, Z are zero mea id idepede d Gaussia radom d variables. Also x Lz x l z l z, x l z ad hece A L A L. xa x z LA Lz z z z z The Jacobaia of he rasformaio is give by 8

19 J L A / Hece subsiuig hese io 9-7, we obai /. / / Y J A z z l z l z g l z e e l z g l z f z f z dz dz z z l z g l z f z f z dz dz z z z z l z f z dz g l z f z dz l z g l z f z z dz z / e l l u / where u l z This gives. ug u e du, l 9

20 f u u l u / l l g u e du Y l l u dfu u fu u du g u f u du, u sice A LL gives l l Y. { g u f u u E{ g } Hece, g u f u u du } he desired resul, where E[ g ]. Thus if he ipu o a memoryless device is saioary Gaussia, he cross correlaio fucio bewee he ipu ad he oupu is proporioal o o he ipu auocorrelaio fucio.

21 Liear Sysems: L[] represes a liear sysem if Le L{ a a } a L { } a L { { Y L{ } } represe he oupu of a liear sysem. Time-Ivaria Sysem: L[] represes a ime-ivaria sysem if Y L{ } L{ } Y 9-3 i.e., shif i he ipu resuls i he same shif i he oupu also. If L[] saisfies boh 9-8 ad 9-3, he i correspods o a liear ime-ivaria LTI sysem. LTI sysems ca be uiquely represeed i erms of heir oupu o h a dela fucio LTI h Impulse Fig. 9.5 Impulse respose Impulse respose of he sysem

22 he Y Y LTI Y h d arbirary Fig ipu h d 9-3 Eq. 9-3 follows by expressig as d ad applyig 9-8 ad9-3 o Y L { }. Thus Y L{ } L{ L{ L{ d} } d h d d} h By Lieariy By Time-ivariace d

23 Oupu Saisics: Usig 9-33, he mea of he oupu process is give by g y } { } { h d h d h E Y E Y 9 34 Similarly he cross-correlaio fucio bewee he ipu ad oupu i i b. h d h 9-34 processes is give by } {, Y E Y } { } { d h E d h E, h d h Fially he oupu auocorrelaio fucio is give by., h 9-35

24 YY, E{ Y Y } E { h d Y } E{ Y } h d or Y Y,, h, h d 9-36 YY,, h h h Y a, h, Y h, b YY Fig

25 I paricular if is wide-sese saioary, he we have so ha from 9-34 h d c, Y a cosa. Also, so ha 9-35 reduces o 9-38 Y, h d h,. Y 9-39 Thus ady are joily w.s.s. Furher, from 9-36, he oupu auocorrelaio simplifies o, h d, YY Y h. 9-4 Y From 9-37, we obai YY YY h h

26 From , he oupu process is also wide-sese saioary. This gives rise o he followig represeaio wide-sese saioary process sric-sese saioary i process Gaussia process also saioary LTI sysem h a LTI sysem h b Liear sysem c Fig. 9.8 Y wide-sese saioary process. Y sric-sese saioary process see Tex for proof Y Gaussia process also saioary 6

27 Whie Noise Process: W is said o be a whie oise process if WW, q, 9-4 ie i.e., E[W W ] = uless =. W is said o be wide-sese saioary w.s.s whie oise if E[W] = cosa, ad, q q WW If W is also a Gaussia process whie Gaussia process, he all of is samples are idepede radom variables why?. Whie oise W LTI h Fig For w.s.s. whie oise ipu W, we have Colored o oise N h W 7

28 EN [ ] h d, acosa W 9-44 ad q h h qh h q 9-45 where h h h h d Thus he oupu of a whie oise process hrough a LTI sysem represes a colored oise process. Noe: Whie oise eed o be Gaussia. Whie ad Gaussia are wo differe coceps! 8

29 Upcrossigs ad Dowcrossigs of a saioary Gaussia process: Cosider a zero mea saioary Gaussia process wih auocorrelaio fucio. A upcrossig over he mea value occurs wheever he realizaio passes hrough hzero wih posiive slope. Le Upcrossigs represe he probabiliy of such a upcrossig i he ierval,. We wish o deermie. Dowcrossig Fig. 9. Sice is a saioary Gaussia process, is derivaive process is also zero mea saioary i Gaussia wih auocorrelaio i fucio see , Tex. Furher ad are joily Gaussia saioary processes, ad sice see 9-6, Tex d, 9 d

30 we have d d 9-47 d d which for gives E [ ] i.e., he joily Gaussia zero mea radom variables are ucorrelaed ldad dhece id idepeded wih variaces 9-48 ad 9-49 ad respecively. Thus x x 9-5 f x x x x e. 9-5, f f To deermie, he probabiliy of upcrossig rae, 3

31 we argue as follows: I a ierval,, he realizaio moves from = o, ad hece he realizaio iersecs wih he zero level somewhere i ha ierval if,, ad i.e.,. Hece he probabiliy of upcrossig i, is give by x f x, x dxdx x x Fig f x dx x dx f x Differeiaig boh sides of 9-53 wih respec o, we ge f x x f x dx ad leig, Eq reduce o

32 x f x f dx / x f x dx 9-55 [where we have made use of 5-78, Tex]. There is a equal probabiliy for dowcrossigs, ad hece he oal probabiliy for crossig he zero lie i a ierval, equals, where / I follows ha i a log ierval T, here will be approximaely T crossigs of he mea value. If is large, he he auocorrelaio fucio decays more rapidly as moves away from zero, implyig a large radom variaio aroud he origi mea value for, ad he likelihood lih of zero crossigs should icrease wih icrease i, agreeig wih

33 Discree Time Sochasic Processes: A discree ime sochasic process = T is a sequece of radom variables. The mea, auocorrelaio ad auo-covariace fucios of a discree-ime process are gives by add E { T } 9-57, E{ T T } C,, respecively. As before sric sese saioariy ad wide-sese saioariy defiiios apply here also. For example, T is wide sese saioary if ad E { T }, a cosa 9-6 E k T kt r r [ { } { }]

34 i.e.,, = =. The posiive-defiie propery of he auocorrelaio sequece i 9-8 ca be expressed i erms of cerai Hermiia-Toepliz marices as follows: Theorem: A sequece { r } forms a auocorrelaio sequece of a wide sese saioary sochasic process if ad oly if every Hermiia-Toepliz ii marix T give by T r r r r r r r r r r r T 9-6 is o-egaive posiive defiie for,,,,. T Proof: Le a [ a, a,, a ] represe a arbirary cosa vecor. The from 9-6, a T a a a r 9-63 i k k i i k T i, k rk i sice he Toepliz characer gives. Usig 9-6, Eq reduces o r 34

35 a T a aa i ke { kt it } E a k kt. i k k 9-64 From 9-64, if T is a wide sese saioary sochasic process he T is a o-egaive defiie i marix for every,,,,. Similarly he coverse also follows from see secio 9.4, Tex If T represes a wide-sese saioary ipu o a discree-ime sysem {ht}, ad YT he sysem oupu, he as before he cross correlaio fucio saisfies h 9-65 Y ad he oupu auocorrelaio fucio is give by h 9-66 YY Y or h h. YY 9-67 Thus wide-sese saioariy from ipu o oupu is preserved 35 for discree-ime sysems also.

36 Auo egressive Movig Average AMA Processes Cosider a ipu oupu represeaio p ak k bk W k q k, 9-68 where may be cosidered as he oupu of a sysem {h} drive by he ipu W. Z rasform of W h 9-68gives Fig.9. or p q k k k k k k z a z W z b z, a q b z bz bz b z k q hkz p k W z az az ap z k 9-69 B z H z A z

37 represes he rasfer fucio of he associaed sysem respose {h} i Fig 9. so ha h k W k. 9-7 k Noice ha he rasfer fucio Hz i 9-7 is raioal wih p poles ad q zeros ha deermie he model order of he uderlyig sysem. From 9-68, he oupu udergoes regressio over p of is previous values ad a he same ime a movig average based o W, W,, W q of he ipu over q + values is added o i, hus geeraig a Auo egressive Movig Average AMA p, q process. Geerally he ipu {W} represes a sequece of ucorrelaed radom variables of zero mea ad cosa variace W so ha. 9-7 WW W If i addiio, {W} is ormally disribued he he oupu {} also represes a sric-sese sese saioary ormal process. If q =, he 9-68 represes a Ap process all-pole 37 process, ad if p =, he 9-68 represes a MAq

38 process all-zero process. Nex, we shall discuss A ad A processes hrough explici calculaios. A process: A A process has he form see 9-68 a W 9-73 ad from 9-7 he correspodig sysem rasfer H z az a z 9-74 provided a <. Thus h a, a 9-75 represes he impulse respose of a A sable sysem. Usig 9-67 ogeher wih 9-7 ad 9-75, we ge he oupu auocorrelaio sequece of a A process o be k k a { a } { a } a a W W W k a

39 where we have made use of he discree versio of The ormalized i erms of oupu auocorrelaio sequece is give by a, I is isrucive o compare a A model discussed above by superimposig a radom compoe o i, which may be a error erm associaed wih observig a firs order A process. Thus Y V 9-78 where ~ A as i 9-73, ad dv V is a ucorrelaed ldradom sequece wih zero mea ad variace V ha is also ucorrelaed wih {W}. From 9-73, 9-78 we obai he oupu auocorrelaio of he observed process Y o be YY VV a W V 9-79 a V 39

40 so ha is ormalized versio is give by YY Y ca,, 9-8 YY where W c. a 9-8 W V Eqs ad 9-8 demosrae he effec of superimposig p a error sequece o a A model. For o-zero lags, he auocorrelaio of he observed sequece {Y}is reduced by a cosa facor compared o he origial process {}. From 9-78, he superimposed Y error sequece V oly affecs he correspodig erm i Y k k Y erm by erm. However, a paricular erm i he ipu sequece k W affecs ad Y as well as 4 all subseque observaios. Fig. 9.3

41 A Process: A A process has he form a a W 9-8 ad from 9-7 he correspodig rasfer fucio is give by H a z h z az z z z so ha h, h a, h ah ah, ad i erm of he poles ad of he rasfer fucio, from 9-83 we have h b b, ha represes he impulse respose of he sysem. From , 85, we also have b b, b b From 9-83, a, a, 9-86 b b a

42 ad Hz sable implies Furher, usig 9-8 he oupu auocorrelaios saisfy he recursio., } { E } { } ] {[ } { W E m m a m a E m m E ad hece heir ormalized versio is give by 9-87 } { a a m m W E ad hece heir ormalized versio is give by B di l l i i 9 67 h l i a a By direc calculaio usig 9-67, he oupu auocorrelaios are give by h h h h W WW k k h k h W b b b b b b W 9 89

43 where we have made use of From 9-89, he ormalized oupu auocorrelaios may be expressed as c c 9-9 where ecc ad c are appropriae ecosas. s. Damped Expoeials: Whe he secod order sysem i is real ad correspods o a damped expoeial respose, he poles are complex cojugae which gives a 4 a i Thus,, j re r j I ha case c c ce i 9-9 so ha he ormalized correlaios here reduce o e{ c } cr cos. Bu from 9-86 r cos a, r a,

44 ad hece rsi a 4 a which gives Also from 9-88 so ha a 4 a a a a a a a a cr cos a where he laer form is obaied from 9-9 wih =. Bu i 9-9 gives ccos, or c / cos Subsiuig io ad we obai he ormalized oupu auocorrelaios o be

45 / cos a, a cos where saisfies cos cos a a a 9-97 Thus he ormalized auocorrelaios of a damped secod order sysem wih real coefficies subjec o radom ucorrelaed impulses saisfy More o AMA processes From 9-7 a AMA p, q sysem has oly p + q + idepede coefficies, ak, k p, bi, i q, ad hece is impulse respose sequece {h k } also mus exhibi a similar depedece amog hem. I fac accordig o P. Diees The Taylor series, 93, 45

46 a old resul due o Kroecker 88 saes ha he ecessary ad k sufficie codiio for H z o represe a raioal k h k z sysem AMA is ha de H, N for all sufficiely large, 9-99 where h h h h h h h3 h H. 9- h h h h i.e., I he case of raioal sysems for all sufficiely large, he Hakel marices H i 9- all have he same rak. The ecessary par easily follows from 9-7 by cross muliplyig k ad equaig coefficies of like powers of z, k,,,. Amog oher higs God creaed he iegers ad he res is he work of ma. Leopold Kroecker 46

47 This gives b h b h a h b h a ha h q q q m qi qi qi qi 9- ha ha h a h, i. 9- For sysems wih i 9- we ge q p, leig i pq, pq,, pq ha ha h a h p p p p ha h a h a h 9-3 p p p p p p which gives de H p =. Similarly i pq, gives 47

48 ha p ha p hp ha ha h p p p h a h a h 9-4 p p p p p, ad ha gives de H p+ = ec. Noice ha apk, k,, For sufficiecy proof, see Diees. I is possible o obai similar il deermiaial i codiios i for AMA sysems i erms of Hakel marices geeraed from is oupu auocorrelaio sequece. eferrig back o he AMA p, q model i 9-68, he ipu whie oise process w here is ucorrelaed wih is ow pas sample values as well as he pas values of he sysem oupu. This gives Eww { k }, k 9-5 Ewx k k { },

49 Togeher wih 9-68, we obai r E xx i i { i } p q ak{ x k x i} bk{ w k w i} k k p q ar k ik bk{ w kx i} 9-7 k k ad hece i geeral ad p ar k ik ri, i q 9-8 k p k ar r, i q. k ik i 9-9 Noice ha 9-9 is he same as 9- wih {h k } replaced 49

50 by {r k } ad hece he Kroecker codiios for raioal sysems ca be expressed i erms of is oupu auocorrelaios as well. Thus if ~ AMA p, q represes a wide sese saioary sochasic process, he is oupu auocorrelaio sequece {r k } saisfies s where rak D rak D p, k, 9- D k p pk r r r rk r r r r k r r r r 3 k k k k 9- represes he k k Hakel marix geeraed from r, r,, r k,, r k. I follows ha for AMA p, q sysems, we have de D, for all sufficiely large. 9-5