Probability. Stochastic Processes

Transcription

1 Pobablty ad Stochastc Pocesses Weless Ifomato Tasmsso System Lab. Isttute of Commucatos Egeeg g Natoal Su Yat-se Uvesty

2 Table of Cotets Pobablty Radom Vaables, Pobablty Dstbutos, ad Pobablty Destes Statstcal Aveages of Radom Vaables Some Useful Pobablty Dstbutos Uppe Bouds o the Tal Pobablty blt Sums of Radom Vaables ad the Cetal Lmt Theoem Stochastc Pocesses Statstcal Aveages Powe Desty Spectum Respose of a Lea Tme-Ivaat System to a Radom Iput Sgal Dscete-Tme Stochastc Sgals ad Systems Cyclostatoay Pocesses

3 Pobablty Sample space o ceta evet of a de epemet: S,,3,4,5,6 The s outcomes ae the sample pots of the epemet. A evet s a subset of S, ad may cosst of ay umbe of sample pots. Fo eample: A,4 The complemet of the evet A, deoted by A, cossts of all the sample pots S that ae ot A: A,3,5,6 3

4 Pobablty Two evets ae sad to be mutually eclusve f they have o sample pots commo that s, f the occuece of oe evet ecludes the occuece of the othe. Fo eample:,4;,3,6 A B A ad A ae mutually eclusve evets. The uo sum of two evets a evet that cossts of all the sample pots the two evets. Fo eample: C,,3 D A B C,,3, 6 A S 4

5 Pobablty The tesecto of two evets s a evet that cossts of the pots that ae commo to the two evets. Fo eample: E B C,3 Whe the evets ae mutually eclusve, the tesecto s the ull evet, deoted as φ. Fo eample: A A 5

6 Pobablty Assocated wth each evet A cotaed S s ts pobablty PA. Thee postulatos: PA0. The pobablty of the sample space s PS=. Suppose that A, =,,, ae a possbly fte umbe of evets the sample space S such that A A ;,,... The the pobablty of the uo of these mutually eclusve evets satsfes the codto: P A P A 6

7 Pobablty Jot evets ad ot pobabltes two epemets If oe epemet has the possble outcomes A, =,,,, ad the secod epemet has the possble outcomes B, =,,,m, the the combed epemet has the possble ot outcomes A,B, =,,,, =,,,m. Assocated wth each ot outcome A,B s the ot pobablty P A,B whch satsfes the codto: 0 P A, B Assumg that the outcomes B, =,,,m, ae mutually eclusve, t follows that: m P A, B P A If all the outcomes of the two epemets ae mutually eclusve, the: m P A, B P A 7

8 Pobablty Codtoal pobabltes The codtoal pobablty of the evet A gve the occuece of the evet B s defed as: P A, B P A B P B povded PB>0. P A, B P A B P B P B A P A P A, B s t tepetedt as the pobablty blt of A B. That s, P A, B deotes the smultaeous occuece of A ad B. If two evets A ad B ae mutually eclusve, A B, the PAB 0. If B s a subset of A, we have A B B ad P A B. 8

9 Pobablty Bayes theoem: If A,,,...,, ae mutually eclusve evets such that A S ad B s a abtay evet wth ozeo pobablty, the PA B PAB, PB P B A P A PB A PA, P B P B A P B A P A PA epesets the a po pobabltes ad PA B s the a posteo pobablty of A codtoed o havg obseved the eceved sgal B. 9

10 Pobablty Pobablty Statstcal depedece. the, of occuece o the ot deped does of occuece the If A P B A P B A Whe the evets A ad B satsfy the elato, B P A P B P B A P B A P PA,B=PAPB, they ae sad to be statstcally depedet. Thee statstcally depedet evets A, A, ad A 3 must satsfy the followg codtos:, A P A P A A P,, A P A P A A P A P A P A A P 0,, 3 3 A P A P A P A A A P

11 Radom Vaables, Pobablty Dstbutos, ad Pobablty Destes es Gve a epemet havg a sample space S ad elemets s S, we defe a fucto X s whose doma s S ad whose age s a set of umbes o the eal le. The fucto Xs s called a adom vaable. Eample : If we flp a co, the possble outcomes ae head H ad tal T, so S cotas two pots labeled H ad T. Suppose we defe a fucto Xs such that: s H X s - s T Thus we have mapped the two possble outcomes of the co-flppg epemet to the two pots +,- o the eal le. Eample : Tossg a de wth possble outcomes S={,,3,4,5,6}. A adom vaable defed o ths sample space may be Xs=s, whch case the outcomes of the epemet ae mapped to the teges,,6, o, pehaps, Xs=s, whch case the possble outcomes ae mapped to the teges {,4,9,6,5,36}.

12 Radom Vaables, Pobablty Dstbutos, ad Pobablty Destes es Gve a adom vaable X, let us cosde the evet {X } whee s ay eal umbe the teval -,. We wte the pobablty of ths evet as PX ad deote t smply by F,.e., F P X, - The fucto F s called the pobablty dstbuto fucto of the adom vaable X. It s also called the cumulatve lt dstbuto dtbt fucto CDF. 0 F F 0 ad F.

13 Radom Vaables, Pobablty Dstbutos, ad Pobablty Destes es Eamples of the cumulatve dstbuto fuctos of two dscete adom vaables. 3

14 Radom Vaables, Pobablty Dstbutos, ad Pobablty Destes es A eample of the cumulatve dstbuto fucto of a cotuous adom vaable. 4

15 Radom Vaables, Pobablty Dstbutos, ad Pobablty Destes es A eample of the cumulatve dstbuto fucto of a adom vaable of a med type. 5

16 Radom Vaables, Pobablty Dstbutos, ad Pobablty Destes es The devatve of the CDF F, deoted as p, s called the pobablty desty fucto PDF of the adom vaable X. p F df, d p u du, Whe the adom vaable s dscete o of a med type, the PDF cotas mpulses at the pots of dscotuty of F: p P X 6

17 Radom Vaables, Pobablty Dstbutos, ad Pobablty Destes es Detemg the pobablty that a adom vaable X falls a teval,, whee. P X P X P X F F P X P X F F p d The pobablty of the evet X s smply the aea ude the PDF the age X. 7

18 Radom Vaables, Pobablty Dstbutos, ad Pobablty Destes es Multple adom vaables, ot pobablty dstbutos, ad ot pobablty destes: two adom vaables Jot CDF : F, P X, X p u, u dudu Jot PDF : p, F, - - p, d p p, d p The PDFs p ad p obtaed of the vaables ae called - - p, dd magal, fom tegat g ove oe PDFs., F, F, 0. Note that : F F 8

19 Radom Vaables, Pobablty Dstbutos, ad Pobablty Destes es Multple adom vaables, ot pobablty dstbutos, ad ot pobablty destes: multdmesoal adom vaables Suppose that Jot CDF Jot PDF F F X F p,,,...,, ae adom vaables.,, p,...,,...,,,,...,, P X d d,,, 4,..., F,,,..., 0., 4 3, 4, X p p u, 5, u F, 4,...,,,...,,..., u,...,,. X,...,, du du... du 9

20 Statstcal Aveages of Radom Vaables The mea o epected value of X, whch chaactezed by ts PDF p, s defed as: E X m p d Ths s the fst momet of adom vaable X. The -th momet s defed as: E X p d Defe Y=gX, the epected value of Y s: g X E Y E g p d 0

21 Statstcal Aveages of Radom Vaables The -th cetal momet of the adom vaable X s: E Y E X m m p d Whe =, the cetal momet s called the vaace of the adom vaable ad deoted as : E m p d E X E X m X I the case of two adom vaables, X ad X, wth ot PDF p,, we defe the ot momet as: k k X p, E X d d

22 Statstcal Aveages of Radom Vaables The ot cetal momet s defed as: E k X m X m k m m p, dd If k==, the ot momet ad ot cetal momet ae called the coelato ad the covaace of the adom vaables X ad X, espectvely. The coelato betwee X ad X s gve by the ot momet: E X X p, d d

23 Statstcal Aveages of Radom Vaables The covaace betwee X ad X s gve by the ot cetal momet: E[ X m X m ] m m E X X m m p, p, m d m d d d p, m m d m m m m d p, m m d d m m The mat wth elemets μ s called the covaace mat of the adom vaables, X, =,,,. 3

24 Statstcal Aveages of Radom Vaables Two adom vaables ae sad to be ucoelated f EX X =EX EX =m m. Ucoelated Covaace μ = 0. If X ad X ae statstcally depedet, they ae ucoelated. If X ad X ae ucoelated, they ae ot ecessay statstcally ttt depedetly. d d Two adom vaables ae sad to be othogoal f EX X =0. Two adom vaables ae othogoal f they ae ucoelated ad ethe oe o both of them have zeo mea. 4

25 Statstcal Aveages of Radom Vaables Chaactestc fuctos The chaactestc fucto of a adom vaable X s defed as the statstcal aveage: vx E e v e v p d Ψv may be descbed as the Foue tasfom of p. The vese Foue tasfom s: p v e v Fst devatve of the above equato wth espect to v: d v v e p d dv 5 dv

26 Statstcal Aveages of Radom Vaables Chaactestc fuctos cot. Fst momet mea ca be obtaed by: E X m d v dv v 0 Sce the dffeetato pocess ca be epeated, -th momet ca be calculated by: E X d v dv v0 6

27 Statstcal Aveages of Radom Vaables Chaactestc fuctos cot. Detemg the PDF of a sum of statstcally depedet adom vaables: vy Y X Y v E e Eep v X E Sce the adom vaables ae statstcally depedet, vx v e... e p,,..., d d... d p If,,...,, p p... p Y v X X ae d depedet ad detcally dstbuted v v Y X v 7

28 Statstcal Aveages of Radom Vaables Chaactestc fuctos cot. The PDF of Y s detemed fom the vese Foue tasfom of Ψ Y v. Sce the chaactestc fucto of the sum of statstcally depedet adom vaables s equal to the poduct of the chaactestc fuctos of the dvdual adom vaables, t follows that, the tasfom doma, the PDF of Y s the - fold covoluto of the PDFs of the X. Usually, the -fold covoluto s moe dffcult to pefom tha the chaactestc fucto method detemg the PDF of Y. 8

29 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos Bomal dstbuto dscete: Let p X P X P 0 whee the ae statstcally d Y X X what s the pobablty dstbuto fucto of Y? whee the,,,..., ae statstcally d, Y X X!!! k k k p p k k Y P k k : s of PDF 0 k y k Y P y p Y k 0 k y p p k k k k 9 0 k k

30 Some Useful Pobablty Dstbutos Bomal dstbuto: The CDF of Y s: y k k F y P Y y p p k 0 k whee [y] deotes the lagest tege m such that m y. The fst two momets of Y ae: E Y p E Y p p The chaactestc fucto s: p p p v p pe 30

31 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos Ufom Dstbuto The fst two momets of X ae: b a X E 3 b ab b a X E The chaactestc fucto s: a b The chaactestc fucto s: e e va vb 3 a b v

32 Some Useful Pobablty Dstbutos Gaussa Nomal Dstbuto The PDF of a Gaussa o omal dstbuted adom vaable s: m / p e whee m s the mea ad σ s the vaace of fthe adom vaable. u m du t, dt The CDF s: F e du m m ef efc m um / / 3 e t dt

33 Some Useful Pobablty Dstbutos Gaussa Nomal Dstbuto ef ad efc deote the eo fucto ad complemetay eo fucto, espectvely, ad ae defed as: ef t t e dt ad efc e dt ef 0 ef-=-ef, efc-=-efc, ef0=efc =0, ad ef =efc0=. efc0 Fo >m, the complemetay eo fuctos s popotoal to the aea ude the tal of the Gaussa PDF. 33

34 Some Useful Pobablty Dstbutos Gaussa Nomal Dstbuto The fucto that s fequetly used fo the aea ude the tal of the Gaussa PDF s deoted by Q ad s defed as: t / Q e dt efc 0 34

35 Some Useful Pobablty Dstbutos Gaussa Nomal Dstbuto The chaactestc fucto of a Gaussa adom vaable wth mea m ad vaace σ s: v m / vm / v v e e d e The cetal momets of a Gaussa adom vaable ae: E k 3 X m k k k eve k 0 odd k The oday momets may be epessed tems of the cetal momets as: k k k E X m k 0 35

36 Some Useful Pobablty Dstbutos Gaussa Nomal Dstbuto The sum of statstcally depedet Gaussa adom vaables s also a Gaussa adom vaable. whee Y X Y m Theefoe, v v y m Y s ad vaace. X ad Gaussa y y e vm v / vm y v - dstbuted wth mea e m y y / 36

37 Some Useful Pobablty Dstbutos Ch-squae dstbuto If Y=X, whee X s a Gaussa adom vaable, Y has a chsquae dstbuto. Y s a tasfomato of X. Thee ae two type of ch-squae dstbuto: Cetal ch-squae dstbuto: X has zeo mea. No-cetal ch-squae dstbuto: X has o-zeo mea. Assumg X be Gaussa dstbuted wth zeo mea ad vaace σ, we ca apply.-47 to obta the PDF of Y wth a= ad b=0; p Y y p y b / a ] p X [ y b / ] y b/ a ] g[ y b / a ] X [ a g[ 37

38 Some Useful Pobablty Dstbutos Cetal ch-squae dstbuto The PDF of Y s: p Y y / y e, y y 0 The CDF of Y s: F Y y y u p du Y 0 0 y e u u / du The chaactestc fucto of Y s: v Y v / 38

39 Some Useful Pobablty Dstbutos Ch-squae Gamma dstbuto wth degees of feedom. Y X, X,,,...,, ae statstcally depedet ad detcally dstbuted d Gaussa adom vaables wth zeo mea ad vaace. The chaactestc fucto s: Y v / v The vese tasfom of ths chaactestc fucto yelds the PDF: / -y/ σ py y y e, y 0 / σ 39

40 Some Useful Pobablty Dstbutos Ch-squae Gamma dstbuto wth degees of feedom cot.. p s the gamma fucto, defed as : p p t e p 0 t dt, p p -! p a tege 0 3 Whe =, the dstbuto yelds the epoetal dstbuto. 0 40

41 Some Useful Pobablty Dstbutos Ch-squae Gamma dstbuto wth degees of feedom cot.. The PDF of a ch-squae dstbuted adom vaable fo seveal degees of feedom. 4

42 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos Ch-squae Gamma dstbuto wth degees of feedom cot.. The fst two momets of Y ae: 4 4 Y E Y E Th CDF f Y 4 y The CDF of Y s: y u y du e u y F / / 0 Y y du e u y F 0 / 0, 4

43 Some Useful Pobablty Dstbutos Ch-squae Gamma dstbuto wth degees of feedom cot.. The tegal CDF of Y ca be easly mapulated to the fom of the complete gamma fucto, whch hs tabulated by Peaso 965. Whe s eve, the tegal ca be epessed closed fom. Let m=/, whee m s a tege, we ca obta: F Y m y / y y e k! k 0 k, y 0 43

44 Some Useful Pobablty Dstbutos No-cetal ch-squae dstbuto If X s Gaussa wth mea m ad vaace σ, the adom vaable Y=X has the PDF: p Y / ym ym y e cosh y, y 0 The chaactestc fucto coespodg to ths PDF s: Y v v / e m v / v 44

45 Some Useful Pobablty Dstbutos No-cetal ch-squae dstbuto wth degees of feedom Y X, X,,,...,, ae statstcally depedet ad detcally dstbuted d Gaussa adom vaables wth mea m,,,...,, ad detcal vaace equal The chaactestc fucto s: v Y v / v ep m v to. 45

46 Some Useful Pobablty Dstbutos No-cetal ch-squae dstbuto wth degees of feedom The chaactestc fucto ca be vese Foue tasfomed to yeld the PDF: y /4 s y/ s py y e I / y, y 0 s whee, s s called the o-cetalty paamete: s m ad I s the th-ode modfed Bessel fucto of the fst kd, whch may be epeseted by the fte sees: k / I, 0 k 0 k! k 46

47 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos No-cetal ch-squae dstbuto wth degees of feedom The CDF s: y u s Y du s u I e s u y F 0 / / 4 / The fst two momets of a o-cetal ch-squaedstbuted adom vaable ae: s dstbuted adom vaable ae: s Y E s s s Y E 47 4 s y

48 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos No-cetal ch-squae dstbuto wth degees of feedom Whe m=/ s a tege, the CDF ca be epessed tems of the geealzed Macum s Q fucto: /, m a m m b k m Q a b e I a d a b / 0 /, m a b k k k a b b Q a b e I ab a b / 0 whee,, 0 By usg ad let t s easly show: a b k k b Q a b e I ab b a a u s a By usg ad let, t s easly show: a, Y m y s F y Q 48

49 Some Useful Pobablty Dstbutos Raylegh dstbuto Raylegh dstbuto s fequetly used to model the statstcs of sgals tasmtted though ado chaels such as cellula ado. Cosde a cae sgal s at a fequecy ω 0 ad wth a ampltude a: s a ep 0t The eceved sgal s s the sum of waves: t ep t s a ep 0 0 whee ep 0 a ep 49

50 Some Useful Pobablty Dstbutos Raylegh dstbuto Defe : ep a cos a s y We have : a cos ad y a s whee : y cos y s Because s usually vey lage, the dvdual ampltudes a ae adom, ad 3 the phases θ have a ufom dstbuto, t ca be assumed that fom the cetal lmt theoem ad y ae both Gaussa vaables wth meas equal to zeo ad vaace: y 50

51 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos Raylegh dstbuto Raylegh dstbuto Because ad y ae depedet adom vaables, the ot dstbuto p,y s p,y ep, y y p p y p The dstbuto p,θ ca be wtte as a fucto of p,y : y p J p s cos / /,, J y p J p ep cos s / / p y y 5 ep, p

52 Some Useful Pobablty Dstbutos Raylegh dstbuto Thus, the Raylegh dstbuto has a PDF gve by: / e 0 p, R p d 0 0 othewse The pobablty that the evelope of the eceved sgal does ot eceed a specfed value R s gve by the coespodg cumulatve dstbuto fucto CDF: F R 0 u u / / e du ep, 0 5

53 Some Useful Pobablty Dstbutos Raylegh dstbuto Mea: mea E[ R] p d. 533 Vaace: E[ R 0 ] E [ R] 0 p d 0.49 meda p d 0.77 k k / k E[ R ] Meda value of s foud by solvg: meda Moets of R ae: Most lkely value:= ma { p R } =σ. 53

54 Some Useful Pobablty Dstbutos Raylegh dstbuto 54

55 Some Useful Pobablty Dstbutos Raylegh dstbuto Pobablty That Receved Sgal Does t Eceed A Ceta Level R FR p u du u 0 0 ep ep u u 0 ep du 55

56 Some Useful Pobablty Dstbutos Raylegh dstbuto 0-0=0 Mea value: E[ R] p d mea 0 ep d d ep 0 0 ep ep d 0 0 ep d

57 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos Raylegh dstbuto: y g Mea squae value: 3 0 ] [ d p R E 0 0 ep ep d d 0-0=0 0 0 ep ep d 0-0=0 ep 57 0

58 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos Raylegh dstbuto y g Vaace: ] [ ] [ R E R E

59 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos Raylegh dstbuto y g Most lkely value Most Lkely Value happes whe: dp / d = 0 0 dp 0 ep ep 4 d p ep ep p 59

60 Some Useful Pobablty Dstbutos Raylegh dstbuto Chaactestc fucto / R v e 0 e 0 whee F / F, ; v e v cos d / v d e s v 0 v e v F, ; a s the cofluet hyupegeometc yp fucto : ; ; k 0 k k k, k! / 0,,,... d 60

61 Some Useful Pobablty Dstbutos Raylegh dstbuto Chaactestc fucto cot. Beauleu epessed 990 as : has show that k a a F, ; a e k 0 k k! F, ; a may be 6

62 Some Useful Pobablty Dstbutos Rce dstbuto Whe thee s a domat statoay o-fadg sgal compoet peset, such as a le-of-sght LOS popagato path, the small-scale fadg evelope dstbuto s Rce. s scatteed waves dect waves ' ep[ 0t ] Aep 0 [ A y s A A cos y ]ep y t 0 t ep[ t 0 ] 6

63 Some Useful Pobablty Dstbutos Rce dstbuto By followg smla steps descbed Raylegh dstbuto, we obta: A A ep I0 fo A 0, 0 p 0 fo 0 whee A A cos I0 ep d 0 s the modfed zeoth-ode Bessel fucto. I 0 0! 63

64 Some Useful Pobablty Dstbutos Rce dstbuto The Rce dstbuto s ofte descbed tems of a paamete K whch s defed as the ato betwee the detemstc sgal powe ad the vaace of the mult-path. It s gve by K=A /σ o tems of db: A KdB 0log [db] The paamete K s kow as the Rce facto ad completely ltl specfes the Rce dstbuto. As A0, K- db, ad as the domat path deceases ampltude, the Rce dstbuto degeeates to a Raylegh dstbuto. 64

65 Some Useful Pobablty Dstbutos Rce dstbuto 65

66 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos Rce dstbuto t A t a s 0 0 ep ep t A t a 0 0 ep ep t A t 0 0 ep ep ep ' t t y A t A t ep ep ep ep ' a y 0 0 ep ep ' whee ep ep 66

67 Some Useful Pobablty Dstbutos Rce dstbuto Defe : ' ep We have : ad A a a cos a s cos ad y a s y A cos y s y Because s usually vey lage, the dvdual ampltudes a ae adom, ad 3 the phases θ have a ufom dstbuto, t ca be assumed that fom the cetal lmt theoem ad y ae both Gaussa vaables wth meas equal to zeo ad vaace: 67 y

68 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos Rce dstbuto Because ad y ae depedet adom vaables, the ot dstbuto p,y s ep, y y p p y p The dstbuto p,θ ca be wtte as a fucto of p,y : y p J p,, y y J cos s s cos / / / / 68 y y cos s / /

69 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos Rce dstbuto ep, y p s cos ep A cos ep A A cos ep ep ep A A ep ep 69

70 Some Useful Pobablty Dstbutos Some Useful Pobablty Dstbutos Rce dstbuto pobablty desty fucto pdf a dstbut o has Rce The gve by : d p p cos, 0 A A d p p othewse 0 0 cos ep ep 0 d A A 70

71 Some Useful Pobablty Dstbutos Nakagam m-dstbuto Fequetly used to chaacteze the statstcs of sgals tasmtted though mult-path fadg chaels. PDF s gve by Nakagam 960 as: p R The m m m m / m m E R e R ] E [ paamete m s the fadg fgue., defed as m the ato of momets, called 7

72 Some Useful Pobablty Dstbutos Nakagam m-dstbuto The -th momet of R s: E m / R m m / By settg m=, the PDF educes to a Raylegh PDF. 7

73 Some Useful Pobablty Dstbutos Logomal dstbuto: Let X l R, whee X s omally dstbuted wth mea m ad vaace. The PDF of R s gve by: p l m / e The logomal dstbuto s sutable fo modelg the effect of shadowg of the sgal due to lage obstuctos, such as tall buldgs, moble ado commucatos. 73

74 Some Useful Pobablty Dstbutos Multvaate Gaussa dstbuto Assume that X, =,,,, ae Gaussa adom vaables wth meas m, =,,,; vaaces σ, =,,,; ad covaaces μ,,=,,,. The ot PDF of the Gaussa adom vaables X, =,,,, s defed as p,,..., / / det M ep m M m M deotes the covaace mat wth elemets {μ }; deotes the colum vecto of the adom vaables; m deote the colum vecto of mea values m, =,,,. M - deotes the vese of M. deotes the taspose of. 74

75 Some Useful Pobablty Dstbutos Multvaate Gaussa dstbuto cot. Gve v the -dmesoal vecto wth elemets υ, =,,,, the chaactestc fucto coespodg to the - dmetoal ot PDF s: v v E e ep m v v X Mv 75

76 Some Useful Pobablty Dstbutos B-vaate o two-dmesoal Gaussa The bvaate Gaussa PDF s gve by: p, ep m m X,, E m m m M,, 0 M m m m m, M 76

77 Some Useful Pobablty Dstbutos B-vaate o two-dmesoal Gaussa ρ s a measue of the coelato betwee X ad X. Whe ρ=0,, the ot PDF p, factos to the poduct p p, whee p,=,, ae the magal PDFs. Whe the Gaussa adom vaables X ad X ae ucoelated, they ae also statstcally depedet. Ths popety does ot hold geeal fo othe dstbutos. Ths popety ca be eteded to -dmesoal Gaussa adom vaables: f ρ =0 fo, the the adom vaables X, =,,,, ae ucoelated ad, hece, statstcally depedet. 77

78 Uppe Bouds o the Tal Pobablty Chebyshev equalty Suppose X s a abtay adom vaable wth fte mea m ad fte vaace σ. Fo ay postve umbe δ: Poof: P X m d m m p d m p d p d P X m m 78

79 Uppe Bouds o the Tal Pobablty Chebyshev equalty Aothe way to vew the Chebyshev boud s wokg wth the zeo mea adom vaable Y=X-m. Defe a fucto gy as: g Y E Y g Y 0 Y g y p y dy p y dy p y dy P Y Y Uppe-boud gy by the quadatc Y/δ,.e. The tal pobablty E gy Y E Y E g Y y 79

80 Uppe Bouds o the Tal Pobablty Chebychev equalty A quadatc uppe boud o gy used obtag the tal pobablty Chebyshev boud Fo may ypactcal applcatos, the Chebyshev boud s etemely loose. 80

81 Uppe Bouds o the Tal Pobablty Cheoff boud The Chebyshev boud gve above volves the aea ude the two tals of the PDF. I some applcatos we ae teested oly the aea ude oe tal, ethe the teval δ, o the teval -, δ. I such a case, we ca obta a etemely tght uppe boud by ove-boudg the fucto gy by a epoetal havg a paamete that ca be optmzed to yeld as tght a uppe boud as possble. Cosde the tal pobablty the teval δ,. vy Y δ gye ad gy s defed as gy 0 Y δ whee v 0 s the paamete to be optmzed. 8

82 Uppe Bouds o the Tal Pobablty Cheoff boud The epected value of gy s E v Y g y P Y E e Ths boud s vald fo ay υ 0. 8

83 Uppe Bouds o the Tal Pobablty Cheoff boud The tghtest uppe boud s obtaed by selectg the value that mmzes Ee υy-δ. A ecessay codto fo a mmum s: d vy E e dv 0 d dv vy d vy E e E e dv vy E[ Y e ] v vy vy e [ E Ye E e ] 0 vy vy Ye E e 0 E Fd 83

84 Uppe Bouds o the Tal Pobablty Cheoff boud Let vˆ be the soluto, the uppe boud o the oe -sded tal pobablty s : vˆ vy ˆ e E e P Y A uppe boud o the lowe tal pobablty ca be obtaed a smla mae, wth the esult that vˆ vˆ Y e E e 0 P Y 84

85 Uppe Bouds o the Tal Pobablty Cheoff boud Eample: Cosde the Laplace PDF py=e - y /. The tue tal pobablty s: y e dy e P Y 85

86 Uppe Bouds o the Tal Pobablty Cheoff boud Eample cot. vy v E Ye v v vy E e Sce E Ye vˆ P Y v v vy E vy e 0, we obta v v fo : vˆ must be postve e e fo Chebyshev boud P Y 0 86

87 Sums of Radom Vaables ad the Cetal Lmt Theoem Sum of adom vaables Suppose that X, =,,,, ae statstcally depedet ad detcally dstbuted d adom vaables, each havg a fte mea m ad a fte vaace σ. Let Y be defed as the omalzed sum, called the sample mea: The mea of Y s E Y X Y m E X m y X 87

88 Sums of Radom Vaables ad the Cetal Lmt Sums of Radom Vaables ad the Cetal Lmt Theoem Theoem Sum of adom vaables The vaace of Y s: The vaace of Y s: m X X E m E Y m E Y σ m X E X E X E m X X E m E Y m E Y σ y y m X E X E X E σ m m m σ A estmate of a paamete ths case the mea m that satsfes the codtos that ts epected value coveges to the tue value of the paamete ad the vaace coveges to 88 the tue value of the paamete ad the vaace coveges to zeo as s sad to be a cosstet estmate.

89 Stochastc Pocesses May of adom pheomea that occu atue ae fuctos of tme. I dgtal commucatos, we ecoute stochastc pocesses : The chaactezato ad modelg of sgals geeated by fomato souces; The chaactezato of commucato chaels used to tasmt the fomato; The chaactezato of ose geeated a eceve; The desg of the optmum eceve fo pocessg the eceved adom sgal. 89

90 Stochastc Pocesses Itoducto At ay gve tme stat, the value of a stochastc pocess s a adom vaable deed by the paamete t. We deote such a pocess by Xt. I geeal, the paamete t s cotuous, wheeas X may be ethe cotuous o dscete, depedg o the chaactestcs of the souce that geeates the stochastc pocess. The ose voltage geeated by a sgle essto o a sgle fomato souce epesets a sgle ealzato of the stochastc pocess. It s called a sample fucto. 90

91 Stochastc Pocesses Itoducto cot. The set of all possble sample fuctos costtutes a esemble of sample fuctos o, equvaletly, the stochastc pocess Xt. I geeal, the umbe of sample fuctos the esemble s assumed to be etemely lage; ofte t s fte. Havg defed a stochastc pocess Xt as a esemble of sample fuctos, we may cosde the values of the pocess at ay set of tme stats t >t >t 3 > >t, whee s ay postve tege. geeal, the adom vaables X X t,,,...,, ae I t chaactezed statstcally by the ot PDF p,,...,. t t t 9

92 Stochastc Pocesses Statoay stochastc pocesses X X t t Cosde aothe set of adom vaables t t,,,...,, whee t s a abtay tme shft. These adom vaables ae chaactezed by the ot PDF p,,...,. t t t t t t The ot PDFs of the adom vaables X ad X,,,...,, t t t may o may ot be detcal. Whe they ae detcal,.e., whe,,...,,,..., t t t t t t t t t p p fo all t ad all, t s sad to be statoay the stct sese. Whe the ot PDFs ae dffeet, the stochastc pocess s o-statoay. 9

93 Stochastc Pocesses Aveages fo a stochastc pocess ae called esemble aveages. The th momet of the adom vaable X s defed as : E X p t t I geeal, the value of the th momet wll deped o the tme stat Whe the cosequece, t f the PDF of X pocess s statoay, p t t d depeds t t o p fo all Theefoe, the PDF s depede t of tme, ad, as a t t the th momet s depede t of tme. t t. t. 93

94 Stochastc Pocesses Two adom vaables: X X t,,. The coelato s measued by the ot momet: p, t E X X t p d d t t t t t t t Sce ths ot momet depeds o the tme stats t ad t, t s deoted by φt, t., φt, t s called the autocoelato fucto of the stochastc pocess. Fo a statoay stochastc pocess, the ot momet s: E X X t, t t t t t EX X EX X EX X t t t t t t ' ' Aveage powe the pocess Xt: φ0=ex t. Eve Fucto 94

95 Stochastc Pocesses Wde-sese statoay WSS A wde-sese statoay pocess has the popety that the mea value of the pocess s depedet of tme a costat ad whee the autocoelato fucto satsfes the codto that φt,t =φt -t. Wde-sese statoaty s a less stget codto tha stct-sese statoaty. If ot othewse specfed, ay subsequet dscusso whch coelato fuctos ae volved, the less stget codto wde-sese statoaty s mpled. 95

96 Stochastc Pocesses Auto-covaace fucto The auto-covaace fucto of a stochastc pocess s defed as: t, t E Xt m t Xt m t t, t m t m t Whe the pocess s statoay, the auto-covaace fucto smplfes to: t, t t t m fucto of tme dffeece Fo a Gaussa adom pocess, hghe-ode momets ca be epessed tems of fst ad secod momets. Cosequetly, a Gaussa adom pocess s completely chaactezed by ts fst two momets. 96

97 Stochastc Pocesses Aveages fo a Gaussa pocess Suppose that Xt s a Gaussa adom pocess. At tme stats t=t, =,,,, the adom vaables X t, =,,,, ae otly Gaussa wth mea values mt, =,,,, ad auto-covaaces: t, t E Xt m t,,,,...,. Xt m t If we deote the covaace mat wth elemets μt,t by M ad the vecto of mea values by m,, the ot PDF of the adom vaables X t, =,,,, s gve by: p If the Gaussa pocess s wde-sese statoay, t s also stct-sese statoay.,,..., ep m M m / / det M 97

98 Stochastc Pocesses Aveages fo ot stochastc pocesses Let Xt ad Yt deote two stochastc pocesses ad let X t Xt, =,,,, Y t Yt, =,,,m, epeset the adom vaables at tmes t >t >t 3 > >t, ad t >t >t 3 > >t m, espectvely. The two pocesses ae chaactezed statstcally ttt by the thtpdf ot PDF: p t, t,..., t, y, y,..., y t t t Th l f f X dy d t db ' ' ' m The coss-coelato fucto of Xt ad Yt, deoted by φ y t,t, s defed as the ot momet: y t, t E X t Y, t t y t p t y t d t dy The coss-covaace s: t, t t, t m t m t t y y y t 98

99 Stochastc Pocesses Aveages fo ot stochastc pocesses Whe the pocess ae otly ad dvdually statoay, we have φ y t,t =φ y t -t, ad μ y t,t = μ y t -t : E X Y E X Y E Y X y t t y ' ' ' ' t t t t The stochastc pocesses XtadYt ae sad to be statstcally depedet f ad oly f : p,,...,, y ', y ',..., y ' p,,..., p y ', y ',..., y ' t t t t t t m m fo all choces of t ad t ad fo all postve teges ad m. The pocesses ae sad to be ucoelated f t t t t t t t, t E X E Y t, t 0 y t t y 99

100 Stochastc Pocesses Comple-valued stochastc pocess A comple-valued stochastc pocess Zt s defed as: Z t X t Y t whee XtadYt ae stochastc pocesses. The ot PDF of the adom vaables Z t Zt, =,,,, s gve by the ot PDF of the compoets X t, Y t, =,,,. Thus, the PDF that chaactezes Z t, =,,,, s: p,,...,,, y, y,...,, y t t t t t t The autocoelato fucto s defed as: zz t, t E Zt Z t E X t Y t X t Y t t, t yy t, t y t, t y t, t ** 00

101 Stochastc Pocesses Aveages fo ot stochastc pocesses: Whe the pocesses Xt ad Yt ae otly ad dvdually statoay, the autocoelato fucto of Zt becomes: zz t, t zz t t zz φ ZZ τ= φ * ZZ-τ because fom **: zz t, t E Zt Z t zz EZZ t ' ' ' ' t EZ Z EZZ t zz t t t 0

102 Stochastc Pocesses Aveages fo ot stochastc pocesses: Suppose that Zt=Xt+Yt ad Wt=Ut+Vt ae two comple-valued stochastc pocesses. The coss-coelato fuctos of Zt adwt s defed as: t, t E Z W E X t Y t U t V t u t, t yv t, t yu t, t v t, t zw t t Whe Xt, Yt,Ut ad Vt ae pawse-statoay, the coss-coelato fucto become fuctos of the tme dffeece. EZ W zw t t EZ W EWZ t t t t wz ' ' ' ' 0

103 Powe Desty Spectum A sgal ca be classfed as havg ethe a fte ozeo aveage powe fte eegy o fte eegy. The fequecy cotet of a fte eegy sgal s obtaed as the Foue tasfom of the coespodg tme fucto. If the sgal s peodc, ts eegy s fte ad, cosequetly, ts Foue tasfom does ot est. The mechasm fo dealg wth peodc sgals s to epeset them a Foue sees. 03

104 Powe Desty Spectum A statoay stochastc pocess s a fte eegy sgal, ad, hece, ts Foue tasfom does ot est. The spectal chaactestc of a stochastc sgal s obtaed by computg the Foue tasfom of the autocoelato fucto. The dstbuto of powe wth fequecy s gve by the fucto: f e f d The vese Foue tasfom elatoshp s: f e f df 04

105 Powe Desty Spectum 0 f df E X 0 t Sce φ0 epesets the aveage powe of the stochastc sgal, whch s the aea ude Φf, Φf s the dstbuto of powe as a fucto of fequecy. Φ f s called the powe desty spectum of the stochastc pocess. fom defto If the stochastc pocess s eal, φτ s eal ad eve, ad, hece P.94 Φ f s eal ad eve. e e easy to pove fom defto t If the stochastc pocess s comple, φτ=φ*-τ ad Φ f s P.0 eal because: * * f * ' f f e d e ' d ' eal because: f e d f 05

106 Powe Desty Spectum Coss-powe desty spectum Fo two otly statoay stochastc pocesses Xt ad Yt, whch have a coss-coelato fucto φ y τ, the Foue tasfom s: f e f d y y Φ y f s called the coss-powe desty spectum. * * f * f f e d e d y y y e f d y y f If Xt ad Yt ae eal stochastc pocesses f f e d f y f y f * y y y 06

107 Respose of a Lea Tme-Ivaat System to a Radom Iput Sgal Cosde a lea tme-vaat system flte that s chaactezed by ts mpulse espose ht o equvaletly, by ts fequecy espose H f, whee ht ad H f ae a Foue tasfom pa. Let t be the put sgal to the system ad let yt deote the output sgal. y t h t d Suppose that t s a sample fucto of a statoay stochastc pocess Xt. Sce covoluto s a lea opeato pefomed o the put sgal t, the epected value of the tegal s equal to the tegal of the epected value. m y E Y t h E X t d m h d m H 07 0 statoay The mea value of the output pocess s a costat.

108 Respose of a Lea Tme-Ivaat System to a Radom Iput Sgal The autocoelato fucto of the output s: yy * t t E Y Y, t t h EX t X t * * h * h t t h dd dd If the put pocess s statoay, the output s also statoay: * yy h h dd 08

109 Respose of a Lea Tme-Ivaat System to a Radom Iput Sgal The powe desty spectum of the output pocess s: f yy f yy e d h h e ddd * f f H f by ymakg τ 0 =τ+α-β The powe desty spectum of the output sgal s the poduct of the powe desty spectum of the put multpled by the magtude squaed of the fequecy espose of the system. 09

110 Respose of a Lea Tme-Ivaat System to a Radom Iput Sgal Whe the autocoelato fucto φ yyτ s desed, t s usually ease to deteme the powe desty spectum Φ yy f ad the to compute the vese tasfom. f yy yy f e df f f H f e df The aveage powe the output sgal s: 0 yy f H f df Sce φ yy 0=E Y t, we have: f H f df 0 0 vald fo ay H f. 0

111 Respose of a Lea Tme-Ivaat System to a Radom Iput Sgal Suppose we let H f = fo ay abtaly small teval f f f, ad H f =0 outsde ths teval. The, we have: f f f df 0 Ths s possble f a oly f Φ f 0 fo all f. Cocluso: Φ f 0 fo all f.

112 Respose of a Lea Tme-Ivaat System to a Radom Iput Sgal Coss-coelato fucto betwee yt ad t y Fucto of t -t The * * t t E Y X h E X t X t, t t d h stochastc t pocesses t t d y t t X t ad Y t ae tl otly statoay tt. Wth t -t =τ, we have:, y h I the fequecy doma, we have: y f f H f d

113 Dscete-Tme Stochastc Sgals ad Systems Dscete-tme stochastc pocess X cosstg of a esemble of sample sequeces {} ae usually obtaed by ufomly samplg a cotuous-tme stochastc pocess. The mth momet of X s defed as: E m m X X p X dx The autocoelato sequece s: * *, k E X X k X X k p X, X The auto-covaace sequeces s: k, k * EX EX, k k dx dx k 3

114 Dscete-Tme Stochastc Sgals ad Systems Fo a statoay pocess, we have φ,k φ-k, μ,k μ-k, ad k k m whee m =EX s the mea value. A dscete-tme tme statoay pocess has fte eegy but a fte aveage powe, whch s gve as: 0 E X The powe desty spectum fo the dscete-tme pocess s obtaed by computg the Foue tasfom of φ. f e f 4

115 Dscete-Tme Stochastc Sgals ad Systems The vese tasfom elatoshp s: f f e df The powe desty spectum Φ f s peodc wth a peod f p =. I othe wods, Φ f+k=φ f fo k=0,±,±,. The peodc popety s a chaactestc of the Foue tasfom of ay dscete-tme sequece. 5

116 Dscete-Tme Stochastc Sgals ad Systems Respose of a dscete-tme, lea tme-vaat system to a statoay stochastc put sgal. The system s chaactezed the tme doma by ts ut sample espose h ad the fequecy doma by the fequecy espose H f. H f h e f The espose of the system to the statoay stochastc put sgal X s gve by the covoluto sum: y hk k k 6

117 Dscete-Tme Stochastc Sgals ad Systems Respose of a dscete-tme, lea tme-vaat system to a statoay stochastc put sgal. The mea value of the output of the system s: my E y h k E k k 0 P. 07 k m h k m H whee H0 s the zeo fequecy [dect cuet DC] ga of the system. 7

118 Dscete-Tme Stochastc Sgals ad Systems The autocoelato sequece fo the output pocess s: k E k yy k Ey y k h h E k h h k By takg the Foue tasfom of φ yy k, we obta the coespodg fequecy doma elatoshp: f f H f P. 09 yy Φ yy f, Φ f, ad H f ae peodc fuctos of fequecy wth peod f p =. 8

119 Cyclostatoay Pocesses Fo sgals that cay dgtal fomato, we ecoute stochastc pocesses wth statstcal aveages that ae peodc. Cosde a stochastc pocess of the fom: X t a g t T whee {a } s a dscete-tme sequece of adom vaables wth mea m a =Ea fo all ad autocoelato sequece φ aa k=ea* a +k /. The sgal gt s detemstc. The sequece {a } epesets the dgtal fomato sequece that s tasmtted ove the commucato chael ad /T epesets ese the ate of tasmsso ss of the fomato o symbols. s. 9

120 Cyclostatoay Pocesses The mea value s: EX t E a g t T m g t T a The mea s tme-vayg ad t s peodc wth peod T. The autocoelato fucto of Xt s:, t t EX t X t m m E aa g tt gt mt m g tt gt mt m aa 0

121 Cyclostatoay Pocesses We obseve that t kt, tkt t, t fo k=±,±,. Hece, the autocoelato fucto of Xt s also peodc wth peod T. Such a stochastc pocess s called cyclostatoay o peodcally statoay. Sce the autocoelato fucto depeds o both the vaables t ad τ, ts fequecy doma epesetato eques the use of a two-dmesoal Foue tasfom. The tme-aveage autocoelato t fucto ove a sgle peod s defed as: T, T t t dt T T

122 Cyclostatoay Pocesses Thus, we elmate the te depedece by dealg wth the aveage autocoelato fucto. The Foue tasfom of φ τ yelds the aveage powe desty spectum of the cyclostatoay stochastc pocess. Ths appoach allows us to smply chaacteze cyclostatoay pocess the fequecy doma tems of fthe powe spectum. The powe desty spectum s: f f e d