Communications II Lecture 2: Probability and Random Processes

Transcription

1 Commuicatio II Lecture : robabilit ad Radom rocee roeor Ki K. Leug ad Computig Departmet Imperial College Lodo Copright reerved

2 Sigiicace o probabilit theor robabilit i the core mathematical tool or commuicatio ad iormatio theor. Radome pla a importat role i commuicatio. Meage are radom. No radome o iormatio. Noie i radom. Ad ma more dela phae adig...

3 robabilit Sample Space Ω : Set o all radom eperimet outcome Ω { ω : ω i a outcome } ample For toig a coi Ω { H T } Roll a die Ω { 6 } vet Ω Roll a die two time: Ω { HH HT TH TT } vet { HH TT } A collectio o evet F alo called -ield. Obvioul F Ω A probabilit meaure o F i a uctio : F atiig the probabilit aiom: Ω F For evet A B belogig to F i A B AUB A + B 3

4 4 Coditioal robabilit Coider two evet A ad B ot ecearil idepedet o each other Deie A B robabilit o A give B Carr out N idepedet trial eperimet ad cout NA Number o trial with A outcome NAB Number o trial with both A ad B outcome. B deiitio Similarl a B B A B A A A B B A B A N A N A N A N B A N N B A N B N B A N N A N Statitical idepedece betwee A ad B B A B A A B A

5 5 Radom Variable RV For a probabilit pace Ω F a radom variable RV i a mappig rom Ω to a real umber. That i : Ω R A RV ω i a variable whoe value deped o the outcome ω o a radom eperimet with the deied probabilit meaure. Obvioul. ω Ω Coider a gamble i La Vega: Wi 3/8 Draw /4 Lo 3/8 -$5 $5 ω : 5 : : Ω + L L D D W W L D W L D W ω ω ω ω ω ω ω ω ω ω ω ω ω

6 6 robabilit Ditributio Fuctio DF ad robabilit Deit Fuctio pd DF alo ow a Cumulative Ditributio Fuctio CDF decreaig - o i Sice : pd : DF < d df b a d df F d a F b F b a d F d F F

7 Iterpretatio o pd Δ I i uicietl mall + Δ < + Δ d Δ Area Δ pectatio operator epected value or average: Variace: μ d μ μ d μ 7

8 Momet o a RV r r 3 Momet: μ or r... Average mea o : μ or m Cetral momet cetered o the mea Commet r μ r ' m or r 3... μ ' m Variace : Stadard deviatio : m μ ' m μ ' which give a meaure o diperio o about it mea 8

9 9 oio Ditributio Dicrete RV Coider a biomial ditributio with ver large umber o trial ad mall p: α α α α α α α α α α + e B C B p p α!...! lim : above the i Set... iite i where / Let p p C B ercie: Veri α α α α α Ω

10 poetial Ditributio λ λ λe or ercie:veri λ λ

11 Normal Gauia Ditributio A ver importat oe! d e F e m m or π π < < ercie:veri m m

12 Raleigh ad Ricia Ditributio R + Deie a RV where ad are idepedet Gauia with zero mea ad variace R ha Raleigh ditributio: R R r r r + A / Ar r e I r r / e r I ha ozero mea A R ha Ricia ditributio: where I π coθ e dϑ i the modiied zero-order π Beel uctio o the irt id r

13 3 Joit RV DF joit the : pd joit the : : ' F dudv v u F RV ropertie o joit pd: idepedet are 4 3 d b d a dudv v u F F

14 4 Coditioal DF ad pd A F d df dv v dv v du u dudv v u F Δ Δ + < Δ Δ + Δ < + Δ < + Δ + Δ RV : Deie a the coditioal DF or give B coditioal probabilit F The coditioal pd

15 Coditioal Ditributio Fuctio Cot d Coditioa l ditributi o uctio : Coditioa l average : φ φ d Whe φ Coditioa l Coditioa l mea : variace m : m d d 5

16 Coditioal ormal ditributio Coditioa l ditributi o uctio : The margial deit : d Coditioa l deit uctio : π joitl ormall ditribute d with m ρ ep{ π π ρ ρ ep{ m ep{ } ρ ρ + } ρ } Note that thi coditioal pd i alo a ormal deit uctio with mea ρ ad variace ρ. The eect o the coditio havig o i to chage the mea o to ρ ad to reduce the variace b. ρ 6

17 7 Joit Ditributio Fuctio o RV F F F F F : '... F RV The joit DF o RV The joit pd o RV Idepedet RV Ucorrelated RV j i j i j i j i

18 Covariace ad Correlatio Coeiciet RV : Covariace o ad : cov m m Correlatio Coeiciet: ρ cov m m 8

19 9 ropertie o Correlatio Coeiciet. Similarl variace. o deiitio to due i tep lat The / Clearl / / Let V m m U U U V m U m V m U A epectatio o a o-egative RV i o-egative we have ρ ropert : ρ roo: ad + V U V U Thee two iequalitie impl: become above the ad Sice UV V U b V UV U a V UV U ρ ρ ρ + + ρ ρ ρ

20 ropertie o Correlatio Coeiciet ρ ropert : ad are liearl related i ad ol i ± ρ roo: I ad are liearl related let a + b where a b R ρ m From * ρ ρ ± a m m m a ± m m a + b am a * a + bam b b ± b de o m a am ± a

21 ropertie o Correlatio Coeiciet ρ ropert : ad are liearl related i ad ol i ± ρ roo Cot d: Now aume ρ Let ρ U m m / m V UV m / Now coider U V U V U V Thereore ad the U U V m UV + V otherwie U-V with prob. m > i.e. are liearl + related Similarl whe ρ coider U + V ad obtai the ame reult

22 ropertie o Correlatio Coeiciet ρ ropert 3: I ad are idepedet ρ leae tr to prove ropert 3 at home Cautio: The covere o thi propert i ot true i geeral. That i i ad are ot ecearil idepedet! ρ

23 A ample o ucorrelated but depedet R.V. Let θ be uiorml ditributed i π θ or π π Deie R.V. ad a co θ iθ Clearl ad are ot idepedet. I particular or a give θ ad are depedet. θ ρ Locu o ad orthogoal I ad are idepedet we hould ee poible ample poit o aume all poible value o ad i a uit quare. But ad are ucorrelated a ρ π π! m m coθ iθdθ 3

24 Idepedet implie Ucorrelated Ucorrelated doe ot impl Idepedece For ormal RV joitl Gauia Ucorrelated implie Idepedet thi i the ol eceptioal cae! Orthogoal RV i Covariace matri o RV j C cij i μ i j μ j 4

25 Importat robabilit Iequalitie ad Limit Theorem Marov Iequalit: a or o - egative a a d + d a a a a d d a a Marov iequalit i ot tight becaue ol the mea o the RV i utilized Chebhev Iequalit: Let The m ad Sice m m a a a a a i.e. a m a i the variace o b Marov iequalit a m Chebhev iequalit a a Chebhev iequalit i tighter tha Marov iequalit a the ormer ue the variace 5

26 Arithmetic Mea o Sum o RV Deie where That i i W i ' i are idepedet idetical ditributed i.i.d. R.V. W i the ample mea o i ' Let each have a mea ad variace: ad repectivel Appl W i W W ercie :Show W Chebhev i W a i iequalit W Importace: Arithmetic mea o the um o i.i.d. RV will approach it epected value a icreae : a a 6

27 Law o Large Number Wea Law o Large Number For a ε > ad iite lim W ε W Or lim W ε ε < ε a Strog Law o Large Number Or lim W lim W almot urel or with prob. 7

28 Cetral Limit Theorem Deie S i i where i ' are i.i.d. R.V. with μ < Deie a ew RV Cetral Limit Theorem : lim R R < S μ π e / d Importace: The hited ad caled um o a ver large umber o i.i.d. RV ha a ormal ditributio 8

29 9 Setch o roo: Cetral Limit Theorem { }... : epaio Talor the Obtai : or LT The 3 3!! * * 3 / Φ Φ μ μ μ μ μ μ μ μ e e e e e e R i R i R R i i i i i i S R μ μ where * ξ ξ Z that Note * Z e ξ μ μ μ + + e Z * lim large or / to relative eglected be ca term The + μ ξ

30 3 Setch o roo: Cetral Limit Theorem Cot d { } o LT lim } { lim lim lim : have ow We / * * N e e e e R R + Φ + Φ μ μ μ i i S R μ μ

31 What i a Radom roce? A radom proce i a time-varig uctio that aig the outcome o a radom eperimet to each time itat: t; ω. For a ied ample path ω: a radom proce i a time varig uctio e.g. a igal. For ied t: a radom proce i a radom variable I ω ca all poible outcome o the uderlig radom eperimet we hall get a eemble o igal. 3

32 For ied t: the radom proce become a radom variable with epectatio value Autocorrelatio uctio R Auto-covariace uctio μ t t; ω p ; t d t t t; ω t; ω p ; t t dd C t t t; ω μ t t; ω μ t 3

33 Two Radom rocee Cro correlatio: R g Cro covariace: C g t t t; ω g t ; ω t t t; ω μ t g t; ω μg t Ucorrelated radom procee: C g t t t; ω μ t g t; ω μg t 33

34 Statioar Radom rocee A radom proce i tatioar i the trict ee i the joit pd are ivariat to a tralatio o the time ai t p + t t t p t + t t + t t t A radom proce i tatioar i the wide ee i It epectatio value doe ot deped o t ad It autocorrelatio uctio i time-tralatio ivariat R t t t; ω t; ω t + t; ω t + t; ω or a t 34

35 I geeral emble autocorrelatio uctio i a - dimeioal matri. The autocorrelatio uctio Rt t o a wide-ee tatioar radom proce deped ol o the dierece time t t. Thi i eail ee rom the lat equatio o the previou lide. A tatioar radom proce i otherwie ow a a homogeeou proce. 35

36 ercie: Show a iuoid t A co ω t + Θ c with radom phae Θ uiorml ditributed o π i wide-ee tatioar. 36

37 Temporal Statitic o a Radom roce Temporal average: μ ω lim T T T / T / t; ω dt Note: The temporal average above i a uctio o the outcome o which deped i.e. the average i a radom variable. Temporal autocorrelatio uctio o the radom proce: R t T / lim ; ω + T t; ω t t; ω T T / Thi i aother radom variable. dt 37

38 rgodicit A radom proce i ergodic with repect to the mea i iti wide-ee tatioar ad it temporal average i equal to the eemble average. A radom proce i ergodic with repect to the autocorrelatio uctio i it i wide-ee tatioar ad it temporal autocorrelatio uctio i equal to it eemble autocorrelatio uctio: R t lim T T R t t; ω T T ; ω / / t + t t; ω ; ω t + t ; ω dt A radom proce i ergodic whe it i ergodic with repect to the mea ad with repect to it autocorrelatio uctio. 38

39 ercie: Show a iuoid t Acoωct + Θ with radom phae Θ uiorml ditributed o π i ergodic. 39

40 Importat implicatio o ergodicit I a eemble o igal i ergodic we ca calculate it mea ad autocorrelatio uctio b impl calculatig temporal average over a igal o the eemble we happe to have. Statitical propertie o all verio o a igal oe ma ecoutercomputable rom a igle verio o the igal b uig all it ample. 4

41 4

42 Relatiohip o tatitical meaure with meaurable egieerig quatitie I mot cae Gauia oie i commuicatio i a ergodic radom proce. Note that the mea value t locate the ceter o gravit o the area uder the probabilit deit uctio o it value. DC compoet: t t Average power: t t where <.> deote time average. Notice that or a zero-mea proce the variace i equivalet to the average power i.e. t. Thi could be meaured i the lab uig a power meter. 4

43 Autocorrelatio ad ower Spectral Deit The requec cotet o a proce deped o how rapidl the amplitude chage a a uctio o time. Thi ca be meaured b correlatig the amplitude at time t ad t. Autocorrelatio o a real radom proce: R t t t t For a tatioar proce the autocorrelatio deped ol o the time dierece o τ t t + τ R Average power o a waveorm mea quare. Hece t R 43

44 ower Spectral Deit ower pectral deit SD i a uctio that meaure the ditributio o power o a radom igal with requec. SD i ol deied or tatioar igal igal that are homogeeou with repect to the mea ad the autocorrelatio uctio. S T lim T T where width T. T i the Fourier traorm o igal t i a widow o 44

45 Wieer-Khichie Theorem The power pectral deit o a radom proce i equal to the Fourier traorm o it autocorrelatio: S R τ e j π τ dτ See the boo b Lathi or a proo. The the average power o a radom proce ca be oud b itegratig the SD over all requecie: R S d 45