Chapter 1 Random Process

Transcription

1 Chaper Random Process.0 Probabiliies is considered an imporan background or analysis and design o communicaions sysems. Inroducion Physical phenomenon Deerminisic model : No uncerainy abou is imedependen behavior a any insan o ime. e.g. cosw Random model :The uure value is subjec o chance probabiliy e.g. cosw+θ, θis a random variable in he inerval-t,t Example: Thermal noise, Random daa sream 0/9/0

2 .. Example o Sochasic Models Channel noise and inererence Source o inormaion, such as voice 0/9/0

3 .. Relaive Frequency How o deermine he probabiliy o head appearance or a coin? Answer: Relaive requency. Speciically, by carrying ou n coin-ossing experimens, he relaive requency o head appearance is equal o N n A/n, where N n A is he number o head appearance in hese n random experimens. 0/9/0 3

4 A. Relaive Frequency Is relaive requency close o he rue probabiliy o head appearance? I could occur ha 4-ou-o-0 ossing resuls are head or a air coin! Can one guaranee ha he rue head appearance probabiliy remains unchanged i.e., imeinvarian in each experimen perormed a dieren ime insance? 0/9/0 4

5 A. Relaive Frequency Similarly, he previous quesion can be exended o In a communicaion sysem, can we esimae he noise by repeiive measuremens a consecuive bu dieren ime insance? Some assumpions on he saisical models are necessary! 0/9/0 5

6 .. Axioms o Probabiliy Deiniion o a Probabiliy Sysem S, F, P also named Probabiliy Space.Sample space S All possible oucomes sample poins o he experimen.even space F Subse o sample space, which can be probabilisically measured. A F and B F implies A B F. 3. Probabiliy measure P 0/9/0 6

7 .. Axioms o Probabiliy 3. Probabiliy measure P A probabiliy measure saisies: PS = and PEmpySe = 0 For any A in F, 0 PA. For any wo muually exclusive evens A and B, PA B = PA + PB 0/9/0 7

8 .. Axioms o Probabiliy 0/9/0 8

9 ..3 Properies rom Axioms 0/9/0 9

10 ..4 Condiional Probabiliy 0/9/0 0

11 ..5 Random Variable 0/9/0

12 ..5 Random Variable 0/9/0

13 ..5 Random Variable 0/9/0 3

14 ..5 Random Variable 0/9/0 4

15 ..5 Random Variable 0/9/0 5

16 ..6 Random Vecor 0/9/0 6

17 ..6 Random Vecor 0/9/0 7

18 ..6 Random Vecor 0/9/0 8

19 . Mahemaical Deiniion o a Random Process RP The properies o RP a. Funcion o ime. b. Random in he sense ha beore conducing an experimen, no possible o deine he waveorm. Sample space S uncion o ime,,s -T,T mapping A random variable r.v is a real-valued uncion deined on he elemens o he Sample space. 0/9/0 9

20 S,s -T T. T:The oal observaion inerval s j, s j x j. = sample uncion λ which is deerminisic x j A = k, he se{x j k } consiues a random variable RV. To simpliy he noaion, le,s = :Random process, an ensemble o ime uncions ogeher wih a probabiliy rule. Dierence beween RV and RP RV: The oucome is mapped ino a real number RP: The oucome is mapped ino a uncion o ime 0/9/0 0

21 Figure. An ensemble o sample uncions: { j,,, n} x j Noe:, n are saisically independen or any,,, n 0/9/0

22 .3 Saionary Process Saionary Process : The saisical characerizaion o a process is independen o he ime a which observaion o he process is iniiaed. Nonsaionary Process: No a saionary process unsable phenomenon Consider which is iniiaed a =,,, k denoe he RV obained a,, k For he RP o be saionary in he sric sense sricly saionary The join disribuion uncion F τ,..., k τ independen o τ x,.., x F x,...,..., k xk For all ime shi, all k, and all possible choice o,, k k.3 0/9/0

23 .3 Random Process 0/9/0 3

24 and Y are joinly sricly saionary i he join inie-dimensional disribuion o Y ' Y ' j are invarian w.r.. he origin = 0. k and Special cases o Eq..3. k =, x F x F or all and.4 F τ x. k =, = -.5 F x, x F x,, 0, x which only depends on - ime dierence 0/9/0 4

25 .3 Saionary 0/9/0 5

26 0/9/0 6

27 0/9/0 7

28 .3 Sricly Saionary Why inroducing saionariy? Wih saionariy, we can cerain ha he observaions made a dieren ime insances have he same disribuions! 0/9/0 8

29 .4 Mean, Correlaion,and Covariance Funcion Le be a sricly saionary RP The mean o is E x x d x.6 indep. o or all.7 x : he irs order pd which is independen o ime. The auocorrelaion uncion o is R 0/9/0, E R x, x x x x x 0 x x is he second order pd saionary, x, x dx dx or all and.8 dx dx 9

30 C The auocovariance uncion, E R Which is o uncion o ime dierence -. We can deermine C, i and R - are known. Noe ha:. and R - only provide a parial descripion.. I = and R, =R -, hen is wide-sense saionary saionary process. 3. The class o sricly saionary processes wih inie second-order momens is a subclass o he class o all saionary processes. 4. The irs- and second-order momens may no exis. e. g. x, x x 0/9/0.0 30

31 .4 Wide-Sense Saionary WSS 0/9/0 3

32 .4 Wide-Sense Saionary WSS 0/9/0 3

33 .4 Cyclosaionariy 0/9/0 33

34 Properies o he auocorrelaion uncion For convenience o noaion, we redeine. R E τ, or all. The mean-square value R 0 E, τ 0. R x x x x R. 3. R R τ R R /9/0 34

35 35 0 ] [ ] [ ] [ E E E 0 0 R R 0 0 R R R 0 ] [ R E 0 R R Proo o propery 3: 0 ] [ Consider τ E 0/9/0

36 The R provides he inerdependence inormaion o wo random variables obained rom a imes seconds apar 0/9/0 36

37 Example. Acosπc Θ.5, π θ π π.6 0, elsewhere π π A R E τ cos πc θ.7 0/9/0 37

38 Appendix. Fourier Transorm 0/9/0 38

39 We reer o G as he magniude specrum o he signal g, and reer o arg {G} as is phase specrum. 0/9/0 39

40 DIRAC DELTA FUNCTION Sricly speaking, he heory o he Fourier ransorm is applicable only o ime uncions ha saisy he Dirichle condiions. Such uncions include energy signals. However, i would be highly desirable o exend his heory in wo ways:. To combine he Fourier series and Fourier ransorm ino a uniied heory, so ha he Fourier series may be reaed as a special case o he Fourier ransorm.. To include power signals i.e., signals or which he average power is inie in he lis o signals o which we may apply he Fourier ransorm. 0/9/0 40

41 The Dirac dela uncion or jus dela uncion, denoed by, is deined as having zero ampliude everywhere excep a 0, where i is ininiely large in such a way ha i conains uni area under is curve; ha is and 0, 0 A.3 d A.4 g 0 d g 0 A.5 0/9/0 g d g A.6 4

42 0/9/0 4

43 0/9/0 43

44 Example.3 Random Binary Wave / Pulse. The pulses are represened by ±A vols mean=0.. The irs complee pulse sars a d. T d d, T 0, 3. During n T nt, he presence o +A or A is random. 0 d T elsewhere d 4.When k i T, T k and T i are no in he same pulse inerval, hence, k and i are independen. E E E 0 k i k i 0/9/0 44

45 Figure.6 Sample uncion o random binary wave. 0/9/0 45

46 46 Soluion For k and i occur in he same pulse inerval,, 0, 0, i k i k k i T T - - T- i d i k d i.e., i T T A d T A d E T A E i k i k - T- d d d - T- T i k i k d d i k i k i k d A elsewhere 0, - -, 0 0 0/9/0

47 Similar reason or any oher value o k τ A, τ T R T, where τ k- 0, τ T i Wha is he Fourier Transorm o R? S x A T sin c T Reerence : A.Papoulis, Probabiliy, Random Variables and Sochasic Processes, 0/9/0 Mc Graw-Hill Inc. 47

48 Soluion 0/9/0 48

49 0/9/0 49

50 0/9/0 50

51 0/9/0 5

52 0/9/0 5

53 0/9/0 53

54 0/9/0 54

55 Cross-correlaion Funcion o and Y and R R Y Y R Y Noe and are no general even uncions.,u,u, u E The correlaion marix is Y u.9 u.0 E Y R Y, u R, u R, u Y R, u RY, u RY, u R, u, R, u are auocorrelaion uncions x y I and Y are joinly saionary R RY R RY R. Y 0/9/0 where τ u 55

56 56. ] [ ] [ ] [, Le τ R Y E Y E Y E τ R μ τ Y Y Proo o : ] E[ τ Y τ R Y τ R R Y Y 0/9/0

57 Example.4 Quadraure-Modulaed Process where is a saionary process and is uniormly disribued over [0, ]. EE R E τ cos sin = E τ E π π π τ c c c E τ πc πc πcτ d cos sin 0 cosπ sinπ c c, R τ Esin4πc πcτ Esin π cτ. 3 = 0 R τsin πcτ A 0, sin π c τ 0, R 0, 0/9/0 and are orhogonal a some ixed. 57

58 .5 Ergodic Processes Ensemble averages o are averages across he process. in sample space Long-erm averages ime averages are averages along he process in ime domain DC value o random variable μ x T T I is saionary, E T T? x d T T μ T μ T Ex x 0/9/0 T T T μ d d

59 xt represens an unbiased esimae o The process is ergodic in he mean, i a. b. lim T lim var T μ T μ T 0 The ime-averaged auocorrelaion uncion R x τ,t Rx,T is a random variable. I he ollowing condiions hold, is ergodic in he auocorrelaion uncions lim R, T R τ 0/9/0 T lim var T x x T x T T μ x R,T 0 x τ x d 6. 59

60 Linear Time-Invarian Sysems sable a. The principle o superposiion holds b. The impulse response is deined as he response o he sysem wih zero iniial condiion o a uni impulse or δ applied o he inpu o he sysem c. I he sysem is ime invarian, hen he impulse response uncion is he same no maer when he uni impulse is applied d. The sysem can be characerized by he impulse response h e. The Fourier ransorm o h is denoed as H 0/9/0 60

61 .6 Transmission o a random Process Through a Linear Time-Invarian Filer Sysem I E[] is inie and sysem is sable Y h τ τ dτ Y =H - where h is he impulse response o he sysem I is saionary, μy μ H0 :Sysem DC response. 0/9/0 μ Y E Y E h τ h τ h τ - E μ x τ τ τ dτ dτ dτ h τ dτ μ H0,

62 Consider auocorrelaion uncion o Y:, E Y Y R Y I E h τ E[ ] τ dτ h τ is inie and he sysem is sable, μ τ dτ 30. I R Y,μ dτh τ dτ h τ R τ, τ R R Y τ Saionary inpu, Saionary oupu WSS τ, μ τ R μ τ τ h τ h τ R τ Funcion o ime dierence τ τ τ dτ dτ saionary RY 0 E Y h τ h τ R τ τ dτ dτ. 0/9/0 6 33

63 .7 Power Specral Densiy PSD Consider he Fourier ransorm o g, G g g exp G exp jπ jπ d d Le H denoe he requency response, Recall.30 h τ H exp j πτ d E Y H exp j τ d h τ R τ τ dτ dτ le d H dτ h τ R τ τ exp j τ dτ d H dτ hτ exp j τ R exp j τ d * H - E Y Y u E h d h d complex conjugae response o he iler /9/0 63

64 I Δ E Y H : he magniude response Deine: Power Specral Densiy Fourier Transorm o Rτ S E Y Recall E Y h τ h τ R τ τ dτ dτ Le H be he magniude response o an ideal narrowband iler, c D H 0, c D D : Filer Bandwidh c and - R - - d H - R exp πτ d H is S coninuous, d Δ S S in W/Hz E Y S c c τexp j dτ /9/0 64

65 Properies o The PSD S R τ R S τexp j d τexp j d Einsein-Wiener-Khinchine relaions: S R τ S is more useul han τ! R 0/9/0 65

66 66. d S S e.. S τ u du u j u R dτ j τ R d. S. S S E Y c.. d S b. E. d τ R a. S p 48 : wih a pd be associaed The PSD can 47, exp exp 46 or all 0 0 Δ is saionary, I R R 0/9/0

67 Example.5 Sinusoidal Wave wih Random Phase Acos c, ~ U, R A cos c S R exp j d A exp j c exp j c exp j d 4 A c c 4 Appendix, exp j d c Example. c 0/9/0 The specral Analyzer can no deeced he phase, so he phase inormaion is los. 67

68 68 Example.6 Random Binary Wave Example.3 Deine he energy specral densiy o a pulse as.50 sinc A exp 0 0 i i,, T T d j T A S T T T A R m m A A T T.5 S.5 sinc T T A T ε g g 0/9/0

69 69 Example.7 Mixing o a Random Process wih a Sinusoidal Process.55 4 exp exp 4 exp.54 cos cos4 cos cos cos.53 0, ~, cos c c c c Y Y c c c c c c c Y c S S d j j R d j R S R E R E τ E Y τ E Y R U Y We shi he o he righ by, shi i o he le by, add hem and divide by 4. S c c 0/9/0

70 Relaion Among The PSD o The Inpu and Oupu Random Processes S h Y S Y Recall.3 R h h R d d Y S h h R exp j d d d Y Le, or S Y 0 0 H S.3 h h R exp j exp j exp j dτ dτ dτ S H H * /9/0 70

71 Relaion Among The PSD and The Magniude Specrum o a Sample Funcion Le x be a sample uncion o a saionary and ergodic Process. In general, he condiion or Fourier ransormable is This condiion can never be saisied by any saionary x o ininie duraion. T We may wrie, T x exp j d.60 Ergodic x d T Take ime average.59 T R lim x x d T T T I x is a power signal inie average power T x x d, T T T T Time-averaged auocorrelaion periodogram uncion.6.6 0/9/0 For ixed, i is a r.v. rom one sample uncion o anoher 7

72 Take inverse Fourier Transorm o righ side o.6 T x x d, T exp j T d.63 T T From.6,.63,we have R lim, T exp j d.64 T T Noe ha or any given x periodogram does no converge as Since x is ergodic Recall.43 R S exp j d 0/9/0.67 is used o esimae he PSD o x ER R lim E T exp j d T T R lim E T exp j d T T S lim E, T T T T lim E x exp j d T T T T

73 73 Cross-Specral Densiies.7. exp exp may no be real. and.69 exp.68 exp S S S τ R τ R d πτ j S τ R d πτ j S τ R S S d j R S d j R S Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y 0/9/0

74 Example.8 and Y are uncorrelaed and zero mean saionary processes. Consider Z Y SZ S SY.75 Example.9 and Y are joinly saionary. R, u E V Z u VZ h h R, u d d Y R h h R d d.77 VZ E h d h Y u d Le τ u u Y 0/9/0 F S H H S Y VZ 74

75 .8 Gaussian Process Deine : Y as a linear uncional o 泛函數 Y T 0 g d The process is a Gaussian process i every linear uncional o is a Gaussian random variable Y e.g g: y Normalized Y Y exp y g: some uncion and he inegral exiss y Y Y exp y, as N0, /9/0 Fig..3 Normalized Gaussian disribuion 75

76 Cenral Limi Theorem Le i, i =,,3,.N be a saisically independen R.V. and b have mean μ and variance σ. Since hey are independenly and idenically disribued i.i.d. Normalized i Y i Hence, Deine E Y i Y Var V N i i 0,. N N i Y i i,,..., N The Cenral Limi Theorem The probabiliy disribuion o V N approaches N0, as N approaches ininiy. Noe: For some random variables, he approximaion is poor even N is quie large. 0/9/0 76

77 Properies o A Gaussian Process. Y h d Deine Z g h τ d d where g T T 0 0 Y T 0 T 0 g h d τ d Y g dτ g h d By deiniion Z is a Gaussian random variable.8 T Y h d, 0 is Gaussian 0/9/0 Y h Gaussian in Gaussian ou 0 Y 77

78 . I is Gaussisan Then,, 3,., n are joinly Gaussian. Le i,,...,n i E i and he se o covariance uncions be where i C k, i E k, k k,i,,...,n i,,..., n T T Then, x μ Σ x μ.85,..., x..., x exp n n n D T where μ mean vecor,,...., n n Σ covariance marix { C, } k i k, i Ddeerminan o covariance marix Σ 0/9/0 78

79 Supplemenal Maerial 0/9/0 79

80 0/9/0 80

81 0/9/0 8

82 0/9/0 8

83 0/9/0 83

84 3. I a Gaussian process is saionary hen i is sricly saionary. This ollows rom Propery 4. I,,.., n are uncorrelaed as E [ k i ] Then hey are independen Proo : uncorrelaed Σ.85 0/9/0 Σ 0 0, where i n E[ E ], i is also a diagonal marix, Δ=deerminan o Σ i i,,n. T, x μ Σ x μ,, x..., x exp n n n D x x Independen i i i n k where i i and x exp i i i i 0 x i i i 84

85 .9 Noise Sho noise Thermal noise E V TN 4kTRD vols E ITN E V TN 4kT D 4kTGD amps R R k: Bolzmann s consan =.38 x 0-3 joules/k, T is he 0/9/0 absolue emperaure in degree Kelvin. 85

86 Whie noise S N T W e R :equivalen noise emperaure o he receiver W N0.95 S R exp j d W δ 0/9/0 kt e N w δ, Table A6.3 N 0 86

87 87 Example.0 Ideal Low-Pass Filered Whie Noise sinc.97 exp B B N d j N R B B -B N S B B N N 0/9/0

88 Example. Correlaion o Whie Noise wih a Sinusoidal Wave Whie noise w w' T T d 0 T w ' w cos 0 c d T The variance o w' T is w' T T T T E w w cos 0 c cos c T d d 0 T T R 0 0 W, cos c cos c d d T From.95 T T E w 0 0 cos c w cos c d d T N T cos c, c T T T cos c cos c d d 0/9/0 T 0 0 cos 0 c d k T, is ineger.98 N N k

89 .0 Narrowband Noise NBN Two represenaions a. in-phase and quadraure componens cos c,sin c b.envelope and phase. In-phase and quadraure represenaion n n cos n sin n I I and n Q c are low - passsignals Q sample uncion c.00 0/9/0 T d 0 89

90 Imporan Properies.n I and n Q have zero mean..i n is Gaussian hen n I and n Q are joinly Gaussian. 3.I n is saionary hen n I and n Q are joinly saionary. 4. S N I S N Q S N c 0 S N c, -B B oherwise.0 5. n I and n Q have he same variance. 6.Cross-specral densiy is purely imaginary. problem.8 S S N I N Q N Q N I j SN c SN c, -B B.0 0 oherwise 7.I n is Gaussian, is PSD is symmeric abou c, hen n I and n Q are saisically independen. problem.9 0/9/0 N 0 90

91 Supplemen Proo o e.q..0. From igure.9a, we see ha 0/9/0 9

92 0/9/0 9

93 Supplemen. From igure.9a, we see ha 0/9/0 93

94 0/9/0 94

95 Example. Ideal Band-Pass Filered Whie Noise cb N cb 0 N0 RN exp j d exp j d cb cb N B sinc B exp j exp j 0 N B sinc B cos 0 Compare wih.97 a acor o, R R N B sinc B. N I 0/9/0 N Q 0 Low-Pass ilered R N B sinc B N 0 c c c.03 95

96 96. Represenaion in Terms o Envelope and Phase Componens Le N I and N Q be R.V.s obained a some ixed ime rom n I and n Q. N I and N Q are independen Gaussian wih zero mean and variance..07 an Phase.06 Envelop e.05 cos n n n n r r n I Q Q I c 0/9/0

97 97.08 exp,, Q I Q I N N n n n n Q I.. sin.0 cos Le.09 exp,, r dr dψ dn dn ψ r n ψ r n dn dn n n dn dn n n Q I Q I Q I Q I Q I Q I N N Q I 0/9/0

98 Subsiuing.0 -. ino.09 n, n dn dn r, rdrd N, N I Q I Q R, Ψ I Q r r exp drd r r R, Ψ r, exp.3 0 0, Ψ.4 elsewhere 0 R R r is Rayleigh disribuion. r r exp, r 0 r 0 elsewhere r For convenience, le ν. V ν R r Normalized σ V.5 exp, 0 ν.8 0 elsewhere 0/9/0 98

99 Figure. Normalized Rayleigh disribuion. 0/9/0 99

100 .3 Sine Wave Plus Narrowband Noise x Acos n x n cos n n A n I I I c c Q sin.9 I n is Gaussian wih zero mean and variance. ni ' and nq are Gaussian and saisically independen..the mean o ni ' is A and ha o nq is zero. 3.The variance o ni ' and nq is. ni A nq N, N n I, nq exp I Q I Q Le r n n.3 - nq an.4 n I Follow a similar procedure, we have r r A Ar cos R, Ψ r, exp R and are dependen. 0/9/0 c 00

101 R r 0 r R, Ψ r, d exp r A Ar exp 0 cos d.6 The modiied Bessel uncion o he irs kind o zero order is deined as Appendix 3 I0 x exp xcos d 0.7 Ar r r A Ar Le x, exp Rr I 0 σ.8 I is called Rician disribuion. I 0/9/0 A 0, I00 d 0 i is Rayleigh disribuion. =, 0

102 Normalized V v R r v v exp r, A a a I 0 av.3.3 0/9/0 Figure.3 Normalized Rician disribuion. 0