ECS /1 Part IV.2 Dr.Prapun

Transcription

1 Sirindhorn Interntionl Institute of Technology Thmmst University School of Informtion, Computer nd Communiction Technology ECS35 6/ Prt IV. Dr.Prpun.4 Fmilies of Continuous Rndom Vribles Theorem.3 sttes tht ny nonnegtive function f() whose integrl over the intervl (, + ) equls cn be regrded s probbility density function of rndom vrible. In rel-world pplictions, however, specil mthemticl forms nturlly show up. In this section, we introduce couple fmilies of continuous rndom vribles tht frequently pper in prcticl pplictions. The probbility densities of the members of ech fmily ll hve the sme mthemticl form but differ only in one or more prmeters..4. (Continuous) Uniform Distribution Definition.4. The (continuous) uniform distribution on n intervl [, b], is denoted by uniform([, b]) or U([, b]) or simply U(, b). Epressions tht re synonymous with X is uniform rndom vrible re () X is uniformly distributed, (b) X hs uniform distribution, (c) nd X hs uniform density. To specify the support (rnge) of X, we my lso ppend on/over the intervl (, b). 47

2 This fmily is chrcterized by pdf of the form {, <, > b f X () = b, b The constnts nd b re referred to s the prmeters of the uniform distribution..43. Importnt Interprettion: A continuous uniform rndom vrible X on the intervl [, b] is just s likely to be ner ny vlue in [, b] s ny other vlue..44. In MATLAB, () use X = +(b-)*rnd or X = rndom( Uniform,,b) to generte X U(, b), (b) use pdf( Uniform,,,b) nd cdf( Uniform,,,b) to evlute the pdf nd cdf t, respectively., <, Eercise.45. Show tht F X () = b, b,, > b. 84 Probbility theory, rndom vribles nd rndom processes f () F () b Fig. 3.5 The pdf nd cdf for the uniform rndom vrible. Figure 3: The pdf nd cdf for the uniform rndom vrible. [6, Fig. 3.5] f () F () πσ b b μ 48 μ Fig. 3.6 The pdf nd cdf of Gussin rndom vrible.

3 Emple.46 (F). Suppose X is uniformly distributed on the intervl (, ). (X U(, ).) () Plot the pdf f X () of X. (b) Plot the cdf F X () of X..47. The uniform distribution provides probbility model for selecting point t rndom from the intervl [, b]. Use with cution to model quntity tht is known to vry rndomly between nd b but bout which little else is known. Emple.48. [9, E. 4. p. 4-4] In coherent rdio communictions, the phse difference between the trnsmitter nd the receiver, denoted by Θ, is modeled s hving uniform density on [ π, π]. () P [Θ ] = (b) P [ Θ π ] = 3 4 Eercise.49. Show tht when X U([, b]), EX = +b Vr X = (b ), nd E [ X ] ( = 3 b + b + )., 49

4 .4. Gussin Distribution.5. This is the most widely used model for the distribution of rndom vrible. When you hve mny independent rndom vribles, fundmentl result clled the centrl limit theorem (CLT) (informlly) sys tht the sum (or the verge) of them cn often be pproimted by norml distribution The Gussin rndom vrible nd process Signl mplitude (V) () Figure 4: Electricl ctivity of skeletl muscle: () A smple skeletl muscle (emg) signl, nd (b) its histogrm nd pdf fits. [6, Fig. 3.4] t (s) (b) Histogrm Gussin fit Lplcin fit f () (/V) (V) Fig. 3.4 () A smple skeletl muscle (emg) signl, nd (b) its histogrm nd pdf fits. Definition.5. Gussin rndom vribles: [ [ = f ()d] = K e d = K e d frequently = y= in prctice. = K = y= e y dy ] Often clled norml rndom vribles becuse they occur so e ( +y ) ddy. (3.3) 5

5 The Gussin distribution is denoted by N ( m, σ ). It hs two prmeters: m R nd σ >. 84 Probbility theory, rndom vribles nd rndom processes Cution: The second rgument in N ( m, σ ) is σ (not f () F () σ). Severl references use µ insted of m. b This fmily is chrcterized by pdf of the form f X () = e ( m σ ). πσ b b Fig. 3.5 The pdf nd cdf for the uniform rndom vrible. πσ f () F () Fig. 3.6 The pdf nd cdf of Gussin rndom vrible. Figure 5: The pdf nd cdf of N (µ, σ ). [6, Fig. 3.6] μ μ Gussin (or norml) rndom vrible is described by the following pdf: This is continuous rndom vrible tht In Ecel, use NORMDIST(,m,σ,FALSE). In MATLAB, use normpdf(,m,σ) or pdf( Norml,,m,σ). { } f () = ep ( μ) πσ σ, (3.6) Figure 5 nd Figure 7 disply the fmous bell-shped grphs of the Gussin pdf. This curves re lso clled where μ nd σ re two prmeters whose mening is described lter. It is usully denoted s N (μ, the σ ). Figure norml 3.6 shows curves. sketches of the pdf nd cdf of Gussin rndom vrible. The Gussin rndom vrible is the most importnt nd frequently encountered rndom vrible in communictions. This is becuse therml noise, which is the mjor source of noise in communiction systems, hs Gussin distribution. Gussin noise nd the + m to generte X N (m, σ ). Gussin pdf re discussed in more depth t the end of this chpter. The problems eplore other pdf models. Some of these rise when rndom vrible is pssed through nonlinerity. How to determine the pdf of the rndom vrible in this cse is discussed net. In MATLAB, use X = rndom( Norml,m,σ) or X = σ*rndn F X () hs no closed-form epression. However, see.58. In MATLAB, use normcdf(,m,σ) or cdf( Norml,,m,σ). In Ecel, use NORMDIST(,m,σ,TRUE). Functions of rndom vrible A function of rndom vrible y = g() is itself rndom vrible. From the definition, the cdf of y cn be written s.5. EX = m nd Vr X = σ. F y (y) = P(ω : g((ω)) y). (3.7) 5

6 3.5 The Gussin rndom vrible nd process σ = σ = σ = 5 Figure 6: Plots of the zeromen Gussin pdf for different vlues of stndrd devition, σ X. [6, Fig. 3.5] f () Fig. 3.5 Plots of the zero-men Gussin pdf for different vlues of stndrd devition, σ..53. Importnt probbilities: P [ X µ < σ] =.687; Tble 3. Influence of σ on differentquntities j m j fx fx e dt e ω ω F = Rnge (±kσp ) [ X µ > = ωσ. σ] = k.373; k = k = 3 k = 4 3) Fourier trnsform: ( ) ( ) P [ X µ > σ] =.455; α π 4) P(m Note tht kσ < e md + = kσ ) Error probbility α P [ X µ < σ] = Distnce from the men m ) P[ X > ] = QThese σ ; vlues [ ] m m P X < = re Q illustrted = Q σ σ in. Figure 7. numbers gin in Emple.59. P of the pdf X μ < σ re ignorble. =.687, P Indeed when X μ > σ communiction =.373 systems re considered lter it is the presence P ofxthese μ > tils σ tht =.455, results Pin bit X errors. μ < The σ = probbilities.9545 re on the order of 3, very smll, but still significnt in terms of system performnce. It is of interest to see how fr, in terms of σ, one must be from the men vlue to hve the different levels of f X error probbilities. As shll ( ) ( ) f X be seen in lter chpters this trnsltes to the required SNR to chieve specified bit error probbility. This is lso shown in Tble 3.. Hving considered the single (or univrite) Gussin rndom 95% 68% vrible, we turn our ttention to the cse of two jointly Gussin rndom vribles (or the bivrite cse). Agin they re described by their joint pdf which, in generl, is n eponentil whose eponent μ σ μ μ + σ μ σ is qudrtic in the two vribles, i.e., f,y (, y) = Ke ( +b+cy+dy+ey μ μ + σ +f ), where the constnts K,, b, c, d, e, nd f re Q z = chosen to stisfy the bsic properties of vlid joint pdf, e d corresponds to P[ X > z] where X ~ N (,) ; nmely being lwys nonnegtive ( ), hving unit volume, nd lso tht the mrginl pdfs, f () = z π tht is Q( z ) is the f,y(, probbility y)dy ndof f y the (y) til = of f,y(, N (,y)d, ). re vlid. Written in stndrd form the joint pdf is N, 6) Q-function: ( ) the norml distribution. ( ) Q =. ) Q is decresing function with ( ) b) Q( z) = Q( z) c) Q ( Q( z) ) = z π π sin θ d) Q( ) = e dθ. ( ) π 4 sin θ Q = e dθ π..5 We will see these Figure 7: Probbility density function of X N (µ, σ ). The purple res correspond to P [ X µ < σ] =.687 nd P [ X µ < σ] =.9545, respectively. Emple.54. Figure 8 compres severl devition scores nd () Stndrd scores hve men of zero nd stndrd devition of.. Q( z ) (b) Scholstic z Aptitude Test scores hve men of 5 nd stndrd devition of. 5

7 prison of sevnorml distrive men of ition of.. scores hve d devition of FIGURE A Figure 8: Comprison of Severl Devition Scores nd the Norml Distribution scores hve devition of 6. percent of the d one stndrd n one nd two ercent beyond % 4% 34% 34% 4% % Stndrd Scores SAT Scores Binet Intelligence Scle Scores riticl nd Cretive Thinking, Figure 6. Pictures the Comprison of Severl Devition 99 Prentice-Hll, Inc. Reproduced by permission of Person Eduction, Inc. (c) Binet Intelligence Scle 45 scores hve men of nd stndrd devition of 6. In ech cse there re 34 percent of the scores between the men nd one stndrd devition, 4 percent between one nd two stndrd devitions, nd percent beyond two stndrd devitions. [Source: Beck, Applying Psychology: Criticl nd Cretive Thinking.].55. The re under norml probbility density function beyond 3σ from the men is quite smll. In fct, P [ X µ < 3σ] Therefore, pproimtely 99.73% of the probbility of norml distribution is within the intervl (µ 3σ, µ + 3σ). 45 Alfred Binet, who devised the first generl ptitude test t the beginning of the th century, defined intelligence s the bility to mke dpttions. The generl purpose of the test ws to determine which children in Pris could benefit from school. Binets test, like its subsequent revisions, consists of series of progressively more difficult tsks tht children of different ges cn successfully complete. A child who cn solve problems typiclly solved by children t prticulr ge level is sid to hve tht mentl ge. For emple, if child cn successfully do the sme tsks tht n verge 8-yer-old cn do, he or she is sid to hve mentl ge of 8. The intelligence quotient, or IQ, is defined by the formul: IQ = (Mentl Age/Chronologicl Age) There hs been gret del of controversy in recent yers over wht intelligence tests mesure. Mny of the test items depend on either lnguge or other specific culturl eperiences for correct nswers. Nevertheless, such tests cn rther effectively predict school success. If school requires lnguge nd the tests mesure lnguge bility t prticulr point of time in childs life, then the test is better-thn-chnce predictor of school performnce. 53

8 Definition.56. N (, ) is the stndrd Gussin (norml) distribution. We usully use Z to denote stndrd Gussin RV. In Ecel, use NORMSINV(RAND()). In MATLAB, use rndn. The stndrd norml cdf is denoted by Φ(z). It inherits ll properties of cdf. Moreover, note tht Φ( z) = Φ(z)..57. Reltionship between N (, ) nd N (m, σ ). () An rbitrry Gussin rndom vrible with men m nd vrince σ cn be represented s σz+m, where Z N (, ). This reltionship cn be used to generte generl Gussin RV from stndrd Gussin RV. (b) If X N ( m, σ ), the rndom vrible Z = X m σ is stndrd norml rndom vrible. Tht is, Z N (, ). Creting new rndom vrible by this trnsformtion is referred to s stndrdizing. The stndrdized vrible is clled stndrd score or z-score..58. It is impossible to epress the integrl of Gussin PDF between non-infinite limits (e.g., ()) s function tht ppers on most scientific clcultors. 54

9

10 An old but still populr technique to find integrls of the Gussin PDF is to refer to tbles tht hve been obtined by numericl integrtion. An emple of such tble is Tble 4, which lists Φ(z) for mny vlues of positive z. For X N ( m, σ ), we cn show tht the CDF of X cn be clculted from ( ) m F X () = Φ. σ Emple.59. Suppose Z N (, ). Evlute the following probbilities. () P [ Z ] (b) P [ Z ] Emple.6. Suppose X N (, ). Find P [ X ]. Emple.6. Signl Detection: Assume tht in the detection of digitl signl, the bckground noise follows norml distribution with men of volt nd stndrd devition of.45 volt. The system ssumes digitl hs been trnsmitted when the 55

11 z (z) z (z) z (z) z (z) z (z) z (z) Tble 4: The stndrd norml CDF: Φ(z) 56

12 voltge eceeds.9. (Otherwise, it ssumes digitl hs been N trnsmitted), Wht is the probbility of detectingerf ( z digitl ) when none ws sent? [Montgomery nd Runger, 3, E. 4-5] Q( z ).6. Q-function: Q (z) = π e d corresponds to P [Z > z] z where Z N (, ); tht is Q (z) is the probbility of the til of N (, ). The Q function is then complementry cdf (ccdf). z N (,).9.8 Q( z ) z z Figure 9: Q-function () Q is decresing function with Q () =. (b) Q ( z) = Q (z) = Φ(z) (c) Tble 5 lists the vlues of Q(z) for z between 3 to 5. For z between to 3, we use Q(z) = Φ(z). For z 5, the vlue of Q(z) is etremely smll. We my ssume Q(z)..63. Error function (MATLAB): erf (z) = π Q ( z ) () It is n odd function of z. z e d = (b) For z, it corresponds to P [ X < z] where X N (, ). (c) lim z erf (z) = 57

13 z Q(z) z Q(z) z Q(z) z Q(z) z Q(z) 3..35E E E E E E E E E E E E E 5 4..E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E 7 Tble 5: The stndrd norml complementry CDF: Q(z) 58

14 k, k odd = = k 35 ( k ) σ, k even k E ( X μ) ( k ) E ( X μ) k k 4 6 ( k ) σ, k odd X μ E = π [Ppoulis p ]. k 35 (d) erf ( z) = erf (z) ( k ) σ, k even 4 Vr X ( = 4μ σ + ( σ. )) ( ) (e) Φ() = + erf = () erfc, odd 8) For N (,) nd k, ( ) k k k E X = k E X = 35 ( k, ) k even (f) The complementry error function: erfc (z) = erf (z) = Q ( z 9) Error function (Mtlb): ( ) = z ) = ( π = erf z e d Q z) corresponds to π z e d P X < z where X ~ N,. N, erf ( z ) Q ( z ) ) erf ( z) lim = z Figure 3: erf-function nd Q-function b) erf ( z) = erf ( z).4.3 Eponentil Distribution Definition.64. The eponentil distribution is denoted by E (λ). () λ > is prmeter of the distribution, often clled the rte prmeter. (b) Chrcterized by { λe f X () = λ, >,, { e F X () = λ, >,, z 59

15 (c) MATLAB: X = eprnd(/λ) or rndom( ep,/λ) f X () = eppdf(,/λ) or pdf( ep,,/λ) F X () = epcdf(,/λ) or cdf( ep,,/λ).65. The eponentil distribution is intimtely relted to the Poisson process. In fct, the rndom vrible X tht equls the distnce (or length or durtion) between (ny) successive events of Poisson process with prmeter λ is n eponentil rndom vrible with the sme prmeter. Emple.66. Eponentil distribution is often used s probbility model for the (witing) time until the net rre event occurs. time elpsed until the net erthquke in certin region decy time of rdioctive prticle time between independent events such s rrivls t service fcility or rrivls of customers in shop. durtion of cell-phone cll time it tkes computer network to trnsmit messge from one node to nother..67. In Emple.35, we showed tht EX = λ. Emple.68. Suppose X E(λ), find P [ < X < ]..69. Survivl-, survivor-, or relibility-function: 6

16 Eercise.7. Eponentil rndom vrible s continuous version of geometric rndom vrible: Suppose X E (λ). Show tht X G (e λ ) nd X G (e λ ) Emple.7. Phone Compny A chrges $.5 per minute for telephone clls. For ny frction of minute t the end of cll, they chrge for full minute. Phone Compny B lso chrges $.5 per minute. However, Phone Compny B clcultes its chrge bsed on the ect durtion of cll. If T, the durtion of cll in minutes, is eponentil with prmeter λ = /3, wht re the epected revenues per cll E [R A ] nd E [R B ] for compnies A nd B? Solution: First, note tht ET = λ = 3. Hence, nd E [R B ] = E [.5 T ] =.5ET = $.45. E [R A ] = E [.5 T ] =.5E T. ( Now, recll, from Eercise.7, tht T G ) e λ. E T = Therefore, e λ E [R A ] =.5E T.59. Hence,.7. Memoryless property: The eponentil r.v. is the only continuous 46 r.v. on [, ) tht stisfies the memoryless property: P [X > s + X > s] = P [X > ] for ll > nd ll s > [8, p ]. In words, the future is independent of the pst. The fct tht it hsn t hppened yet, tells us nothing bout how much longer it will tke before it does hppen. Imgining tht the eponentilly distributed rndom vrible X represents the lifetime of n item, the residul life of n item hs the sme eponentil distribution s the originl lifetime, regrdless of how long the item hs been lredy in use. In other words, there is no deteriortion/degrdtion over time. If it is still currently working fter yers of use, then tody, its condition is just like new. 46 For discrete rndom vrible, geometric rndom vribles stisfy the memoryless property. 6

17 In prticulr, suppose we define the set B+ to be { + b : b B}. For ny > nd set B [, ), we hve P [X B + X > ] = P [X B] becuse P [X B + ] P [X > ] = B+ λe λt dt e λ τ=t = B λe λ(τ+) dτ e λ. Emple.73. The eponentil distribution is often used in relibility studies s the model for the time until filure of device. For emple, the lifetime of semiconductor chip might be modeled s n eponentil rndom vrible with men of 4, hours. The lck of memory property of the eponentil distribution implies tht the device does not wer out. Tht is, regrdless of how long the device hs been operting, the probbility of filure in the net hours is the sme s the probbility of filure in the first hours of opertion..74. The lifetime L of device with filures cused by rndom shocks might be ppropritely modeled s n eponentil rndom vrible. However, the lifetime L of device tht suffers slow mechnicl wer, such s bering wer, is better modeled by other distributions such s the Weibull distribution..75. Summry: X Support S X f X () = { Uniform U(, b) (, b), < < b, b, otherwise. Norml (Gussin) N (m, σ ) R πσ e ( m { λe Eponentil E(λ) (, ) λ, >,, σ ) Tble 6: Emples of probbility density functions. Here, λ, σ >. 6

18 .5 Function of Continuous Rndom Vribles: SISO Reconsider the derived rndom vrible Y = g(x). Recll tht we cn find EY esily by (): EY = E [g(x)] = g()f X ()d. However, there re cses when we hve to evlute probbility directly involving the rndom vrible Y or find f Y (y) directly. Recll tht for discrete rndom vribles, it is esy to find p Y (y) by dding ll p X () over ll such tht g() = y: p Y (y) = p X (). (3) R :g()=y For continuous rndom vribles, it turns out tht we cn t 47 simply integrte or dd the pdf of X to get the pdf of Y..76. For Y = g(x), if you wnt to find f Y (y), the following two-step procedure will lwys work nd is esy to remember: () Find the cdf F Y (y) = P [Y y]. (b) Compute the pdf from the cdf by finding the derivtive f Y (y) = d dy F Y (y) (s described in.5). Emple.77. Suppose X E(λ). Let Y = 5X. Find f Y (y). 47 When you pplied Eqution (3) to continuous rndom vribles, wht you would get is =, which is true but not interesting nor useful. 63

19 .78. Liner Trnsformtion: Suppose Y = X + b. Then, the cdf of Y is given by [ ] P X y b, >, F Y (y) = P [Y y] = P [X + b y] = [ ] P X y b, <. Now, by definition, we know tht [ P X y b ] ( ) y b = F X, nd P [ X y b ] [ = P X > y b ] + P ( ) y b = F X + P [ For continuous rndom vrible, P F Y (y) = ( ) F y b X F X ( y b X = y b, ) >,, <. [ X = y b ] [ X = y b ]. ] =. Hence, Finlly, fundmentl theorem of clculus nd chin rule gives ( ) f Y (y) = d dy F Y (y) = f y b X, >, ( ) f y b X, <. Note tht we cn further simplify the finl formul by using the function: f Y (y) = ( ) y b f X,. (4) Grphiclly, to get the plots of f Y, we compress f X horizontlly by fctor of, scle it verticlly by fctor of /, nd shift it to the right by b. Of course, if =, then we get the uninteresting degenerted rndom vrible Y b. 64

20 .79. Suppose X N (m, σ ) nd Y = X+b for some constnts nd b. Then, we cn use (4) to show tht Y N (m+b, σ ). Emple.8. Amplitude modultion in certin communiction systems cn be ccomplished using vrious nonliner devices such s semiconductor diode. Suppose we model the nonliner device by the function Y = X. If the input X is continuous rndom vrible, find the density of the output Y = X. 65

21 .8. There is no gurntee tht function of continuous rndom vrible will lso give continuous rndom vrible. Emple.8. Let X U(, ) nd Y = g(x) where {, <.6 g() =,.6. Before going deeply into the mth, it is helpful to think bout the nture of the derived rndom vrible Y. The definition of g() tells us tht Y hs only two possible vlues, Y = nd Y =. Thus, Y is discrete rndom vrible..83. Suppose X is continuous rndom vrible. To check whether rndom vrible Y = g(x) is continuous rndom vrible, there re two importnt techniques: () Check tht the cdf F Y (y) is continuous (no jump) function (for ll y). (b) Check tht P [Y = y] = for ll y. Emple.84. Consider Y = X when X is continuous rndom vrible. Emple

22 Emple.86. Consider Y = cos(x) when X is continuous rndom vrible..87. Let X be continuous rndom vrible nd Y = g(x). Suppose, for ech y, the collection of vlues tht stisfy y = g() is t most countble. Then Y is continuous rndom vrible. To see this, let B y be the collection of vlues tht stisfy y = g(). Then, [Y = y] = [X B y ] nd hence P [Y = y] = P [X B y ]. If B y is countble, then we cn write B y s countble disjoint union of events [X = ] for the B y. By the countble dditivity iom (P3), P [Y = y] = B y P [X = ]. (5) Becuse X is continuous, we know tht P [X = ] = for ny. Hence, the sum bove is. Emple.88. In Emple.8, there re uncountbly mny vlues tht stisfy g() =. Therefore, the condition in.87 is not stisfied nd hence we cn t use.87. Emple.89. Suppose X E(λ). Let Y = X. () Check tht Y is continuous rndom vrible. (b) Find F Y (y). 67

23 (c) Find f Y (y). Eercise.9 (F). Suppose X is uniformly distributed on the intervl (, ). (X U(, ).) Let Y = X. () Find f Y (y). (b) Find EY. Eercise.9 (F). Consider the function {, g() =, <. Suppose Y = g(x), where X U(, ). Remrk: The function g opertes like full-wve rectifier in tht if positive input voltge X is pplied, the output is Y = X, while if negtive input voltge X is pplied, the output is Y = X. () Find EY. (b) Plot the cdf of Y. (c) Find the pdf of Y 68

24 P [X B] = Discrete p X () B Continuous f X ()d P [X = ] = p X () = F X () F X ( ) B Intervl prob. EX = P X ((, b]) = F X (b) F X () P X ([, b]) = F X (b) F X ( ) P X ([, b)) = F X ( b ) F X ( ) P X ((, b)) = F X ( b ) F X () p X () = P X ((, b]) = P X ([, b]) = P X ([, b)) = P X ((, b)) b f X ()d = F (b) F () + f X ()d f Y (y) = d P [g(x) y]. dy For Y = g(x), p Y (y) = : g()=y p X () Alterntively, f Y (y) = k f X ( k ) g ( k ), For Y = g(x), P [Y B] = E [g(x)] = E [X ] = Vr X = p X () :g() B g()p X () p X () ( EX) p X () k re the rel-vlued roots of the eqution y = g(). f X ()d + {:g() B} + + g()f X ()d f X ()d ( EX) f X ()d Tble 7: Importnt Formuls for Discrete nd Continuous Rndom Vribles 69