Sirindhorn Interntionl Institute of Technology Thmmst University School of Informtion, Computer nd Communiction Technology ECS35 3/ Prt IV. Dr.Prpun.4 Fmilies of Continuous Rndom Vribles Theorem.4 sttes tht ny nonnegtive function f() whose integrl over the intervl (, + ) equls cn be regrded s probbility density function of rndom vrible. In rel-world pplictions, however, specil mthemticl forms nturlly show up. In this section, we introduce couple fmilies of continuous rndom vribles tht frequently pper in prcticl pplictions. The probbility densities of the members of ech fmily ll hve the sme mthemticl form but differ only in one or more prmeters..4. Uniform Distribution Definition.35. For uniform rndom vrible on n intervl [, b], we denote its fmily by uniform([, b]) or U([, b]) or simply U(, b). Epressions tht re synonymous with X is uniform rndom vrible re X is uniformly distributed, X hs uniform distribution, nd X hs uniform density. This fmily is chrcterized by {, <, > b f X () = b, b The rndom vrible X is just s likely to be ner ny vlue in [, b] s ny other vlue. 7
In MATLAB, () use X = +(b-)*rnd or X = rndom( Uniform,,b) to generte the RV, (b) use pdf( Uniform,,,b) nd cdf( Uniform,,,b) to clculte the pdf nd cdf, respectively. {, <, > b Eercise.36. Show tht F X () = b, b 84 Probbility theory, rndom vribles nd rndom processes f () F () b Fig. 3.5 The pdf nd cdf for the uniform rndom vrible. Figure 5: The pdf nd cdf for the uniform rndom vrible. [6, Fig. 3.5] f () F () Emple.37 (F). Suppose X is uniformly distributed on πσ the intervl (, ). (X U(, ).) b b () Plot the pdf f X () of X. Fig. 3.6 (b) Plot the cdf F X () of X. The pdf nd cdf of Gussin rndom vrible. μ μ Gussin (or norml) rndom vrible is described by the following pdf: f () = This is continuous rndom vrible tht.38. The uniform distribution provides probbility model for selecting point t rndom from the intervl [, b]. { } ep ( μ) πσ σ, (3.6) Use with cution to model quntity tht is known to vry where μrndomly σ re two between prmeters nd whose b but mening bout is described which little lter. Itelse is usully is known. denoted s N (μ, σ ). Figure 3.6 shows sketches of the pdf nd cdf of Gussin rndom vrible. The Gussin rndom vrible is the most importnt nd frequently encountered rndom vrible in communictions. This is becuse 8 therml noise, which is the mjor source of noise in communiction systems, hs Gussin distribution. Gussin noise nd the Gussin pdf re discussed in more depth t the end of this chpter. The problems eplore other pdf models. Some of these rise when rndom vrible
Emple.39. [9, E. 4. p. 4-4] In coherent rdio communictions, the phse difference between the trnsmitter nd the receiver, denoted by Θ, is modeled s hving uniform density on [ π, π]. () P [Θ ] = (b) P [ Θ π ] = 3 4 Eercise.4. Show tht EX = +b (b ), Vr X =, nd E [ X ] ( = 3 b + b + )..4. Gussin Distribution.4. This is the most widely used model for the distribution of rndom vrible. When you hve mny independent rndom vribles, fundmentl result clled the centrl limit theorem (CLT) (informlly) sys tht the sum (techniclly, the verge) of them cn often be pproimted by norml distribution. Definition.4. Gussin rndom vribles: () Often clled norml rndom vribles becuse they occur so frequently in prctice. (b) In MATLAB, use X = rndom( Norml,m,σ) or X = σ*rndn + m. (c) f X () = πσ e ( m σ ). In Ecel, use NORMDIST(,m,σ,FALSE). In MATLAB, use normpdf(,m,σ) or pdf( Norml,,m,σ). Figure 6 displys the fmous bell-shped grph of the Gussin pdf. This curve is lso clled the norml curve. 9
84 Probbility theory, rndom vribles nd rndom processes f () F () (d) F X () hs no closed-form epression. However, see.48. Inb MATLAB, use normcdf(,m,σ) or cdf( Norml,,m,σ). In Ecel, use NORMDIST(,m,σ,TRUE). Fig. 3.5 b (e) We write X N ( m, σ ). The pdf nd cdf for the uniform rndom vrible. b πσ f () F () Fig. 3.6 The pdf nd cdf of Gussin rndom vrible. Figure 6: The pdf nd cdf of N (µ, σ ). [6, Fig. 3.6] μ μ Gussin (or norml) rndom vrible is described by the following pdf:.43. EX = m nd Vr X = σ. This is continuous rndom vrible tht { } f () = ep ( μ) πσ σ, (3.6).44. Importnt probbilities: P [ X µ < σ] =.687; P [ X µ > σ] =.373; P [ X µ > σ] =.455; P [ X µ < σ] =.9545 These vlues re illustrted in Figure 9. where μ nd σ re two prmeters whose mening is described lter. It is usully denoted s N (μ, σ ). Figure 3.6 shows sketches of the pdf nd cdf of Gussin rndom vrible. The Gussin rndom vrible is the most importnt nd frequently encountered rndom vrible in communictions. This is becuse therml noise, which is the mjor source of noise in communiction systems, hs Gussin distribution. Gussin noise nd the Gussin pdf re discussed in more depth t the end of this chpter. The problems eplore other pdf models. Some of these rise when rndom vrible is pssed through nonlinerity. How to determine the pdf of the rndom vrible in this cse is discussed net. Emple.45. Figure compres severl devition scores nd the norml distribution: Functions of rndom vrible A function of rndom vrible y = g() is itself rndom vrible. From the definition, the cdf of y cn be written s () Stndrd scores hve men of zero nd stndrd devition of.. F y (y) = P(ω : g((ω)) y). (3.7) (b) Scholstic Aptitude Test scores hve men of 5 nd stndrd devition of. 3
9 3.5 The Gussin rndom vrible nd process.6 ().4 Signl mplitude (V)...4.6.8..4.6.8 t (s) 4 3.5 3 (b) Histogrm Gussin fit Lplcin fit f () (/V).5.5.5.5.5 (V) Fig. 3.4 () A smple skeletl muscle (emg) signl, nd (b) its histogrm nd pdf fits. Figure 7: Electricl ctivity of skeletl muscle: () A smple skeletl muscle (emg) signl, nd (b) its histogrm nd pdf fits. [6, Fig. 3.4] [ [ ] = f ()d] = K e d = K = K e d = = y= 3 y= e y dy e ( +y ) ddy. (3.3)
3.5 The Gussin rndom vrible nd process f ().4.35.5..5. j m jω 3) Fourier trnsform:.5 ( f ) f ( ) e dt e ω F = = ωσ. X 4) Note tht 5 5 5 5 α π e d = α. Fig. 3.5 Plots of the zero-men Gussin m pdf for different vlues of stndrd devition, σ. 5) P[ X > ] = Q σ ; [ ] m m P X < = Q = Q. σ σ X Figure 8: Plots of the zero-men Gussin pdf for different vlues of stndrd devition, σ X. [6, Fig. 3.5] 6) Q-function: ( ), pictures the comprison of sevtion scores nd the norml distritndrd scores hve men of d stndrd devition of.. ic Aptitude Test scores hve 5 nd stndrd devition of P X μ < σ =.687, P X μ > σ =.373 σ = σ = Rnge (±kσ ) k = k = k = 3 k = 4 of the pdf re ignorble. μ Indeed σ μ μwhen + σ communiction systems re considered lter it is the μ σ μ μ + σ presence of these tils tht results in bit errors. The probbilities re on the order of 3, very smll, but still significnt in terms of system performnce. It is of interest to Figure 9: Q zprobbility = e d see how fr, in terms of z π density corresponds function to P[ X > zof ] where X X N ~ N(µ, (,σ ) ;). σ, one must be from the men vlue to hve the different levels of tht is Q error probbilities. ( z ) is the probbility of the til of N As shll be seen in lter chpters (, this ). FIGURE A trnsltes to the required SNR to chieve specified bit error probbility. N (, This ) is lso shown in Tble 3.. Hving considered the single (or univrite) Gussin rndom vrible, we turn our ttention to the cse of two jointly Gussin rndom vribles (or the bivrite cse). Agin they re described by their joint pdf which, in generl, is n eponentil whose eponent is qudrtic in the two vribles, i.e., f,y (, y) = QKe ( z ) ( +b+cy+dy+ey +f ), where the constnts K,, b, c, d, e, nd f re chosen to stisfy the bsic properties.5 of vlid joint pdf, nmely being lwys nonnegtive ( ), hving unit volume, nd lso tht the mrginl pdfs, f () = f z,y(, y)dy nd f y (y) = % 4% 34% 34% f,y(, 4% y)d, re % vlid. Written in stndrd form) the Q is joint decresing pdf is function with Q ( ) =. 3..... 3. Q z = Q z Stndrd Scores Intelligence Scle scores hve nd stndrd devition of 6. cse there re 34 percent of the tween the men nd one stndrd b) ( ) ( ), 4 percent between one nd c) Qtwo ( Q( z )) = z3 4 5 6 7 8 devitions, nd percent beyond π SAT π Scores 4 sin θ rd devitions. sin θ d) Q( ) = e dθ π 5 68. Q ( 84 ) = e dθ π 6. 3 48 Binet Intelligence Scle Scores d Q stndrd score. 6.3 Applictions of Norml Distributions The norml distribution cn help us to determine probbilities. 6.4 Nottion The z nottion is criticl in the use of norml σ = 5.3 distributions. 6.5 Norml Approimtion of the Binomil Binomil probbilities cn be estimted by using norml distribution. l Probbility Distributions nce Scores l probbility distribution P is X considered μ > σ Tble =.455, 3. Influence the P X single of μ σ< on σ different = most.9545 quntities importnt probtribution. An unlimited number of continuous rndom vribles hve either norml oimtely norml distribution. f X P(m kσ < m + ( kσ ) ( ) f X ).683.955.997.999 ll fmilir with IQ (intelligence Error probbility quotient) scores nd/or 3 SAT (Scholstic 4 Aptitude 6 Test) 8 scores hve men of Distnce nd from stndrd the men devition 3.9 of 6. SAT 3.7scores hve 4.75 men 5.6 of 95% 68% stndrd devition of. But did you know tht these continuous rndom vribles norml distribution? k, Applying Psychology: Criticl nd Cretive e) Thinking, ( ) = Figure e ; 6. Q Pictures ( f ( ) ) = the Comprison e of f ( Severl ). Devition the Norml Distribution, Figure 99 Prentice-Hll, : Comprison d Inc. Reproduced of π Severl dby permission Devition of πperson Scoresd Eduction, nd the Norml Inc. Distribution d ( f ( ) ) d 3
(c) Binet Intelligence Scle 4 scores hve men of nd stndrd devition of 6. In ech cse there re 34 percent of the scores between the men nd one stndrd devition, 4 percent between one nd two stndrd devitions, nd percent beyond two stndrd devitions. [Source: Beck, Applying Psychology: Criticl nd Cretive Thinking.].46. N (, ) is the stndrd Gussin (norml) distribution. In Ecel, use NORMSINV(RAND()). In MATLAB, use rndn. The stndrd norml cdf is denoted by Φ(z). It inherits ll properties of cdf. Moreover, note tht Φ( z) = Φ(z)..47. Reltionship between N (, ) nd N (m, σ ). () An rbitrry Gussin rndom vrible with men m nd vrince σ cn be represented s σz+m, where Z N (, ). 4 Alfred Binet, who devised the first generl ptitude test t the beginning of the th century, defined intelligence s the bility to mke dpttions. The generl purpose of the test ws to determine which children in Pris could benefit from school. Binets test, like its subsequent revisions, consists of series of progressively more difficult tsks tht children of different ges cn successfully complete. A child who cn solve problems typiclly solved by children t prticulr ge level is sid to hve tht mentl ge. For emple, if child cn successfully do the sme tsks tht n verge 8-yer-old cn do, he or she is sid to hve mentl ge of 8. The intelligence quotient, or IQ, is defined by the formul: IQ = (Mentl Age/Chronologicl Age) There hs been gret del of controversy in recent yers over wht intelligence tests mesure. Mny of the test items depend on either lnguge or other specific culturl eperiences for correct nswers. Nevertheless, such tests cn rther effectively predict school success. If school requires lnguge nd the tests mesure lnguge bility t prticulr point of time in childs life, then the test is better-thn-chnce predictor of school performnce. 33
This reltionship cn be used to generte generl Gussin RV from stndrd Gussin RV. (b) If X N ( m, σ ), the rndom vrible Z = X m σ is stndrd norml rndom vrible. Tht is, Z N (, ). Creting new rndom vrible by this trnsformtion is referred to s stndrdizing. The stndrdized vrible is clled stndrd score or z-score..48. It is impossible to epress the integrl of Gussin PDF between non-infinite limits (e.g., ()) s function tht ppers on most scientific clcultors. An old but still populr technique to find integrls of the Gussin PDF is to refer to tbles tht hve been obtined by numericl integrtion. One such tble is the tble tht lists Φ(z) for mny vlues of positive z. For X N ( m, σ ), we cn show tht the CDF of X cn be clculted by ( ) m F X () = Φ. σ Emple.49. Suppose Z N (, ). Evlute the following probbilities. () P [ Z ] 34
(b) P [ Z ] Emple.5. Suppose X N (, ). Find P [ X ]..5. Q-function: Q (z) = z N, π e z d corresponds to P [X > z] where X N (, ); tht is Q (z) is the probbility of the til of N (, ). The Q function is then complementry cdf (ccdf). erf ( z ) Q ( z ) N (,).9.8 Q( z ).7.6.5.4.3. z. -3 - - 3 z Figure : Q-function () Q is decresing function with Q () =. (b) Q ( z) = Q (z) = Φ(z).5. Error function (MATLAB): erf (z) = π Q ( z ) () It is n odd function of z. z e d = (b) For z, it corresponds to P [ X < z] where X N (, ). (c) lim z erf (z) = 35
k, k odd = = k 35 ( k ) σ, k even k E ( X μ) ( k ) E ( X μ) k k 4 6 ( k ) σ, k odd X μ E = π [Ppoulis p ]. k 35 (d) erf ( z) = erf (z) ( k ) σ, k even 4 Vr X ( = 4μ σ + ( σ. )) ( ) (e) Φ() = + erf = () erfc, odd 8) For N (,) nd k, ( ) k k k E X = k E X = 35 ( k, ) k even (f) The complementry error function: erfc (z) = erf (z) = Q ( z 9) Error function (Mtlb): ( ) = z ) = ( π = erf z e d Q z) corresponds to π z e d P X < z where X ~ N,. N, erf ( z ) Q ( z ) ) erf ( z) lim = z Figure : erf-function nd Q-function b) erf ( z) = erf ( z).4.3 Eponentil Distribution Definition.53. The eponentil distribution is denoted by E (λ). () λ > is prmeter of the distribution, often clled the rte prmeter. (b) Chrcterized by { λe f X () = λ, >,, { e F X () = λ, >,, z 36
Survivl-, survivor-, or relibility-function: (c) MATLAB: X = eprnd(/λ) or rndom( ep,/λ) f X () = eppdf(,/λ) or pdf( ep,,/λ) F X () = epcdf(,/λ) or cdf( ep,,/λ) Emple.54. Suppose X E(λ), find P [ < X < ]. Eercise.55. Eponentil rndom vrible s continuous version of geometric rndom vrible: Suppose X E (λ). Show tht X G (e λ ) nd X G (e λ ) Emple.56. The eponentil distribution is intimtely relted to the Poisson process. It is often used s probbility model for the (witing) time until rre event occurs. time elpsed until the net erthquke in certin region decy time of rdioctive prticle time between independent events such s rrivls t service fcility or rrivls of customers in shop. durtion of cell-phone cll time it tkes computer network to trnsmit messge from one node to nother. 37
.57. EX = λ Emple.58. Phone Compny A chrges $.5 per minute for telephone clls. For ny frction of minute t the end of cll, they chrge for full minute. Phone Compny B lso chrges $.5 per minute. However, Phone Compny B clcultes its chrge bsed on the ect durtion of cll. If T, the durtion of cll in minutes, is eponentil with prmeter λ = /3, wht re the epected revenues per cll E [R A ] nd E [R B ] for compnies A nd B? Solution: First, note tht ET = λ = 3. Hence, nd E [R B ] = E [.5 T ] =.5ET = $.45. E [R A ] = E [.5 T ] =.5E T. Now, recll tht T G ( e λ ). Hence, E T = e λ 3.53. Therefore, E [R A ] =.5E T.59..59. Memoryless property: The eponentil r.v. is the only continuous 4 r.v. on [, ) tht stisfies the memoryless property: P [X > s + X > s] = P [X > ] for ll > nd ll s > [8, p. 57 59]. In words, the future is independent of the pst. The fct tht it hsn t hppened yet, tells us nothing bout how much longer it will tke before it does hppen. Imgining tht the eponentilly distributed rndom vrible X represents the lifetime of n item, the residul life of n item hs the sme eponentil distribution s the originl lifetime, regrdless of how long the item hs been lredy in use. In 4 For discrete rndom vrible, geometric rndom vribles stisfy the memoryless property. 38
other words, there is no deteriortion/degrdtion over time. If it is still currently working fter yers of use, then tody, its condition is just like new. In prticulr, suppose we define the set B+ to be { + b : b B}. For ny > nd set B [, ), we hve P [X B + X > ] = P [X B] becuse P [X B + ] P [X > ] = B+ λe λt dt e λ τ=t = B λe λ(τ+) dτ e λ..5 Function of Continuous Rndom Vribles: SISO Reconsider the derived rndom vrible Y = g(x). Recll tht we cn find EY esily by (): EY = E [g(x)] = g()f X ()d. However, there re cses when we hve to evlute probbility directly involving the rndom vrible Y or find f Y (y) directly. Recll tht for discrete rndom vribles, it is esy to find p Y (y) by dding ll p X () over ll such tht g() = y: p Y (y) = p X (). (3) R :g()=y For continuous rndom vribles, it turns out tht we cn t 43 simply integrte the pdf of X to get the pdf of Y. 43 When you pplied Eqution (3) to continuous rndom vribles, wht you would get is =, which is true but not interesting nor useful. 39
.6. For Y = g(x), if you wnt to find f Y (y), the following two-step procedure will lwys work nd is esy to remember: () Find the cdf F Y (y) = P [Y y]. (b) Compute the pdf from the cdf by finding the derivtive f Y (y) = d dy F Y (y) (s described in.3)..6. Liner Trnsformtion: Suppose Y = X + b. Then, the cdf of Y is given by F Y (y) = P [Y y] = P [X + b y] = Now, by definition, we know tht [ P X y b ] ( ) y b = F X, nd P [ X y b ] [ = P X > y b ] + P ( ) y b = F X + P [ For continuous rndom vrible, P F Y (y) = ( ) F y b X F X ( y b X = y b [ ] P X y b, >, [ ] P X y b, <., ) >,, <. [ X = y b ] [ X = y b ]. ] =. Hence, Finlly, fundmentl theorem of clculus nd chin rule gives ( ) f Y (y) = d dy F Y (y) = f y b X, >, ( ) f y b X, <. Note tht we cn further simplify the finl formul by using the function: f Y (y) = ( ) y b f X,. (4) 4
Grphiclly, to get the plots of f Y, we compress f X horizontlly by fctor of, scle it verticlly by fctor of /, nd shift it to the right by b. Of course, if =, then we get the uninteresting degenerted rndom vrible Y b..6. Suppose X N (m, σ ) nd Y = X+b for some constnts nd b. Then, we cn use (4) to show tht X N (m+b, σ ). Emple.63. Amplitude modultion in certin communiction systems cn be ccomplished using vrious nonliner devices such s semiconductor diode. Suppose we model the nonliner device by the function Y = X. If the input X is continuous rndom vrible, find the density of the output Y = X. 4
Eercise.64 (F). Suppose X is uniformly distributed on the intervl (, ). (X U(, ).) Let Y = X. () Find f Y (y). (b) Find EY. Eercise.65 (F). Consider the function {, g() =, <. Suppose Y = g(x), where X U(, ). Remrk: The function g opertes like full-wve rectifier in tht if positive input voltge X is pplied, the output is Y = X, while if negtive input voltge X is pplied, the output is Y = X. () Find EY. (b) Plot the cdf of Y. (c) Find the pdf of Y 4
P [X B] = Discrete p X () B Continuous f X ()d P [X = ] = p X () = F () F ( ) B Intervl prob. EX = P X ((, b]) = F (b) F () P X ([, b]) = F (b) F ( ) P X ([, b)) = F ( b ) F ( ) P X ((, b)) = F ( b ) F () p X () = P X ((, b]) = P X ([, b]) = P X ([, b)) = P X ((, b)) b f X ()d = F (b) F () + f X ()d f Y (y) = d P [g(x) y]. dy For Y = g(x), p Y (y) = : g()=y p X () Alterntively, f Y (y) = k f X ( k ) g ( k ), For Y = g(x), P [Y B] = E [g(x)] = E [X ] = Vr X = p X () :g() B g()p X () p X () ( EX) p X () k re the rel-vlued roots of the eqution y = g(). f X ()d + {:g() B} + + g()f X ()d f X ()d ( EX) f X ()d Tble 4: Importnt Formuls for Discrete nd Continuous Rndom Vribles 43