STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 216 Néhémy Lim Chpter 5 : Continuous Rndom Vribles Nottions. N {, 1, 2,...}, set of nturl numbers (i.e. ll nonnegtive integers); N {1, 2,...}, set of ll positive integers; R (, ), set of ll rel numbers; R, set of ll rel numbers except ; R + [, ), set of ll nonnegtive rel numbers; The indictor function of set S is defined by : { 1 if x S 1 S (x) if x S 1 Quick review on Rndom Vribles Definition 1.1 (Probbility Spce). A probbility spce (Ω, A, P) consists of three prts: A smple spce, Ω, which is the set of ll possible outcomes of rndom experiment. A set of events A, where ech event E A is subset of Ω. Formlly, A is σ-lgebr on Ω. A function mpping P : A [, 1], clled probbility mesure tht ssigns probbilities to the events. A rel-vlued rndom vrible (r.r.v.) X ssigns number to outcomes ω Ω nd serves to describe some numericl property tht outcomes in Ω my hve. Formlly, the set of vlues tht rndom vrible X cn tke on should be endowed with σ-lgebr. In this course, we will lwys consider rndom vlues tht tke vlues on R. In this cse, the σ-lgebr tht we will work with is clled the Borel σ-lgebr on R nd is defined s follows. Definition 1.2 (Borel σ-lgebr on R). The Borel σ-lgebr on R is the smllest (in the sense of set inclusion) σ-lgebr tht contins ll open intervls in R. The Borel σ-lgebr on R is denoted B(R). Reminder: Open intervls in R re of three kinds : 1
(, b), with, b R; (, ), with R; (, b), with b R Remrks: Actully, prcticlly ll rel sets of interest nd ll you my think of re Borel sets. Definition 1.3 (Rndom vrible). Let (Ω, A) be mesurble spce of events on the smple spce Ω. A rel-vlued rndom vrible (r.r.v.) X is function mpping with domin Ω, i.e. X : Ω R, such tht for ny borel set B B(R): {ω Ω X(ω) B} A (1) The event {ω Ω X(ω) B} is simply denoted {X B}. Definition 1.4 (Discrete rndom vrible). A rel-vlued rndom vrible X is sid to be discrete if X cn tke : either finite number of vlues : X(Ω) {x i R, i..., n} for given n N or countbly infinite number of vlues : X(Ω) {x i R, i I} for given subset I N. The following exmples illustrte the difference between the two kinds of rndom vribles. X : number of burgers someone ets per yer. X is discrete since X(Ω) N Y : time spent on working STAT395 in hours per week. Y cn tke ny nonnegtive vlues nd they cnnot be listed or indexed. Y tkes n uncountbly infinite number of vlues : Y (Ω) R +. Thus Y is not discrete. Lst qurter, we studied thoroughly discrete rel-vlued rndom vribles. We re now focusing on continuous rel-vlued rndom vribles in this chpter. 2 Probbility Density Function nd Distribution Function Definition 2.1 (Probbility density function). A r.r.v. is sid to be (bsolutely) continuous if there exists probbility density function f X such tht, for ny Borel set B B(R): P(X B) f X (x) dx (2) 2 B
Then f X is clled the probbility density function (pdf) of the rndom vrible X. In prticulr, for ny rel numbers nd b, with < b, letting B [, b], we obtin from Eqution (2) tht : P( X b) b Property 2.1. If X is continuous r.r.v., then For ll R, f X (x) dx (3) P(X ) (4) In other words, the probbility tht continuous rndom vrible tkes on ny fixed vlue is zero. For ny rel numbers nd b, with < b P( X b) P( X < b) P( < X b) P( < X < b) (5) The bove eqution sttes tht including or not the bounds of n intervl does not modify the probbility of continuous rrv. Proof. Let us first prove Eqution (4) : P(X ) P(X [, ]) To prove Eqution (5), we simply notice tht f X (x) dx P( X b) P( X < b) P( < X b) P( < X < b) b f X (x) dx Theorem 2.1. A probbility density function completely determines the distribution of continuous rel-vlued rndom vrible. Remrk : This theorem mens tht two continuous rel-vlued rndom vribles X nd Y tht hve exctly the sme probbility density functions follow the sme distribution. We sy tht they re identiclly distributed. Definition 2.2 (Cumultive distribution function). Let (Ω, A, P) be probbility spce. The (cumultive) distribution function (c.d.f.) of rel-vlued rndom vrible X is the function F X given by F X (x) P(X x), for ll x R (6) 3
Property 2.2. Let F X be the distribution function of rndom vrible X. Following re some properties of F X : F X is incresing : x y F X (x) F X (y) lim x F X (x) nd lim x F X (x) F X is càdlàg : F X is right continuous : lim x x F X (x) F X (x ), for x R F X hs left limits : lim x x F X (x) exists, for x R Property 2.3 (Distribution function of continuous r.r.v.). Let X be continuous r.r.v. with pdf f X. Then the distribution function F X of X is given by : Proof. We hve the following : F X (x) F X (x) P(X x) f X (t) dt (7) P(X (, x)) f X (t) dt Theorem 2.2. A distribution function completely determines the distribution of continuous rel-vlued rndom vrible. Lemm 2.3. Let X be continuous r.r.v. with pdf f X nd distribution function F X. Then F X (x) f X(x), if f X is continuous t x; For ny rel numbers nd b with < b, P( X b) F X (b) F X () (8) Proof. This is direct ppliction of the Fundmentl Theorem of Clculus. Proposition 2.1. Let X be continuous r.r.v. on probbility spce (Ω, A, P) with pdf f X. Then, we hve : f X is nonnegtive on R : f X (x), for ll x R; (9) 4
f X is integrble on R nd f X (x) dx (1) Proof. Proof of (9) : Property 2.2 sttes tht the distribution function F X is incresing on R. Therefore F X (x). According to Lemm 2.3, F X (x) f X(x) if f X is continuous t x. This completes the proof. Proof of (1) : f X (x) dx P(X (, )) P(X R) P(Ω) In other words, the event tht X tkes on some vlue (X R) is the sure event Ω. Definition 2.3. A rel-vlued function f is sid to be vlid pdf if the following holds : f is nonnegtive on R : f is integrble on R nd f(x), for ll x R; (11) f(x) dx (12) This mens tht if f is vlid pdf, then there exists some continuous r.r.v. X tht hs f s its pdf Exmple. A continuous r.r.v. X is sid to follow uniform distribution on [, 1/2] if its pdf is : f X (x) { c if x 1/2 otherwise c1 [,1/2] (x) (13) Questions. (1) Determine c such tht f X stisfies the properties of pdf. (2) Give the cdf of X. 5
Proof. (1) Since f X is pdf, f X (x) should be nonnegtive for ll x R. This is the cse for x (, ) nd x (1/2, ) where f X (x) equls zero. On the intervl [, 1/2], f X (x) c. This implies tht c should be nonnegtive s well. Let us now focus on the second condition, f X (x) dx cx 1/2 f X (x) dx + dx + c 1 2 c 2 1/2 1/2 c dx + And we check tht indeed c 2 is nonnegtive. f X (x) dx + 1/2 1/2 dx (2) The distribution function F X of X is piecewise like its pdf : If x <, then f X (t) for ll t (, x]. F X (x) f X (t) dt If x 1/2, then f X (t) 2 for ll t [, x]. F X (x) + 2 t 2x x f X (t) dt f X (t) dt + 2 dt dt f X (t) dt f X (x) dx 6
If x > 1/2, then f X (t) for ll t [1/2, x]. F X (x) 1/2 f X (t) dt F X (1/2) + 2 1 2 f X (t) dt + f X (t) dt 1/2 In nutshell, F X is given by : if x < F X (x) 2x if x 1/2 1 if x > 1/2 3 Expecttion nd Vrince Definition 3.1 (Expected Vlue). Let X be continuous r.r.v. with pdf f X. If xf X(x) dx is bsolutely convergent, i.e. x f X(x) dx <, then we sy tht X is integrble. Then, the mthemticl expecttion (or expected vlue or men) of X exists, is denoted by E[X] nd is defined s follows : E[X] xf X (x) dx (14) Definition 3.2 (Expected Vlue of Function of Rndom Vrible). Let X be continuous r.r.v. with pdf f X. Let g : R R be piecewise continuous function. If rndom vrible g(x) is integrble. Then, the mthemticl expecttion of g(x) exists, is denoted by E[g(X)] nd is defined s follows : E[g(X)] g(x)f X (x) dx (15) Exmple. Compute the expecttion of continuous r.r.v. X following uniform distribution on [, 1/2]. As seen erlier, its pdf is given by: f X (x) { 2 if x 1/2 otherwise 2 1 [,1/2] (x) 7
Proof. The expecttion of X cn be computed s follows : E[X] xf X (x) dx xf X (x) dx + 1/2 xf X (x) dx + 1/2 1/2 + x 2 dx + x 2 1/2 4 xf X (x) dx Property 3.1. Let X be continuous r.r.v. with pdf f X. for ll c R, E[c] c (16) If g : R R is nonnegtive piecewise continuous function nd g(x) is integrble. Then, we hve : E[g(X)] (17) If g 1 : R R nd g 2 : R R re piecewise continuous functions nd g 1 (X) nd g 2 (X) re integrble such tht g 1 g 2. Then, we hve : E[g 1 (X)] E[g 2 (X)] (18) Proof. Proof of Eqution (16) : Here we consider the function of the rndom vrible X defined by : g(x) c. We then get : E[c] c c cf X (x) dx f X (x) dx }{{} 1 since f X is pdf Proof of Eqution (17) : This comes from the nonnegtivity of the integrl for nonnegtive functions. 8
Proof of Eqution (18) : This is direct ppliction of Eqution (17) pplied to function g 2 g 1. Property 3.2 (Linerity of Expecttion). Let X be continuous r.r.v. with pdf f X. If c 1, c 2 R nd g 1 : R R nd g 2 : R R re piecewise continuous functions nd g 1 (X) nd g 2 (X) re integrble. Then, we hve : E[c 1 g 1 (X) + c 2 g 2 (X)] c 1 E[g 1 (X)] + c 2 E[g 2 (X)] (19) Note tht Eqution (19) cn be extended to n rbitrry number of piecewise continuous functions. Proof. E[c 1 g 1 (X) + c 2 g 2 (X)] (c 1 g 1 (x) + c 2 g 2 (x))f X (x) dx c 1 g 1 (x)f X (x) dx + c 2 g 2 (x)f X (x) dx c 1 E[g 1 (X)] + c 2 E[g 2 (X)] Definition 3.3 (Vrince Stndrd Devition). Let X be rel-vlued rndom vrible. When E[X 2 ] exists, the vrince of X is defined s follows : Vr(X) E[(X E[X]) 2 ] (2) Vr(X) is sometimes denoted σx 2. The positive squre root of the vrince is clled the stndrd devition of X, nd is denoted σ X. Tht is: σ X Vr(X) (21) Property 3.3. The vrince of rel-vlued rndom vrible X stisfies the following properties : Vr(X) If, b R re two constnts, then Vr(X + b) 2 Vr(X) Proof. This property is true for ny kind of rndom vribles (discrete or continuous). See proof of Property 4.1 given in the lecture notes of Chpter 4. 9
Theorem 3.1 (König-Huygens formul). Let X be rel-vlued rndom vrible. When E[X 2 ] exists, the vrince of X is lso given by : Vr(X) E[X 2 ] (E[X]) 2 (22) Proof. This property is true for ny kind of rndom vribles (discrete or continuous). See proof of Theorem 4.1 given in the lecture notes of Chpter 4. In the cse of continuous rndom vribles, Eqution (22) becomes : ( ) 2 Vr(X) x 2 f X (x) dx xf X (x) dx Exmple. Compute the vrince of continuous r.r.v. X following uniform distribution on [, 1/2]. As seen erlier, its pdf is given by: f X (x) 2 1 [,1/2] (x) Proof. We computed previously the expecttion of X tht is E[X] /4. Computing the vrince of X thus boils down to clculting E[X 2 ] : Therefore, E[X 2 ] + x 2 f X (x) dx x 2 f X (x) dx + 1/2 2 1 12 3 x3 1/2 x 2 2 dx + 1/2 Vr(X) 12 x 2 f X (x) dx + ( ) 2 1 4 48 1/2 x 2 f X (x) dx 4 Common Continuous Distributions 4.1 Uniform Distribution Definition 4.1. A continuous r.r.v. is sid to follow uniform distribution U(, b) on segment [, b], with < b, if its pdf is f X (x) b 1 [,b](x) (23) 1
Proof. Let us prove tht the pdf of uniform distribution is ctully vlid pdf : Is f X (x) for x R? If x < or x > b, then f X (x). If x [, b], then f X (x) /(b ) > since < b. Let us show tht f x(x) dx : f x (x) dx b f x (x) dx + b 1 + b dx + b b x f x (x) dx + b f x (x) dx Property 4.1 (Men nd Vrince for Uniform Distribution). If X follows uniform distribution U(, b), then its expected vlue is given by : E[X] + b 2 (24) its vrince is given by : Vr(X) (b )2 12 (25) Proof. Expecttion : E[X] b b xf X (x) dx b 1 x dx b 2 x2 b 2 2 2 b + b 2 11
Vrince : Let us first compute E[X 2 ] E[X 2 ] b b x 2 f X (x) dx b 1 x 2 dx b 3 x3 b 3 3 3 b b2 + b + 2 3 Then using the shortcut formul (22), we find : Vr(X) b2 + b + 2 3 ( + b)2 4 (b )2 12 Motivtion. Most computer progrmming lnguges include functions or librry routines tht provide rndom number genertors. They re often designed to provide rndom byte or word, or floting point number uniformly distributed between nd 1. The qulity i.e. rndomness of such librry functions vries widely from completely predictble output, to cryptogrphiclly secure. There re couple of methods to generte rndom number bsed on probbility density function. These methods involve trnsforming uniform rndom number in some wy. Becuse of this, these methods work eqully well in generting both pseudo-rndom nd true rndom numbers. 4.2 Gussin Distribution Exmple. A resturnt wnts to dvertise new burger they cll The Qurterkilogrm. Of course, none of their burgers will exctly weigh exctly 25 grms. However, you my expect tht most of the burgers weights will fll in some smll intervl centered round 25 grms. Definition 4.2. A continuous rndom vrible is sid to follow norml (or Gussin) distribution N (µ, σ 2 ) with prmeters, men µ nd vrince σ 2 if its pdf f X is given by: f X (x) σ 2π exp { 1 ( ) } 2 x µ, for x R (26) 2 σ We lso sy tht X is normlly distributed or X is norml (or Gussin) r.r.v. with prmeters µ nd σ 2. 12
Property 4.2 (Men nd Vrince for Norml Distribution). If X follows norml distribution N (µ, σ 2 ), then its expected vlue is given by : its vrince is given by : E[X] µ (27) Vr(X) σ 2 (28) Remrk. The norml distribution is one of the most (even perhps the most) importnt distributions in Probbility nd Sttistics. It llows to model mny nturl, physicl nd socil phenomenons. We will see lter in this course how ll the distributions re somehow relted to the norml distribution. Property 4.3. If X follows norml distribution N (µ, σ 2 ). Then its pdf f X hs the following properties: (1) f X is symmetric bout the men µ : (2) f X is mximized t x µ. f X (µ x) f X (µ + x), for x R (29) (3) The limit of f X (x), s x pproches or, is : lim f X(x) lim f X(x) (3) x x Definition 4.3 (Stndrd Norml Distribution). We sy tht continuous rrv X follows stndrd norml distribution if X follows norml distribution N (, 1) with men nd vrince 1. We lso sy tht X is stndrd norml rndom vrible. In prticulr, the cdf of stndrd norml rndom vrible is denoted Φ, tht is : Φ(x) F X (x) 1 2π e t2 2 dt, for x R (31) Property 4.4. The cdf of stndrd norml rndom vrible stisfies the following property: Φ( z) Φ(z), for z R (32) Property 4.5. Let R nd b R. If X follows norml distribution N (µ, σ 2 ), then rndom vrible X + b follows norml distribution N (µ + b, 2 σ 2 ). 13
Proof. Let us ssume tht >. Consider rndom vrible Y X+b. We will prove tht F Y, the cdf of Y is the cdf of norml distribution N (µ + b, 2 σ 2 ). For y R, we hve tht : F Y (y) P(Y y) P(X + b y) ( P X y b ) ( ) y b F X { y b 1 σ 2π exp 1 2 ( ) } 2 x µ dx σ Now, consider the chnge of vrible : t x + b Hence, we hve : dt dx or dx dt. F Y (y) P(Y y) ( y 1 t b σ 2π exp 1 µ ) 2 dt 2 σ { y 1 σ 2π exp 1 ( ) } 2 t (µ + b) dt 2 σ We now recognize the cdf of norml distribution N (µ + b, 2 σ 2 ), which completes the proof. ( ) If is negtive, F Y (y) turns out to be 1 F y b X (why?) Corollry 4.1. If X follows norml distribution N (µ, σ 2 ), then rndom vrible Z defined by: Z X µ σ is stndrd norml rndom vrible. Finding Norml probbilities. If X N (µ, σ 2 ). The bove property leds us to the following strtegy for finding probbilities P( X b) : (1) Trnsform X,, nd b, by: Z X µ σ (2) Use the stndrd norml N (, 1) Tble to find the desired probbility. Exmple. Let X be the weight of so-clled qurter-kilogrm burger. Assume X follows norml distribution with men 25 grms nd stndrd devition 15 grms. 14
() Wht is the probbility tht rndomly selected burger hs weight below 24 grms? (b) Wht is the probbility tht rndomly selected burger hs weight bove 27 grms? (c) Wht is the probbility tht rndomly selected burger hs weight between 23 nd 265 grms? 4.3 Quntiles Previously, we lerned how to use the stndrd norml curve N (, 1) to find probbilities concerning rndom vrible following norml distribution N (µ, σ 2 ). Wht hppens if we wnt to find some rnge of vlues of X in order to rech some probbility? Definition 4.4 (Quntile). Let X be r.r.v. with distribution function F X nd α [, 1]. A quntile of order α for the distribution of X, denoted q α, is defined s follows : q α inf {x R F X (x) α } (33) Remrks : The quntile of order 1/2 is clled the medin of the distribution If F X is bijective function, then q α F 1 X (α) Property 4.6. For α (, 1), the quntile function of stndrd norml rndom vrible is given by : Φ 1 (α) Φ 1 (1 α) (34) Finding the Quntiles of Norml Distribution. vlue of norml rndom vrible X N (µ, σ 2 ): In order to find the (1) Find in the Tble the z (x µ)/σ vlue ssocited with the desired probbility. (2) Use the trnsformtion x µ + zσ. Exmple. Suppose X, the grde on midterm exm, is normlly distributed with men 7 nd stndrd devition 1. () The instructor wnts to give 15% of the clss n A. Wht cutoff should the instructor use to determine who gets n A? (b) The instructor now wnts to give the next 1% of the clss n A-. For which rnge of grdes should the instructor ssign n A-? 15
4.4 Exponentil Distribution Definition 4.5 (Approximte Poisson process). Let Y denote the number of events in given continuous intervl. Then Y follows n pproximte Poisson process with prmeter µ > if : (1) The number of events occurring in non-overlpping intervls re independent. (2) The probbility of exctly one event in short intervl of length h /n is pproximtely µh µ/n (3) The probbility of exctly two or more events in short intervl is essentilly zero. Motivting exmple. Suppose Y, following n pproximte Poisson process, equls the number of people joining the line to visit the Eiffel Tower in n intervl of one hour. If µ, the men number of people rriving in n intervl of one hour, is 8, we re interested in the continuous rndom vrible X, the witing time until the first visitor rrives. Definition 4.6. A continuous rndom vrible is sid to follow n exponentil distribution E(λ) with λ > if its pdf f X is given by: f X (x) λe λx 1 R+ (x), for x R (35) Property 4.7 (Men nd Vrince for n Exponentil Distribution). If X follows n exponentil distribution E(λ), then its expected vlue is given by : E[X] λ (36) its vrince is given by : Vr(X) λ 2 (37) Exmple. Suppose tht people join the line to visit the Eiffel Tower ccording to n pproximte Poisson process t men rte of 8 visitors per hour. Wht is the probbility tht nobody joins the line in the next 3 seconds? 16