B. Maddah ENMG 622 Smulaton 2/22/ Probablty and Random Varable Prmer Sample space and Events Suppose that an eperment wth an uncertan outcome s performed (e.g., rollng a de). Whle the outcome of the eperment s not known n advance, the set of all possble outcomes s known. Ths set s the sample space, Ω. For eample, when rollng a de Ω = {, 2, 3, 4, 5, 6}. When tossng a con, Ω = {H, T}. When measurng lfe tme of a machne (years), Ω = {, 2, 3, }. A subset E Ω s known as an event. E.g., when rollng a de, E = {} s the event that one appears and F = {, 3, 5} s the event that an odd number appears. Probablty of an event If an eperment s repeated for a number of tmes whch s large enough, the fracton of tme that event E occurs s the probablty that event E occurs, P{E}. E.g., when rollng a far de, P{} = /6, and P{, 3, 5} = 3/6 = /2. When tossng a far con, P{H} = P{T} = /2. In some cases, events are not repeated many tmes. For such cases, probabltes can be a measure of belef (subjectve probablty).
Aoms of probablty () For E Ω, P{E} ; (2) P{ Ω} = ; (3) For events E, E 2,, E,, wth E Ω, E E j =, for all and j, P E = P{ E} =. = Implcatons The aoms of probablty mply the followng results: o For E and F Ω, P{E or F} = P{E F} = P{E} + P{F} P{E F} ; o If E and F are mutually eclusve (.e., E F = ), then P{E F} = P{E} + P{F}; o For E Ω, let E c be the complement of E (.e., E E c = Ω), P{E c } = P{E}; o P{ } =. Condtonal probablty The probablty that event E occurs gven that event F has already occurred s PE { F} PE { F} =. PF { } P{E F} = P{E and F}. 2
Independent events For E and F Ω, P{E F} = P{E F}P{F}. Two events are ndependent f an only f P{E F} = P{E}P{F}. That s, P{E F} = P{E}. Eample Suppose that two far cons are tossed. What s the probablty that ether the frst or the second con falls heads? In ths eample, Ω = {(H, H), (H, T), (T, H), (T,T)}. Let E (F) be the event that the frst (second) con falls heads, E ={(H, H), (H, T)} and F = {(H, H), (T, H)}, and E F ={H, H}. The desred probablty s PE { F} = PE { } + PF { } PE { F} = /2+ /2 /4 = 3/4. Eample 2 When rollng two far dce, suppose the frst de s 3, what s the probablty the sum of the two dce s 7? Let E be the event that the sum of the two dce s 7, E = {(, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, )}, and F be the event that the frst de s 3, F= {(3, ), (3, 2), (3, 3), (3, 4), (3, 5), (3, 6)}. Then, PE { F} P{(3,4)} PE { F} = = PF { } P{(3, ), (3, 2), (3, 3), (3, 4), (3, 5), (3, 6)} /36 = =. 6/36 6 Fndng Probablty by Condtonng Suppose that we know the probablty of event B once event A s realzed (or not). We also know P{A}. That s, we know P{B A}, and P{B A c } and P{A}. What s P{B}? 3
Note that B = (A B) (A c B) P{B} = P{A B} + P{A c B}. Therefore, P{B} = P{B A}P{A} + P{B A c }P{A c } = P{B A}P{A} + P{B A c }( P{A}). Here we are fndng P{B} by condtonng on A. In general, f the realzaton of B depends on a partton A of Ω, A A2 A =Ω, A A =, j, n j n P{ B} = P{ B A} P{ A}. = Bayes Formula Ths follows from condtonal probabltes. For two events, PA { B} PB { APA } { } PAB { } = =. c c PB { } PB { APA } { } + PB { A} PA { } Wth a partton A, Eample 3 PA { j B} PB { Aj} PA { j} PA { j B} = =. n PB { } P{ B A} P{ A} Consder two urns. The frst urn contans three whte and seven black balls, and the second contans fve whte and fve black balls. We flp a con and then draw a ball from the frst urn or the second urn dependng on whether the outcome was heads or tals. What s the probablty that a whte ball s selected? = 4
P{W} = P{W H}P{H} + P{W T}P{T} = (3/)(/2) + (5/)(/2) = 2/5. What s the probablty that a black ball s selected? P{B} = P{W} = 3/5. What s the probablty that the con has landed heads gven that a whte ball s selected? PW { H} PH { } (3/)(/ 2) 3 From Bayes formula, PH { W} = = =. PW { } 2/5 8 Random Varables Consder a functon that assgns real numbers to events (outcomes) n Ω. Such real-valued functon s a random varable. E.g., when rollng two far dce, defne as the sum of the two dce. Then, s a random varable wth P{ = 2} = P{(,)}=/36, P{ = 3} = P{(, 2), (2, )}=2/36=/8, etc. E.g., the salvage value of a machne, S, s $,5 f the market goes up (wth probablty.4) and $, f the market goes down (wth probablty.6). Then, S s a random varable wth P{S = 5} =.4 and P{S = } =.6. If the random varable can take on a lmted number of values. Then, ths s a dscrete random varable. E.g., the random varable representng the sum of two dce. 5
If the random varable can take on an uncountable number of values. Then, ths s a contnuous random varable. E.g., the random varable H representng heght of an AUB student. If s a dscrete random varable, the functon f () = P{ = } s the probablty mass functon (pmf) of. The functon F () = P{ } = f ( ) s the cumulatve dstrbuton functon (cdf) of. E.g., for the random varable S representng salvage value of a machne above,.6 f s= f s< fs() s =.4 f s= 5, FS() s =.6 f s< 5. othewse f s 5 For a contnuous random varable,, the cdf s defned based on a functon f () called the densty functon, where P { } = F( ) = f ( tdt ). Fact. For a dscrete random varable f( ) =. For a contnuous random varable, f ( ) =. Independent Random varables Two random varables and Y are sad to be ndependent f P{, Y y} = P{ } P{ Y y} = F ( ) F ( y). Y 6
Epectaton of a random varable The epectaton of a dscrete random varable s. E [ ] = P { = } = f ( ) The epectaton of a contnuous random varable s E[ ] = f ( d ). The epectaton of a random varable s the value obtaned f the underlyng eperence s repeated for a number of tmes whch s large enough and the resultng values are averaged. The epectaton s lnear. That s, for two random varables and Y, E[a + by] = ae[] + be[y]. The epectaton of a functon of random varable, g(), s Eg [ ( )] = g( ) f ( d ). An mportant measure s the n th moment of, n =, 2, n n E[ ] = f ( d ) Measures of varablty The varance of a random varable s ( ) 2 2 2 Var[ ] E[( E[ ]) ] E[ ] E[ ] = =. The standard devaton of a random varable s σ = Var[ ]. 7
The coeffcent of varaton of s CV[] = σ /E[]. The varance (standard devaton) measures the spread of the random varable around the epectaton. The coeffcent of varaton s useful when comparng varablty of dfferent alternatves. Note that Var[a+b] =a 2 Var[], for any real number a and random varable a. Jont dstrbuton The jont dstrbuton functon of two random varables s F, Y(, y) = P{, Y y}. If and Y are dscrete random varables then, F (, y) = P{ =, Y = j} = f (, j), Y, Y,, j y, j y where f,y (.) s the jont pmf of and Y. If and Y are contnuous random varables then, y F (, y ) f (, y ) ddy, = Y, Y, where f,y (.) s the jont pdf of and Y. Fact. F, (, y) = F ( ) F ( y) f and only f (ff) and Y are ndependent. Y Y 8
Covarance The covarance measures the dependence of two random varables. For two random varables and Y, where, σ Y = Cov[, Y ] = E[( E[ ])( Y E[ Y ])] = E[ Y ] E[ ] E[ Y ], E [ Y ] yf (, y ) ddy, = Y, If σ > (<), and Y are sad to be postvely (negatvely) correlated. σ y = ff and Y are ndependent. Propertes of covarance Cov[, ] = Var[ ], Cov[ Y, ] = Cov[ Y, ], Cov[ a, Y ] = acov[ Y, ], Cov[, Y + Z] = Cov[, Y] + Cov[, Z], Cov[ Y, ] = σ σ σ. Y Y σ Y The coeffcent of correlaton s defned as ρ Y =. σ σ Note that ρ Y Note that Var[ + Y] = Var[ ] + 2Cov[, Y] + Var[ Y]. If and Y are ndependent, Var[+Y] = Var[] + Var[Y]. Y 9
The Bernoull Random Varable Suppose an eperment can result n success wth probablty p and falure wth probablty (w.p.) p. We defne a Bernoull random varable as = f the eperment outcome s a success and =, otherwse. The pmf of s p f = f ( ) = P{ = } = p f =. The epected value of s E[] = ( p) + (p) = p. The second moment of s E[ 2 ] = 2 ( p) + 2 (p) = p. The varance of s Var[] = E[ 2 ] (E[]) 2 = p p 2 = p( p). The Bnomal Random Varable Consder n ndependent trals, each of whch can results n a success w.p. p and falure w.p. p. We defne a Bnomal random varable,, as the number of successes n the n trals. The pmf of s defned as n n f () = P{ = } = p ( p), =,, n n n! where =. ( n )!!
.3 f ().2. Note that 2 3 4 5 n n n n f () = p ( p) =, =,, n. = = Fact. Let =, =,, n, f the th tral results n success and =, otherwse. Then n =. = Note that are ndependent and dentcally dstrbuted (d) Bernoull random varable wth parameter p. Therefore, n E [ ] = E [ ] = np, Var [ ] = Var [ ] = np( p). Eample 4 = = A far con s flpped 5 tmes. What s the probablty that two heads are obtaned? The number of heads,, s a bnomal random varable wth parameters n = 5 and p =.5. Then, the desred probablty s P{ = 2} = [5!/(2! 3!)] (.5) 2 (.5) 3 =.33. n
The Geometrc Random Varable Suppose ndependent trals, each havng a probablty p of beng a success, are performed. We defne the geometrc random varable (rv) as the number of trals untl the frst success occurs. The pmf of s defned as f = P = = p p = () { } ( ),, 2,.5 f ()..5 5 5 2 Note that f () defnes a pmf snce f ( ) = p ( p) = p ( p) = p/[ ( p)] =. = = = Let q = p. The frst two moments and varance of are [ ], E = q p= = p 2 p 2 2 [ ] = =, 2 = p E p q p = E E = p 2 2 Var[ ] [ ] ( [ ]). 2 2
Eample 5. When rollng a de repettvely, what s the probablty that the frst 6 appears on the sth roll? Let be the number of rolls untl a 6 appears. Then, s a geometrc rv wth parameter p = /6, and the desred probablty s P{ = 6} = (5/6) 5 (/6) =.667. What s the epected number of rolls untl a 6 appears? E[] = /p = 6. The Posson Random Varable A rv, takng on values,,, s sad to be a Posson random varable wth parameter λ > f λ λ f () = P{ = } = e, =,,!.5 f ()..5 2 4 6 8 f () defnes a pmf snce λ () = = ( )( ) =. λ λ λ f e e e = =! The Posson rv s a good model for demand, arrvals, and certan rare events. 3
The frst two moments and the varance of are E [ ] = λ, E λ λ 2 2 [ ] = +, Var E E 2 2 [ ] = [ ] ( [ ]) =. λ Let and 2 be two ndependent Posson rv s wth means λ and λ 2. Then, Z = + 2 s a Posson rv wth mean λ + λ 2. Eample 6. The monthly demand for a certan arplane spare part of Fly Hgh Arlnes (FHA) fleet at Berut arport s estmated to be a Posson random varable wth mean.5. Suppose that FHA wll stock one spare part at the begnnng of March. Once the part s used, a new part s ordered. The delvery lead tme for a part s 2 months. What s the probablty that the spare part wll be used durng March? Let be the demand for the spare part. The desred probablty s P{ } = e λ = e.5 =.393. What s the probablty that FHA wll face a shortage on ths part n March? The desred probablty s P{ > } = P{ = } P{ = } = e.5.5e.5 =.9. The Unform Random Varable A rv that s equally lke to be near any pont of an nterval (a, b) s sad to have a unform dstrbuton. 4
The pdf of s, f a< < b f ( ) = b a, otherwse Note that f () defnes a pdf snce b a b f ( ) = d. b a = a.3 f ( ).2. 2 3 4 5 6 The cdf of s, f < a a F( ) = f ( tdt ) =, f a b b a, otherwse The frst two moments of are b b 2 2 b a b a + E [ ] = f ( d ) = d= =, b a 2( b a) 2 a a b b 2 3 3 2 2 2 2 b a a + ab+ b [ ] = ( ) = = =. b a 3( b a) 3 a a E f d d The varance of s E[ 2 ] (E[]) 2 = (b a) 2 / 2. 5
The Eponental Random Varable An eponental rv wth parameter λ s a rv whose pdf s f λe λ, f ( ) =, othewse.5 f ( )..5 5 5 2 Note that f () defnes a pdf snce λ λ f ( ) = λe d= e =. The eponental rv s a good model for tme between arrvals or tme to falure of certan equpments. The cdf of s λt λ λ λ F ( ) = f ( t) dt = e dt = e = e,. A useful property of the eponental dstrbuton s that P{ > } = e λ. 6
The frst two moments and the varance of are E [ ] =, λ 2 E = λ 2 [ ], 2 Var = E E = λ 2 2 [ ] [ ] ( [ ]). 2 Preposton. The eponental dstrbuton has the memoryless property. I.e., P { > t+ u > t} = P { > u}. Proof. P { > t+ u, > t} P { > t+ u} P { > t+ u > t} = = P { > t} P { > t} λ ( t+ u) e λu = = e = P{ > u}. λt e The memoryless property allows developng tractable analytcal models wth the eponental dstrbuton. It makes the eponental dstrbuton very popular n modelng. Preposton. Let and 2 be two ndependent eponental random varables wth parameters λ `and λ 2. Let = mn(, 2 ). Then, s an eponental random varable wth parameter λ `+ λ 2. Proof. P{ > } = P{ >, 2 > } = P{ > }P{ 2 > } λ λ2 ( λ+ λ2) = e e = e. 7
Eample 7. The amount of tme one spends n the bank s eponentally dstrbuted wth mean mnutes. A customer arrves at : PM. What s the probablty that the customer wll be n the bank at :5 PM? Let be the tme the customers spends n the bank. Then, s eponentally dstrbuted wth parameter λ = /. The desred probablty s P{ > 5} = e 5λ = e 5/ =.223. It s now :2 PM and the customer s stll n the bank? What s the probablty that the customer wll be n the bank at :35 PM?.223 (by the memoryless property). The Normal Random Varable We say that a random varable s a normal rv wth parameters μ and σ > f t has the followng pdf: 2 2 ( μ) /(2 σ ) e f ( ) =, (, ). 2πσ.5 f ( )..5 5 5 5 8
Note that f () defnes a pdf. Wth a change of varable z = ( μ)/σ and usng the fact that z 2 /2 2 2 e e dz = 2 π, ( μ) /(2 σ ) 2 z /2 f ( ) d = d = e dz =. 2πσ 2 π The normal rv s a good model for quanttes that can be seen as sums or averages of a large number of rv s. The cdf of, F( ) = f( t) dt, has no closed-form. The frst two moments of and varance of are E[ ] = μ, E 2 2 2 [ ], Var = σ + μ = σ 2 [ ]. Fact. If s a normal rv, then Z = ( μ)/σ s a standard normal r.v. wth parameters and. Proof. Note that μ + σ z 2 2 ( t μ ) /(2 σ ) μ e P{ Z < z} = P{ < z} = P{ < μ+ σz} = dt. σ 2πσ Let u = (t μ)/σ, then z u 2 /2 e P{ Z < z} = du, whch s the cdf of 2π the standard normal. Ths fact mples that = μ + σ Z. 9
The cdf of, F (), s evaluates through the cdf of Z, F Z (z), whch s often tabulated, μ μ P { < } = PZ { < } F( ) = FZ σ σ. Proposton If and 2 are two ndependent normal rvs wth means μ and varances σ 2, =,2, then Z = + 2 s normal wth mean μ + μ 2 and varance σ 2 + σ 2 2. Theorem (central lmt theorem). If, =, 2,, n, are d rv s wth mean μ and varance σ 2. Then, for n large enough, n = s normally dstrbuted wth mean nμ and varance nσ 2. Eample 8. The heght of an AUB male student s a normal rv wth mean 7 cm and standard devaton 8 cm. What s the probablty that the heght of an AUB student s less than 8 cm? Let be the heght of the student. Then, the desred probablty s P{ < 8} = P{Z < (8 7)/8} = P{Z <.25} =.894. 2