Review of Proility Distriutions CS1538: Introduction to Simultions
Some Well-Known Proility Distriutions Bernoulli Binomil Geometric Negtive Binomil Poisson Uniform Exponentil Gmm Erlng Gussin/Norml Relevnce to simultions: Need to use distriutions tht re pproprite for our prolem The closer the chosen distriution mtches the distriution in relity, the more ccurte our model Why not lwys mke userdefined distriution specific to our prolem? 2
Discrete Distriutions Bernoulli Binomil Geometric Negtive Binomil Poisson 4
The Bernoulli Distriution Bernoulli Tril: n experiment with only two possile outcomes: Success or Filure The proility of success is p (where 0 p 1) Let q e the proility of filure. Wht is q in terms of p? Exmples? 5
The Bernoulli Distriution Let B e rndom vrile over the outcome of the experiment: B = 1 for success; B = 0 for filure Expecttion of B E[B] = 1*p + 0*(1-p) = p Vrince of B Vr(B) = E[B 2 ]- (E[B]) 2 = (1 2 p+0 2 (1-p)) p 2 = p p 2 = p(1 p) 6
Binomil Distriution Experiment: Repet Bernoulli Tril for n times Good for determining the proility of getting k defective items in tch size of n Let rndom vrile X e the numer of successes Note: the order doesn t mtter For exmple: suppose we toss ised coin 3 times (with heds=success). X(HHT)=2; X(THH)=2; X(HTH)=2 7
Binomil Distriution The proility mss function for X: p( x) n p x 0, x q nx, x 0,1,, n otherwise E[X] = E[B 1 + B 2 + B n ] = S E[B i ] = np Vr(X) = Vr(B 1 + B 2 + B n ) = S Vr(B i ) = np(1-p) Note: n k = n! k! n k! 8
Geometric Distriution Keep on repeting Bernoulli trils until successful Let r.v. X e the numer of trils until the first success The proility mss function for X: p( x) x q 0, 1 p, x 1,2, otherwise Wht is q? 9
Geometric Distriution E[X] = S xq (x-1) p = p S xq (x-1) = 1/p We know tht S xq (x-1) dq = S q x where x = 1,2, We lso know tht S q y for y = 0,1,2 = 1/(1-q) So S xq (x-1) dq = 1/(1-q) 1 = q/1-q d/dq q/(1-q) = q/(1-q) 2 + 1/(1-q) = 1/(1-q) 2 = 1/p 2 So p S xq (x-1) = d/dq S xq (x-1) dq = p (1/p 2 ) = 1/p Vr(X) = q/p 2 10
Exmple Question Suppose product hs p% chnce of filure. Wht s the chnce tht it will still e working fter 3 uses? 11
Exmple Question Wht s the chnce tht the product is still working fter 7 uses, given tht it works fter 3 uses? 13
Geometric Distriution Properties Pr(X > t) = q t Memoryless Pr(X > s+t X > s) = Pr(X > t) 15
Negtive Binomil Distriution Keep on running Bernoulli Trils until we get k successes. Like Geometric, ut now k successes insted of 1 success Let r.v. X e the totl numer of trils The proility mss function for X: p( x) x k 0, 1 xk k q p, x 1 otherwise k, k 1, k 2 Expecttion: E[X] = k/p Vr(X) = kq/p 2 16
Poisson Distriution Computes the proility of the numer of events tht my occur in period, given the rte of occurrence in tht period Good for modeling rrivls Proility mss function: p( x) x e, x 0,1, x! 0, otherwise Where is fixed vlue tht must e positive. It represents the verge rte of the event of interest occurring. 17
Exmple The Prussin Cvlry/Horse study [Bortkiewicz, 1898; cf. Lrsen&Mrx] 10 cvlry corps monitored over 20 yers. X = numer of ftlity due to horse kicks x = # of deths Oserved # of corps-yers in which x ftlity occurred 0 109 1 65 2 22 3 3 4 1 Totl 200 Expected # of corps-yer using Poisson 18
Poisson Distriution Cumultive distriution function: F( x) Note tht when x, F(x) 1 i=0 E X = α Vr(X) = x i e i0 i! e α α i i! = e α i=0 α i i! = e α e α = 1 20
Exmple Suppose tht I see students t the rte of five per office hours. Wht is the chnce tht I ll see 4 students t office hours tody? Wht is the chnce tht I ll see less thn 3 students t office hours tody? Wht is the chnce tht I ll see t lest 3 students t office hours tody? 24
Poisson: Reltionship to Binomil Theorem: Let e fixed numer nd n e n ritrry positive integer. For ech nonnegtive integer x, where p = /n lim n n x px (1 p) n x = e α α x x! This mkes it esier to clculte the inomil distriution, especilly for lrge n s 26
Poisson Arrivl Process Recll represents verge rte of occurrence How do we represent time more explicitly? Let N(t) e rndom vrile tht represents the numer (positive integer) of events tht occurred in time [0, t] We wnt to know Pr(N(t) = n) If we gurntee few properties, we cn use the Poisson Distriution 27
Poisson Arrivl Process Assumptions: Arrivls occur one t time Arrivl rte (l) does not chnge over time We ll see how to relx this The numer of rrivls in given period re independent of ech other We cn now rewrite the Poisson Distriution: = lt How does this ffect the expecttion nd vrince? 28 P[ N( t) e n] 0, lt n ( lt), n 0,1, n! otherwise
Properties of the Poisson Arrivl Process Rndom Splitting Consider Poisson Process N(t) with rte l. Assume tht rrivls cn e divided into two groups, A nd B with proility p nd (1-p), respectively N A is Poisson Process with rte lp nd N B is Poisson Process with rte l(1-p) N(t) = N A (t) + N B (t) 29
Properties of the Poisson Arrivl Process Pooled Process Consider two Poisson Processes N 1 (t) nd N 2 (t), with rtes l 1 nd l 2 The sum of the two processes is lso Poisson Process; it hs rte of l 1 + l 2 Pooling cn e used in situtions where multiple rrivl processes feed single queue 30
Discrete Proility Summry Binomil Distriution Geometric Distriution Negtive Binomil Distriution These descrie experiments sed on repeted Bernoulli Trils Poisson Distriution Doesn t hve n esy to descrie underlying structure, ut seems to e good fit for mny rel dt sets
Continuous Rndom Vriles Rndom vrile X is continuous if its rnge spce is n intervl or collection of intervls There exists non-negtive function f(x), clled the proility density function, such tht for ny set of rel numers, f(x) >= 0 for ll x in the rnge spce (i.e., the totl re under f(x) is 1) f(x) = 0 for ll x not in the rnge spce Note tht f(x) does not give the proility of X = x rngespce f ( x) dx 1 Unlike the pmf for discrete rndom vriles 32
Continuous Rndom Vriles The proility tht X lies in given intervl [,] is P ( X ) f ( x) dx k "re under the curve" Note tht for continuous rndom vriles, Pr(X = x) = 0 for ny x Consider the proility of x within (very smll) rnge The cumultive distriution function (cdf), F(x) is now the integrl from - to x or F ( x) f ( t) dt x This gives us the proility up to x 33
Continuous Rndom Vriles Expected Vlue for continuous rndom vrile Similr to the discrete cse, except tht we integrte insted of summing E [ X ] xf ( x) dx Vrince: sme formultion s its discrete counterprt (though clculting E[X 2 ] will involve integrls gin). Vr(X) = E[X 2 ] (E[X]) 2
Uniform Distriution over rnge [,] Proility density function: 1 if x f ( x) 0 otherwise Cumultive distriution function: x x 1 F( x) f ( y) dy dy if x Wht out F(x) when x < or x >? x 35
Uniform Distriution over rnge [,] 36 Expecttion Wht does the expected vlue of discrete uniform distriution look like? Vrince 2 ) 2( ) )( ( ) 2( ) 2( ) ( 2 2 2 x dx x X E 12 ) ( ) 12( ) ( ) 3( ) 4( 4 ) ( ) 3( 4 ) ( ) 3( ]) [ ( ]) [ ( ] [ ) ( 2 2 3 3 2 3 3 2 3 2 2 2 2 x X E dx x X E X E X Vr
Norml Distriution Proility density function: f x = 1 σ 2π exp 1 2 x μ σ 2, - < x < where we supply the men nd vrince: m: men s: squre-root of vrince The norml distriution is lso denoted s: N(m,s 2 ) Some of its properties: lim mx( x f ( m x) f ( x) f ( x)) 0 f ( m f ( m) x) lim x f ( x) 37
Norml Distriution F(x) = P(X x) = Doesn t hve closed form. Cn use numericl methods, ut wnt to void evluting integrls for ech pir (m,s 2 ) Trnsform to stndrd norml distriution Let z = (t-m)/s then we cn rewrite the ove s: F(x) = P(X x) = P(Z (x-m)/s) = ( x m )/ s x 1 exp s 2 1 exp 2 1 2 z 2 1 2 dz t m s 2 dt 38
CDF tle for N(0,1) There re few vrints. On the wesite: A typicl tle from stts ook lets you specify z to 2 significnt digits: (z=column 1 +row 1 )
Exmple (from textook 5.21) Suppose we hve norml distriution such tht: X is r.v. from N(50, 9) Wht is the chnce tht X 56? We wish to compute F(56) (i.e., P(X 56)).
Exponentil Distriution Models Interrrivl times Service times Lifetime of component tht fils instntneously Prmeter λ indictes rte
Exponentil Distriution Proility density function: f ( x) le 0, lx, x 0 otherwise where l >0 is prmeter tht we supply Since the exponent is negtive, the pdf will decrese s x increses
Exponentil Distriution 44 2 0 1 ) ( 1 ] [ 0, 1 0 0, ) ( l l l l l X Vr X E x e dt e x x F x x t
Exponentil Distriution: l l represents rte: numer of occurrences per time unit x: mt. of time it took for some occurrence to tke plce F(x): pro. tht the event hppened during intervl [0,x] 1-F(x): pro. tht the event doesn t hppen until fter x Exmple 5.17 Suppose lifespn of type of lmp is exponentilly distriuted with filure rte l = 1/3000 hrs. Wht s the chnce prticulr lmp will et the verge? Hs reltionship to the Poisson Arrivl Process
Exponentil Distriution Like the geometric distriution, the exponentil distriution is memoryless P(X > s+t X > s) = P(X > t) The proof is similr to how we showed tht the geometric distriution is memoryless 47
Reltionship etween Poisson Distriution nd Exponentil Distriution (1/3) Events occur t rte of l Poisson: chnce of n events tking plce is given y: Pr[N(t)=n] = e -lt (lt) n /n! Let E e the first occurrence of n event since the strt of the clock t 0; let t represent the time when E hppens Wnt to find: distriution for t Look for CDF F(t) the proility tht E hppened some time within the intervl [0,t] Then we cn tke the derivtive of F(t) to find f(t) Wnt to show: F(t) fits Exponentil
Reltionship etween Poisson Distriution nd Exponentil Distriution (2/3) F(t) = The chnce tht E occurs in some intervl [0,t] = 1 Pr(E occurred fter t) = 1 Pr(nothing took plce in [0,t]) Nothing took plce in [0,t] reltes counting discrete event occurrences with mesuring continuous time durtion Pr(nothing took plce in [0,t]) = Pr[N(t)=0] F(t) = 1 Pr(E occurred fter t) = 1 Pr[N(t) = 0] = 1 e lt which is the CDF for exponentil distriution So the durtion etween the 0 th nd 1 st event of Poisson rrivl process follows n exponentil distriution
Reltionship etween Poisson Distriution nd Exponentil Distriution (3/3) More generlly, for ny two events E i nd E j tht follows the Poisson rrivl process, the durtion etween them lso follows the exponentil distriution Let i e the time when E i occurs nd j e the time when E j occurs We cn use the sme nlysis s efore (just lign i with 0 nd j with t in the previous cse) This is ecuse Poisson rrivl process ssumes constnt rte Also mkes sense exponentil distriution is memoryless
Exercises with Exponentil Rte: 0.5 rrivls per minute F(t) = 1-e -0.5t = proility of time elpse etween rrivls Wht s the proility tht the next customer shows up efore 30 seconds hve pssed? within 1 minute? within 2 minutes? Wht s the chnce tht customer shows up etween 5 nd 7 minutes?
Reltionship etween Gmm Distriution nd Poisson Distriution Suppose some event occurs over time intervl of length x t the verge rte of l per unit time. How long would it tke for r events to hppen? Divide up x into smll independent suintervls such tht the chnce of more thn one event hppening in the suintervl is negligily smll Let W e r.v. counting the # of occurrences of the event in the totl durtion [0,x]. W is Poisson r.v. w/ prms lx F X (x) = Pr(X x) = 1-Pr(X>x) = 1-F W (r-1) =1 r 1 k=0 e λx (λx)k f X (x) = F X (x) = λ r r 1! xr 1 e λx k!
The Gmm function: If r cn e non-integer, then we need to replce (r-1)! with continuous function of r. We ll cll this function Gmm of r: G(r) For ny rel numer r > 0, the gmm function G(r) is: G(1) = 1 G(1/2) = sqrt() Γ β = G(r+1) = r G(r) for ny positive rel r 0 x β 1 e x dx G(r+1) = r! if r is nonnegtive integer n+r 1 = G(n+r) n G(n+1)G(r) G(r)G(s) G(r+s) = 1 0 u r 1 (1 u) s 1 du
Gmm Distriution (generl form) Useful for when witing times etween events is relevnt (e.g. witing time etween Poisson events) Let X e rndom vrile such tht f X x = βθ G(β) (βθx)β 1 e βθx, x > 0, β > 0 0, otherwise β is referred to s shpe prmeter θ is referred to s scle prmeter E[X] = 0 x f x dx = 1 Vr(X) = 1 βθ 2 x CDF: F X x = 0 f t dt = 1 x f t dt θ
Erlng Distriution When β is n ritrry positive integer, the Gmm Distriution is clled the Erlng Distriution of order k (k= β) Cn simplify the CDF to: F X = 1 k 1 e kθx (kθx) i i=0 x > 0 i! 0 x 0 This is the sum of Poisson terms with men α = kθx k is the numer of events θ is the rte of the collection of events λ = kθ is the rte of one event (events per unit time) 56
Erlng Exmple Suppose node in network does not trnsmit until it hs ccumulted 5 messges in its uffer. Suppose messges rrive independently nd re exponentilly distriuted with men of 100 ms etween messges. Suppose trnsmission ws just mde; wht s the proility tht more thn 552 ms will pss efore the next trnsmission? 57