Discrete Probability Fuctios Daiel B. Rowe, Ph.D. Professor Departmet of Mathematics, Statistics, ad Computer Sciece Copyright 017 by 1
Outlie Discrete RVs, PMFs, CDFs Discrete Expectatios Discrete Momets
Discrete Radom Variables A radom variable is a fuctio from a sample space S ito the real umbers X. Examples: Experimet Toss two die Toss a coi 5 times Apply differet amouts of fertilizer to plats Radom Variable X=sum of umbers X= # heads i 5 tosses X=yield/acre 3
Discrete Probability Mass Fuctios Example: Toss a coi twice. S={HH,HT,TH,TT} X=umber of heads. Sample space outcomes X(s) is a mappig from S to X. Real Numbers s 1 =HH, s =HT, s 3 =TH, s 4 =TT X 1 =, X =1, X 3 =1, X 4 =0 P(X) is a mappig from X to [0,1]. s HH HT TH TT X(s) 1 1 0 x 0 1 P(X=x) 1/4 1/ 1/4 Real Numbers Uit iterval 4
Discrete Cumulative Distributio Fuctios The cdf of a radom variable X, F X (x), is F ( x) P ( X x), for all x. X X Example: Tossig a coi twice x 0 1 P(X=x) 1/4 1/ 1/4 F X 0 / 4 x 0 1/ 4 0 x 1 ( x) 3 / 4 1 x 4 / 4 x 1.00 0.75 0.50 0.5 0.00 F ( x) P ( X x) X Xx 0 1 X 5
Discrete RVs, PMFs, ad CDFs Assume that the discrete radom variable (RV) takes o values x, x, x... 1 3 if the probability of each of value x is P( X x ) the, the probability mass fuctio (PMF) is give by f ( x ) P( X x ) for 1,,3... where θ are ay parameters that the PMF depeds o. 6
Discrete RVs, PMFs, ad CDFs Further, the cumulative distributio fuctio (CDF) is give by i.e. where x k (largest value) x. Additioally, ay PMF must satisfy 1) F( x ) f ( x ) ) ( ) 1. x x F( x ) f ( x )... f ( x ) 0 f( x ) 1 f x x k 7
Discrete Expectatio Give a arbitrary discrete probability mass f(x θ), we wat to compute quatitative populatio summaries of it such as populatio mea, populatio variace, x f ( x ) populatio stadard deviatio, 1 x 1 ( ) f ( x ) 8
Discrete Expectatio These populatio umerical summaries are foud by expectatio E[ g( X ) ] g( x ) f ( x ) The mea is 1. EX ( ) ad the variace is E[( X ) ]. 9
Discrete Expectatio Liearity Property: E[ g ( X ) ] E[ g ( X ) ] 1 1 i.e. E[ g ( X ) g ( X ) ] E[ g ( X ) ] E[ g ( X ) ] 1 1 E[ g ( X ) g ( X ) ] [ g ( x ) g ( x )] f ( x ) 1 1 i i i 1 1. [ g ( x ) f ( x ) g ( x ) f ( x )] 1 g ( x ) f ( x ) g ( x ) f ( x ) 1 1 1 10
Discrete Expectatio Example: x 0 1 P(X=x) 1/4 1/ 1/4 E[ g( X ) ] g( x ) f ( x ) g( X ) 1 X (0)(1/ 4) (1)( / 4) ()(1/ 4) 1 11
Discrete Expectatio Example: x 0 1 P(X=x) 1/4 1/ 1/4 E[ g( X ) ] g( x ) f ( x ) 1 g( X ) ( X ) (0 1) (1/ 4) (1 1) ( / 4) ( 1) (1/ 4) 1/ 1
Discrete Momets For each iteger, the th momet of X, 1, is EX ( ). The th cetral momet of X,, is E[( X ) ] where. 1 EX ( ) 13
Discrete Momet Geeratig Fuctio The momet geeratig fuctio (MGF) of a discrete PMF is tx M ( t) E( e ) X x tx e f ( x). The MGF does ot always exist! Whe the MGF exists, it is uique ad completely determies the distributio. 14
Discrete Momet Geeratig Fuctio If two RVs have the same MGF, they have the same dist. Let X have a PMF M X( t) e f ( x ) 1 tx f( x ) ad Y have PMF gx ( ) Recall: e x x 0! MY( t) e g( y ) 1 ty If X ad Y have the same distributio, the M ( t) M ( t). X Y 15
Discrete Momet Geeratig Fuctio Why is this expectatio importat? If we differetiate the MGF ad set the derivatives t=0, we get the momets of the distributio. E( X ) M X ( t) t t M X () t t0 t0 M () X t t t0 16
Discrete Momet Geeratig Fuctio Deote the th derivative of the MGF evaluated at t=0 as M (0) M ( t) t t0. The, the MGF ca be writte as 1 M (0) M (0) M (0) M ( t) 1 t t t 1!!! E( X ) E( X ) E( X ) M ( t) 1 t t t 1!!! where the X subscript has bee suppressed. 17
Discrete Momet Geeratig Fuctio The MGF ca be writte as M ( t) e f ( x ) 1 1 k0 k0 1 k0 tx f( x ) ( tx ) () t k! k! k k E X ( tx ) k! f( x ) k k 18
Discrete Momet Geeratig Fuctio If we differetiate the MGF w.r.t. t ad evaluate at t=0, k ( tx ) M ( t) f ( x ) t t k! 1 k0 1 k0 f( x ) t ( tx ) k! k ( tx ) ( tx ) ( tx ) f( x) 1...... 1 t 1!!! 1 f x x tx 1 ( )... t=0 EX ( ) 19
Uiform: A radom variable x has a discrete uiform distributio, x ~uiform(n) if P( X x N) 1 N where,,. N 1,,... x1,,3,... N i.e. flippig a coi, rollig a die. 0
Uiform: It ca be show that 1 x P( X x ) N 1 1 N N 1 Note: 1 3... N N( N 1) 1
Uiform: that x P( X x ) 1 N 1 N 1 1 N N 1 1 Note: N( N 1)( N 1) 1 3... N 6
Uiform: ad that M ( t) e P( X x ) X 1 tx N 1 1 N e t e e t N(1 e ) t ( N 1) t. Note: 1 x x... x N 1 x 1 x N 1 3
Uiform: P( X x N) 1 N where,,. N 1,,3,... x1,,3,... N Uiform discrete radom variates ca be geerated by either partitioig the uit iterval ito N bis, geeratig uiform [0,1] umbers ad for each radom umber covert to if it is i the th bi or N=6; x=uidrd(n) 5 4
Uiform: 0.18 Marquette Uiversity P( X x) 1 6 1800 0.16 1600 0.14 1400 0.1 100 0.1 1000 0.08 800 0.06 600 0.04 400 0.0 00 0 N=6; xpop=(1:n); ypop = uidpdf(x,n); bar(xpop,ypop,1) 1 3 4 5 6 0 1 1.5.5 3 3.5 4 4.5 5 5.5 6 N=6;, um=10000; xsamp=uidrd(n,um,1); hist(xsamp,n) 5
Uiform: 1 P( X x) 1 6 1 0.9 0.8 0.7 0.6 0.5 0.4 0 0.1667 0.3333 0.5000 0.6667 0.8333 1.0000 0.9 0.8 0.7 0.6 0.5 0.4 0 0.1675 0.3330 0.4958 0.6716 0.8361 1.0000 0.3 0.3 0. 0. 0.1 1 1.5.5 3 3.5 4 4.5 5 5.5 6 ducdf =uidcdf(x,6) stairs(x, ducdf,'liewidth',) 0.1 1 1.5.5 3 3.5 4 4.5 5 5.5 6 [F,x]=ecdf(xsamp); stairs(x,f,'liewidth',) 6
Uiform: P( X x) 1 6 uidstat(n) True Simulated mea(xsamp) var(xsamp) 3.5 3.4960.9167.8987 7
Beroulli: A radom variable x has a discrete Beroulli distributio, x~beroulli(p) if 1 ( ) x x P X x p p (1 p) where,,. 0 p 1 x 0,1 x = the umber of successes out of 1 trial. p = the probability of success o the trial. 8
Beroulli: It ca be show that 1 x P( X x ) (0)(1 p) (1) p p 9
Beroulli: that x P( X x ) 1 0 (1 ) 1 p(1 p) p p p p 30
Beroulli: ad that M ( t) e P( X x ) X 1 tx t(0) t(1) e q e p t ( q pe ). q (1 p) 31
Beroulli: 1 ( ) x x P X x p p (1 p) 0 p 1 x 0,1 where,,. Beroulli discrete radom variates ca be geerated by either partitioig the uit iterval ito bis, geeratig uiform [0,1] umbers ad for each radom umber covert to -1 if it is i the th bi or p=1/; =1; x=biord(1,p) 1 3
Beroulli: 1 ( ) x x P X x p p (1 p) =1; p=1/; um=10^4; x=biord (,p,um,1); mea(x) var(x) hist(x,) 6000 5000 4000 3000 True Simulated 0.5 0.4971 0.5 0.500 Ca also fid ad plot ECDF. 000 1000 0 0 0.1 0. 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 33
Biomial: For the Beroulli distributio, x 1 x. P( X x p) p (1 p) Let X 1 ~Beroulli(p) ad X ~Beroulli(p), Defie X=X 1 +X. i.e. X=total umber of successes The distributio of X is biomial(,p). This ca be repeated to get biomial(,p). 34
Biomial: A radom variable x has a discrete biomial distributio, x~biomial(,p) if! x P( X x, p) p (1 p) x!( x)! (x successes) where, 0 p 1, 1,,3,..., x 0,1...,. = umber of trials or times we repeat the experimet. x = the umber of successes out of trials. x P(x successes ad -x failures) p = the probability of success o a idividual trial. 35
Biomial: It ca be show that x0 1 x P( X x )! x x p (1 p) x!( x)! p x 36
Biomial: that x0 x P( X x ) 1! x!( x)! x x ( x p) p (1 p) p(1 p) 37
Biomial: ad that M ( t) e P( X x ) X x0 1 tx! e p (1 p) x!( x)! t ( q pe ). tx x x q (1 p) 38
Biomial: =5; p=1/;um=10^4; x=biord (,p,um,1); mea(x) var(x) hist(x,6) True Simulated.5.498 1.5 1.455! x P( X x, p) p (1 p) x!( x)! 3500 3000 500 000 1500 1000 500 x Ca also fid ad plot ECDF. 0 0 0.5 1 1.5.5 3 3.5 4 4.5 5 39
Poisso: A radom variable x has a discrete Poisso distributio, x~poisso(λ) if P( X x ) x! x e where,,. 0 x 0,1... λ is called the itesity parameter 40
Poisso: It ca be show that 1 x P( X x ) x0 x e x x! 41
Poisso: that x P( X x ) 1 x0 ( x ) x e x! 4
Poisso: ad that M ( t) e P( X x ) X 1 tx x0 e e tx x e x! t ( e 1). 43
Poisso: lam=5;um=10^4; x=poissrd(lam,um,1); mea(x) var(x) hist(x,15) True Simulated 5 5.0181 5 5.1069 P( X x ) x! x e 1800 1600 1400 100 1000 800 600 400 00 Ca also fid ad plot ECDF. 0 0 5 10 15 44
Geometric: A radom variable x has a discrete geometric distributio, x~geometric(p) if P( X x p) p(1 p) x 1 where,,. 0 p 1 x 1,... x = the trial at which first successes occurs. p = the probability of success o the trial. 45
Geometric: It ca be show that 1 x1 x P( X x ) xp(1 p) 1 p x1 q (1 p) 46
Geometric: that x P( X x ) 1 1 ( x ) p(1 p) p x1 x1 q p q (1 p) 47
Geometric: ad that M ( t) e P( X x ) X 1 x1 tx tx e p(1 p) t pe 1 qe t x1. q (1 p) 48
Geometric: P( X x p) p(1 p) x 1 p=1/;um=10^4; x= geord(p,um,1)+1; mea(x) var(x) hist(x,1) True Simulated 1.9994.0358 8000 7000 6000 5000 4000 3000 000 1000 0 Ca also fid ad plot ECDF. 0 4 6 8 10 1 14 49
Homework 3: 1) Derive the MGF for the discrete uiform, Beroulli, biomial, Poisso, ad geometric distributios. ) Determie the first two momets from the MGF of the discrete uiform, Beroulli, biomial, Poisso, ad geometric distributios. 50
Homework 3: 3) Geerate 10 6 observatios from the discrete uiform, Beroulli, biomial, Poisso, ad geometric distributios. Use the parameter values I used. Compute ad plot a icremetal sample mea ad variace. Whe do you thik you get covergece? 51