Week 2: Probability review Bernoulli, binomial, Poisson, and normal distributions Solutions

Size: px

Start display at page:

Download "Week 2: Probability review Bernoulli, binomial, Poisson, and normal distributions Solutions"

Jody McGee
5 years ago
Views:

1 Wee 2: Probability review Beroulli, biomial, Poisso, ad ormal distributios Solutios A Biomial distributio. To evaluate the mea ad variace of a biomial RV B with parameters, p), we will rely o the relatio betwee the biomial ad the Beroulli. First, let {L i } i=,..., be idepedet Beroulli RVs with probability of success p. The, the expected value of a sigle L i ca be computed as E [L i ] = p + p) = p. ) Ideed, the expected value is the sum of possible outcomes weighted by their probabilities by defiitio. Usig the fact that B is the sum of idepedet ad idetically distributed i.i.d.) Beroulli RVs L i, we ca derive its expected value from ) by liearity: [ ] B = L i E [B ] = E L i = E [L i ] [from the liearity of expectatio] = i= p = p. i= i= i= We ca proceed i a similar way for the variace. First, we compute some secod-order momets of Beroulli RVs. Explicitly, E [ L 2 i ] = 2 p + 2 p) = p, E [L i L j ] = E [L i ] E [L j ] = p p = p 2, where we recall that {L i, L j } are idepedet for i j. The, [ varb ) = E B EB )) 2] = E [ ) B] 2 E [B ]) 2 = E L i = i= j= i= j= E [L i L j ] 2 p 2 = p + )p 2 2 p 2 = p p). L j p) 2 A MATLAB script that computes the values of the biomial is preseted below. % Taes vector ad scalars ad p ad returs a vector of the same size 2 % as, whose etries represet the probability of a biomial RV with 3 % parameters ad p at the poits of the elemets of. 5 fuctio = biomial,, p) 6

2 7 = zerossize)); % Iitialize output 8 9 for i = :legth) % Chec that the elemet of is i the support of the biomial,p). % Otherwise, =. 2 if i) >= ) && i) <= ) 3 i) = choose,i)) * pˆi) * -p)ˆ-i)); ed 5 ed 6 7 ed You could write a similar code replacig choose by the fuctio factorial, but be aware that it is oly accurate for umbers up to 2. This is because MATLAB represets umbers i double precisio which have roughly 5 usable digits. The fuctio choose uses differet approximatios for large. We ca use biomial.m to plot the s ad s requested i part A. Notice we use of stem to show the ad stairs for the, sice these are discrete RV. % Delete all variables ad close figures 2 clear all 3 close all 5 vector = [6,, 2, 5]; 6 7 for i = :legth vector) 8 = vectori); 9 p = 5/; % E[B ] = 5 h = figure); 2 subplot2,2,i); 3 stem:, biomial :,, p), '.'); title[' = ' um2str)]); 5 xlabel''); 6 ylabel''); 7 grid; 8 xlim[,5]); 9 ylim[,.5]); 2 2 h2 = figure2); 22 subplot2,2,i); 23 stairs:, cumsumbiomial :,, p)), 'LieWidth', 2); 2 title[' = ' um2str)]); 25 xlabel''); 26 ylabel''); 27 grid; 28 xlim[,5]); 29 ylim[,]); 3 ed 3 32 %%% Export figure %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 33 seth,'color','w'); 3 export fig'-q', '-pdf', 'HW2 A.pdf', h); 35 seth2,'color','w'); 36 export fig'-q', '-pdf', '-apped', 'HW2 A.pdf', h2); 37 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% The s ad s are show i Figures ad 2. 2

3 B Biomial ad Poisso distributios. The expected value of a Poisso RV P with parameter λ is give by e λ λ ) E [P ] = = λe λ λ = λe λ d ) λ!! dλ! = = = = λe λ d dλ e λ) = λe λ e λ = λ. Notice that i the first equality i the secod lie comes from recogizig that the ifiite sum is the Taylor series of the expoetial. The MATLAB code to plot the Poisso distributio is show below. % Delete all variables ad close figures 2 clear all 3 close all 5 lambda = 5; 6 = :5; 7 8 poisso = exp-lambda) * lambda.ˆ)./ factorial); 9 % Note that we use the "dot" to perform elemetwise operatios ad evaluate % the for poits i at oce. 2 figure); 3 stem, poisso, '.'); title['poisso with \lambda = ' um2strlambda)]); 5 xlabel''); 6 ylabel''); 7 grid; %%% Export figure %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 2 setgcf,'color','w'); 22 export fig'-q', '-pdf', 'HW2 B.pdf'); 23 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% The result ca be foud i Figure 3. Note the similarity with Figures for large. To calculate the MSE, we first chec a few values of the Poisso to fid the cut-off poit for the exercise. Simple trial ad error shows that we eed oly cosiders values for [3, 9]. Hece, we ca use the followig code to evaluate ad plot the MSE. % Delete all variables ad close figures 2 clear all 3 close all 5 lambda = 5; 6 = 3:9; 7 8 poisso pdf = exp-lambda)*lambda.ˆ)./ factorial); 9 vector=[6 2 5]; mse = zeroslegth vector), ); 2 3 for i=:legth vector) = vectori); 5 msei) = sum biomial,, lambda/) - poisso pdf).ˆ2.* poisso pdf ); 6 ed 7 8 figure); 3

4 9 plot vector, mse, 'x', 'LieWidth', 2); 2 title'mse betwee Poisso ad biomial s') 2 xlabel''); 22 ylabel'mse'); 23 grid; dispmse); %%% Export figure %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 3 setgcf,'color','w'); 3 export fig'-q', '-pdf', 'HW2 B2.pdf'); 32 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% The resultig plot is show i Figure. Values are reported i Table. Table : MSE betwee the of a Poisso with λ = 5 ad biomials with parameters, λ/) for = 6,, 2, 5. MSE The table above shows that the MSE betwee the biomial ad Poisso distributios rapidly decreases to zero as icreases, idicatig that the distributios become icreasigly similar for large. C Biomial ad Poisso distributios agai. We start by writig out the of B. Explicitly, P [B = ] =! )!! p p) =! )!! The, otice that we ca rearrage the first two terms to read ) λ λ. ) P [B = ] = λ ) 2) + )! λ ) = λ! ) 2) + ) λ ) Fially, we ca tae the limit as. Recall that the limit of the product is the product of the limit as log as all limits exist, which is the case here: lim P [B = ] = λ! lim ) 2) = λ ) lim! + ) λ ) lim λ ) λ ).

5 The last limit is simply. Also, by defiitio, we have ) λ = e λ. lim Thus, lim P [B = ] = λ! e λ = P [P = ]. D Biomial ad ormal distributios. If Z is a ormal RV with zero mea ad uit variace also ow as a stadard ormal), the from Z = i= Y i µ σ, it holds that Y = i= Y i is also ormally distributed but with mea p ad variace σ 2, where σ 2 = p p) is the variace of a sigle Beroulli Y i. The MATLAB code to display this approximatio is give below. % Delete all variables ad close figures 2 clear all 3 close all 5 vector = [ 2 5]; 6 p =.5; 7 8 for i = :legth vector) 9 = vectori); Y mea = *p; 2 Y var = *p*-p); 3 = :; 5 figure); 6 subplot3,,i); 7 stairs, cumsumbiomial,, p)), 'LieWidth', 2); 8 hold o 9 stairs, orm, Y mea, sqrty var)), 'LieWidth', 2); 2 title['=',um2str)]); 2 xlabel''); 22 ylabel''); 23 leged'biomial', 'Normal'); 2 grid; 25 ed %%% Export figure %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 3 setgcf,'color','w'); 3 export fig'-q', '-pdf', 'HW2 D.pdf'); 32 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% The resultig plot ca be foud i Figure 5. E Normal ad Poisso approximatios. I ll provide you with a hit for this part: The Poisso limit theorem is about coutig a large umber of icreasigly improbable evets. I particular, ote that for the distributio of a sum of i.i.d. Beroulli RVs i.e., a biomial RV) to coverge to a Poiso distributio with mea λ, the probability of success of each Beroulli trial 5

6 must be p = λ/, which goes to zero as. O the other had, p is fixed for the CLT. Hece, the CLT ad the Poisso limit theorem are addressig fudametally differet limits. I leave more cotemplatio o this matter to you! = 6 = = = Figure : Biomial for = 6,, 2, 5 ad p = 5/ part A). 6

7 = 6 = = 2 2 = Figure 2: Biomial for = 6,, 2, 5 ad p = 5/ part A)..8 Poisso with = Figure 3: The of a Poisso RV with λ = 5 part B). Note that the support of the Poisso distributio is the whole o-egative iteger lie. However, we display oly the first 5 poits. 7

8 .8 MSE betwee Poisso ad biomial s.6..2 MSE Figure : MSE betwee the of a Poisso with λ = 5 ad biomials with parameters, λ/) for = 6,, 2, 5 part B)..5 = Biomial Normal =2.5 Biomial Normal =5.5 Biomial Normal Figure 5: Cumulative distributio fuctio of the biomial RV Y for =, 2, 3 ad its approximatio by a ormal part D). 8

Probability review (week 2) Solutions

Probability review (week 2) Solutios A. Biomial distributio. BERNOULLI, BINOMIAL, POISSON AND NORMAL DISTRIBUTIONS. X is a biomial RV with parameters,p. Let u i be a Beroulli RV with probability of success