Solutions to Homework 2 - Probability Review

Size: px

Start display at page:

Download "Solutions to Homework 2 - Probability Review"

Molly Walsh
5 years ago
Views:

1 Solutios to Homework 2 - Probability Review Beroulli, biomial, Poisso ad ormal distributios. A Biomial distributio. Sice X is a biomial RV with parameters, p), it ca be writte as X = B i ) where B,..., B are i.i.d. Beroulli RVs with parameter p i.e., P B i = ) = p). From the liearity of the epectatio operator, we have from ) [ ] E [X ] = E B i = E [B i ] = E [B ]. i= The epectatio of the Beroulli RVs is E [B ] = P B = ) + P B = ) = p. Hece, it follows that i= i= E [X ] = E [B ] = p. Now for the variace, sice B,..., B are idepet ad hece ucorrelated RVs we ca write [ ] var [X ] = var B i = var [B i ] = var [B ]. i= To calculate var [B ] = E [ B E [B ]) 2] = E [ B 2 ] E [B ] 2 we should fid the secod momet E [ B 2 ] we already kow E [B ] 2 = p 2 ). But this is straightforward because for Beroulli B, it holds that B = B 2. Hece E [ B 2 ] = p ad the sought variace of X is i= var [X ] = var [B ] = p p 2 ) = p p). The followig Matlab fuctio calculates the pmf of a biomial RV. % This fuctio returs a vector the same size as u where etries % are the biomial pmf with parameters ad p, calculated % at each elemet of u. This tries to mimic Matlab s % built-i fuctio f= bio,u,,p) % For how Matlab itself calculates the biomial, see bio.m fuctio f=my_biomial_pmfu,,p) f=zerossizeu)); %iitializatio of f for i=:legthu) if ui)>=) && ui)<=) % sice support of biomial,p) is,,.., fi)=choosek,ui))*pˆui)*-p)ˆ-ui)); % pmf epressio of biomial RV with parameters,p) Usig the fuctio my_biomial_pmf the followig mai script plots the required pmfs ad s, for fi E [X ] = 5 ad = 6,, 2, 5. Notice that for discrete RVs, we ca use the commad stem for plottig the pmf ad stairs for the. clear all; close all; clc; _vector=[6,,2,5];

2 i=; for =_vector p=5/; figure) % pmfs subplot2,2,i); % stem:, bio,:,,p),. ); % cheat lie! stem:,my_biomial_pmf:,,p),. ); title[ =,um2str)]); label ); ylabel pmf ); grid o; ais[,5,,.5]); figure2) % s subplot2,2,i); % stairs:, bio,:,,p),. ); % cheat lie! % stairs:,my_biomial_:,,p),. ); % see my_biomial_ below stairs:,cumsummy_biomial_pmf:,,p)), LieWidth,); % with cumsum title[ =,um2str)]); label ); ylabel ); grid o; ais[,5,,]); i=i+; I case you prefer a separate fuctio to calculate the biomial : % fuctio takes vector u, ad scalars ad p, % ad returs a vector of the same size as u, where etries % are the biomial with parameters ad p, calculated % at the poits of elemets of u. tryig to mimic Matlab s % built-i fuctio f= bio,u,,p) % This fuctio calls my_biomial_ ad calculates a cumulative sum % over possible values up to ui) for each etry of u. fuctio F=my_biomial_u,,p) F=zerossizeu)); %iitializatio of F for i=:legthu) Fi)=summy_biomial_:ui),,p)); The obtaied pmf ad plots are show i Figs. ad 2. B Biomial ad Poisso distributios. For a Poisso RV X p with parameter λ, we wet through the calculatio of E [X P ] i class check the lecture slides). A differet way of evaluatig the epected value is E [X P ] = kp X p = k) = k e λ λ k = λe λ kλ k. 2) k= Recall the Taylor series epasio e λ = k= Pluggig this result back i 2) yields kλ k k= λk = d dλ E [X P ] = λe λ k= k= k=, ad from the liearity of the differetiatio operator we have ) λ k = d e λ ) = e λ. dλ kλ k k= = λe λ e λ = λ.

3 .5 =6.5 = = = Fig.. Biomial pmf for = 6,, 2, 5 ad p = 5/. Part A). =6 = = = Fig. 2. Biomial for = 6,, 2, 5 ad p = 5/. Part A). The Matlab code to plot the pmf of a Poisso distributio with parameter λ = 5 follows. clear all; close all; clc; figure lambda=5;

4 =:5; % stem, poiss,,lambda),. ); % cheat lie! my_poisso_pmf=ep-lambda)*lambda.ˆ)./factorial); % Note the use of "dot" for elemet-wise operatio % Thus, my_poisso_pmf is ow a vector with the same size as. stem,my_poisso_pmf,. ); title[ Poisso distributio with \lambda =,um2strlambda)]); label ); ylabel pmf ); grid o; ais[,5,,.5]); The obtaied pmf is depicted i Fig Poisso Distributio with λ = Fig. 3. Pmf of the Poisso distributio with λ = 5. Note that the support of a Poisso RV are the oegative itegers. However, the pmf for oly the first 5 poits is show. Note the similarity with Fig. for large. Now we eed to calculate the mea-squared error MSE) betwee the biomial ad Poisso pmfs. The MSE is defied as X, X P ) = = P X = ) P X P = ) ) 2 P XP = ). To umerically evaluate the MSE, the ifiite sum eeds to be trucated by e.g., eglectig probabilities smaller tha 5 2. To idetify those small probabilities, the followig code reveals that we eed ot go beyod = 8 i the MSE summatio ote from Fig. 3 that the Poisso distributio has a decliig tail, which meas that as, P X P = ) ). clear all; close all; clc; =:2; lambda=5; my poisso=ep-lambda)*lambda.ˆ./factorial); test=my poisso<5e-2 % Etries equal to oe idicate small probabilities More precisely, it turs out we should cosider oly the terms for = 2,..., 8 sice P X P = ) ad P X P = ) are also small. Note that by lookig at Fig. 3, you ca also idetify those values for which the probability mass is greater that 5 2. The followig Matlab script evaluates the MSE, ad geerates the plot i Fig. 4. close all; clear all; clc; lambda=5;

5 =2:8; _ide=; _vector=[6 2 5]; my_mse=zeros,4); for =_vector _ide=_ide+; my_poisso_pmf=ep-lambda)*lambda.ˆ)./factorial); my_mse,_ide)=summy_biomial_pmf,,lambda/)... -my_poisso_pmf).ˆ2.*my_poisso_pmf); stem_vector,my_mse, * ); label ); ylabel MSE ); grid o; ais[,5,,.2]); The results reported i Table I show the MSE betwee the biomial ad Poisso distributios for = 6,, 2, 5. As icreases, the MSE falls rapidly to zero, idicatig the distributios become more ad more similar for larger s. It is critical here that for give λ i the Poisso distributio, p = λ/ i the biomial. The decreasig MSE is also apparet from Fig. 4. TABLE I MEAN-SQUARED ERROR MSE) BETWEEN THE PMFS OF A POISSON WITH λ = 5 AND BINOMIALS WITH PARAMETER, λ/) FOR = 6,, 2, 5. THE POISSON PROBABILITIES SMALLER THAN.5 ARE NEGLECTED. MSE MSE Fig. 4. Mea-Square-Error betwee of a Poisso with λ = 5 ad Biomials of, λ/) for = 6,, 2, 5. The probabilities i the Poisso less tha.5 are eglected. As we see, by icreasig, the MSE rapidly vaishes. part B)

6 C Biomial ad Poisso distributios agai. This is a iterestig situatio where we ca aalytically establish what simulatios are suggestig. Specifically, as we did i class we will show here that the pmf of a biomial RV X with parameters, λ/) coverges to the pmf of a Poisso RV with parameter λ, as. Startig from the epressio for the pmf of X we have ) ) λ p X ) = λ )! λ = ) λ. 3) )!! ) From the defiitio of factorials, we ca simplify! )... + ) )! = = )... + ) 4) )! )! ad rewrite 3) after some reorderig of terms to obtai )! λ p X ) = λ ) = )!! )... + ) I order to take the limit as it is useful to recogize that )... + ) lim =, ad lim λ =. ) I additio to the Taylor series e λ = the sequece λ) k k= lim ) ) λ λ )! λ. 5), the fuctio e λ ca be equivaletly defied as the limit of ) λ = e λ. Usig all these results whe takig the limit as i 6) yields the desired result, amely ) ) lim p )... + ) λ λ X ) = lim )! λ = λ e λ. 6)! D Biomial ad ormal distributios. Recall that a biomial RV X with parameters, p) ca be writte as X = i= where B,..., B are i.i.d. Beroulli RVs with parameter p. We also showed that E [B i ] = p ad var [B i ] = p p). From the CLT, for sufficietly large the distributio of i= Z = B i p = X p 7) p p) p p) is approimately stadard ormal. From the properties of ormal RVs, 7) also implies that B i X = p p)z + p is approimately ormal distributed with mea p ad variace p p). I coclusio, for sufficietly large the of X ca be approimated as F X ) = P X ) 2πp p) e u p)2 /2p p) du. The Matlab code to approimate a biomial with a ormal for p =.5 ad =, 2, 5 follows: close all; clear all; clc; _vector=[ 2 5]; p=.5; _ide=; for =_vector _ide=_ide+;

7 mea_ormal=*p; %defiig the mea std_ormal=sqrt*p*-p)); %defiig the stadard deviatio =:; subplot3,,_ide); %plottig graphs stairs,[my_biomial_,,p)),... orm,mea_ormal,variace_ormal) ], LieWidth,2); title[ =,um2str)]); label ); ylabel ); leg Biomial, Normal ); grid o; ais[,5,,]); The resultig plots are depicted i Fig 5. As epected from the CLT, by icreasig the ormal distributio offers a icreasigly accurate approimatio of the biomial distributio. = Biomial Normal =2 Biomial Normal =5 Biomial Normal Fig. 5. Cdf of the biomial RV X for =, 2, 5, ad its approimatio by a ormal. As we ca see, by icreasig the ormal distributio offers a icreasigly accurate approimatio of the biomial distributio. This follows from the CLT. Part D). E Normal ad Poisso approimatios. I provide you with a hit for this part: the Poisso limit theorem also kow as law of rare evets) is about accumulatio of icreasigly improbable evets. I particular, ote that for covergece of the distributio of sum of i.i.d. Beroulli RVs which is a biomial RV) to a Poisso distributio with mea λ, we eeded the success probability i the Beroulli RV to be p = λ/. Accordigly, as this probability goes to zero. O the other had, for the CLT p is fied ad ot ecessarily small. Hece, the CLT ad Poisso limit theorem are addressig basically differet limits. I leave more cotemplatio o this matter to you!

Probability review (week 2) Solutions

Probability review (week 2) Solutios A. Biomial distributio. BERNOULLI, BINOMIAL, POISSON AND NORMAL DISTRIBUTIONS. X is a biomial RV with parameters,p. Let u i be a Beroulli RV with probability of success