Charles Bocelet, Probability, Statistics, ad Radom Sigals," Oxford Uiversity Press, 016. ISBN: 978-0-19-00051-0 Chater 6: BINOMIAL PROBABILITIES Sectios 6.1 Basics of the Biomial Distributio 6. Comutig Biomial Probabilities 6.3 Momets of the Biomial Distributio 6.4 Sums of Ideedet Biomial Radom Variables 6.5 Distributios Related to the Biomial 6.5.1 Coectios Betwee Biomial ad Hyergeometric Probabilities 6.5. Multiomial Probabilities 6.5.3 The Negative Biomial Distributio 6.5.4 The Poisso Distributio 6.6 Parameter Estimatio for Biomial ad Multiomial Distributios 6.7 Alohaet 6.8 Error Cotrol Codes 6.8.1 Reetitio-by-Three Code 6.8. Geeral Liear Block Codes 6.8.3 Coclusios Summary Problems Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 1 of 0 ECE 3800
Basics of the Biomial Distributio The biomial distributio arises from the summatio of Ideedet ad Idetically Distributed (IID) Beroulli R.V. 1, 0,0, 1, 1 The, 1 0. "" As described reviously, the umber of sequeces with exactly k oes i N trials is based o the combiatorial comutatio of the biomial coefficiet. The total summatio (which must equal oe) ca be erformed as From revious examles it was demostrated that 1 1 Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 of 0 ECE 3800
Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 3 of 0 ECE 3800 Determie the exected value x P X x X E 0 x x x x x X E 0 1 X E Determie the d momet 0 x P X x X E X E 1 Determie the variace X E X E X E 1 X E X E q X E 1 The iitial roofs for these comutatios were rovided i the otes for Chater 4. Textbook examle. 138: Let = 5 ad =0.7. 5 0.73.5 1 5 4 0.7 5 0.7 9.8 3.5 13.3 13.3 3.5 13.3 1.5 1.05 Usig the Biomial_hist.m with some ugradig
0.4 Biomial Theory (bar) vs. Simulatio (stem) 0.35 0.3 0.5 0. 0.15 0.1 0.05 0 0 1 3 4 5 bmea = 3.5000 bvar = 1.0500 bdmomet = 13.3000 Ad the theoretical ad simulatio robability values are as = 0 0.004 0.003 1.0000 0.084 0.03.0000 0.133 0.174 3.0000 0.3087 0.3046 4.0000 0.3601 0.3669 5.0000 0.1681 0.1666 Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 4 of 0 ECE 3800
Additive relatioshi of biomial From the combiatorial aalysis oeratios show i Chater 3. with The icororatio of aother elemet or coi ca be derived from the revious umber of elemets ad the ext beig oe or the other of the ossible trial outcomes. This derivatio ca use coditioal robability for the ext coi Defiig., 1 1 0 0 which ca also be cosidered from the revious trial as 1 1 0 but the revious result is ot based o the th result, so that or usig the b otatio 1., 1. 1, 1., This is related to the Matlab homework roblem for formig Pascal s triagle! HW 3.5 % The fast way to a ritout of the biomial coefficiets M = 10; =0.3, q=0.7; row = 1 for k=1:m-1 row = [0 row]* + [row 0]*q ed This ca also be observed i the followig figure. Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 5 of 0 ECE 3800
6. Comutig Biomial Probabilities Comutig the idividual robabilities., But what about the robability of a articular iterval?., It should be oted that b(,k,) ca be comuted recursively. Therefore, oce the first value of the sum is kow the remaiig ca be quickly comuted. The recursive relatioshi ca be see as.,. 1, 1., 1. 1, 1! 1!!! Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 6 of 0 ECE 3800
With this ratio., 1. 1, First, this may be easier to comute tha the combiatorial coefficiet comutatios. Secod, otice that the relative magitude (icreasig or decreasig) is also established by this relatioshi. The successive robabilities are icreasig if or ad decreasig if The sequece maximum occurs for 1 1 1 1 1 1 1 1 1 Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 7 of 0 ECE 3800
6.3 Comutig the Momets (a alterative) Determie the exected value Usig a idetity 1 1 1 1 A chage i variable for l=k-1 1 1 1 Determie the variace is erformed usig a alterate aalysis, agai laig to use a idetity. Solvig for 1 1 1 1 1 Usig a idetity Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 8 of 0 ECE 3800 1 1 1 1
A chage i variable for l=k- 1 1 1 1 1 1 1 1 1 1 Fiishig 1 1 1 Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 9 of 0 ECE 3800
6.4 Sums of Ideedet Biomial R.V. The sum of biomial R.V. give equivalet values of is biomial. Basis for N trials which is a sum of the two R.V. trials As the sum of ideedet radom variables, the MGF ca be used but the MGF is For the sum this is equivalet to Which is the desired result for a sigle biomial R.V. with The coditioal robability relatig the two R.V. Give N 1 ad N are Biomial R.V ad the sum is formed N=N 1 +N We ca defie a coditioal robability relatioshi: For But the iitial R.V. are ideededt Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 10 of 0 ECE 3800
Exadig to show the comlete mf s The coditioal robability is hyer-geometric! Iterretatio: The coditioal is based o there beig k heads i the first N 1 trial set ad m-k heads i the secod N trial set. The rocess is comarig this robability to m heads i N trials overall. Therefore, the equatio is describig ways i which the first two trial coditios could occur as comared to a sigle comosite trial. Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 11 of 0 ECE 3800
6.5.3 The Negative Biomial Distributio The biomial distributio is based o I ideedet flis, how may heads ca oe exect? The egative biomial distributio is based o to get k heads, how may ideedet flis,, are eeded? To build u to the kth head o the th fli, the k-1 head must have occurred i -1 flis. Therefore, the rior umber is a biomial robability i -1 ad k-1 ad the fial fli is based o, 1 1, 1 1, As a ote, the egative biomial ca be thought of as the sum of k geometric radom variables (where the geometric RV is thought of as a sequece of tails followed by a head ad oe egative biomial is the summatio of k geometric R.V. ) If this relatio holds, the mea ad variace ca be comuted from the mea ad variace of the geometric sequece! Geometric R.V. 1 Sum of IID 1 1 1 Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 1 of 0 ECE 3800
Baseball Examle Examle: I baseball, each iig there are 3 outs, what is the robability there are 3, 4, 5, etc. batters i the iig if they all hit with robability of 0.3? We really wat to talk about outs istead of hits therefore use =1.0.3=0.7. There are three outs, so k=3. Now figure out for =3, 4, 5, etc. 1 31 0.7 0.3, What would be the umber of exected batters er iig? For =3:10 Pr = 3. 4.86 0.3430 0.3087 0.185 0.096 0.0417 0.0175 0.0070 0.007 Probability Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 13 of 0 ECE 3800
6.6 Biomial Estimatio I exerimetal cases whe success ad failure or ositive ad egative results ca occur based o assumed to be ideedet trials, the statistic ca rovide arameters to model (ad simulate) the uderlyig robabilistic exerimets. Biomial equatio estimatio: The mea ad the variace of the biomial, if estimated rovide! 1 For a exerimet with trials, we observe that k successes occurred. Let Uder our assumtio This is a ubiased estimate. The variace of the estimate ca be comuted as As the variace i the estimate goes to zero as icreases, it is also a cosist estimator. Ubiased: the exected value of the estimator equals the value beig estimated. Cosistet: the variace i the estimator goes to zero as goes to ifiity. For may discrete robability models, the estimatio of the mea or mea ad variace rovide all the iformatio required to defie the mf to be used i the model. Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 14 of 0 ECE 3800
The Poisso Distributio htts://e.wikiedia.org/wiki/poisso_distributio The Poisso distributio ca be used as a aroximatio for the biomial distributio for large ad small. Examles: umber of telehoes i a area code, with the robability they are i use trasmittig large files with the robability of bit errors Makig the aroximatios: For large ad small k, let Cosiderig (1-) For How close is it?!!!!!!!! 1!! Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 15 of 0 ECE 3800
0.5 Figure 6.5a Biomial Poisso 0.5 Figure 6.5b Biomial Poisso 0. 0. 0.15 0.15 0.1 0.1 0.05 0.05 0 0 1 3 4 5 6 7 8 9 10 k 0 0 1 3 4 5 6 7 8 9 10 k See: BFigure6_5.m Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 16 of 0 ECE 3800
ALOHANET Probabilistic cocets origially develoed i Hawaii htts://e.wikiedia.org/wiki/alohaet The cocet is based o wireless commuicatio usig a star etwork toology. Assume that the cetral ode of a start trasmits ackets that are received by all the etwork members. This commuicatios occurs at oe frequecy while ay resoses from the etwork members occurs at a searate frequecy. I resoses, the etwork member commuicatio could collide with other members. Whe a acket was correctly received a ackowledgemet was set. If o ackowledgemet was received, the members would have to rebroadcast their message. I slotted ALOHA, the trasmissio time eriods are eriodically defied as time slots. Trasmissios ca oly begi i a time slot ad ot at ay radom time. The robability of a collisio could be comuted ad messages were cosidered either successes or failures At low umber of users ad caacity, this is t a great roblem but First: robability of a message beig trasmitted by 0, 1, or more Usig egative rob. 0 1 1 1 1 1 1 0 1 11 1 Usig the Poisso aroximatio for the Biomial! 0 1 For maximum throughut at a sigle user, study the Pr(N=1) curve. Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 17 of 0 ECE 3800
Selectig the maximum for 1 0 1 0.3679 1 1 1 0.3679 1 Assume that we use some value to rereset the successful robability of acket trasmissio ad the egative, failed acket trasmissio. 0 11 If we defie the robable umber of trials util successful acket trasmissio (a geometric R.V.) as with 1 1 This ow defies a average umber of trasmissio attemts before a success. Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 18 of 0 ECE 3800
6.8 Error Cotrol Codes We ca assume that there is a defied umber of bit error i ay data trasmissio. If we look at a biary stream, we ca assess usig a biomial, the exected umber of bit errors i a bit trasmissio. The bit errors i a bit symbol or eve the bit errors i a b bit ecoded symbol. If the symbol trasmitted has sufficiet redudacy, the origial symbol ca be recovered eve if oe or more bits are i error. The ecodig rocess ca be called Forward Error Correctio. The simlest system is trasmittig 3 or more redudat bits for each bit desired a reetitio by three code. For a three bit ecodig scheme, the received robabilities ca be see as The bit error robabilities for three bit symbols would be 3 0 3 0 1 3 1 3 3 3 3 3 3 Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 19 of 0 ECE 3800
For a bit-error-rate of 10 0 3 0 0.99700999000000 1 3 1 3 0.00994003000000 3 3.997000000000000 06 3 3 3 1.000000000000000 09 So the symbol would be correct rate would become 0 1 ad i error 1 3.99800 06 This is early a 333 times imrovemet! Geeral Liear Block Code (LBC) Trile trasmissio is a form of LBC. There are may others that have well defied structures ad erformace. Overall redudat bits are added to the data to allow a outut message with a defied umber of bit errors to be correctly decoded. See: B. Sklar ad F. J. Harris, "The ABCs of liear block codes," i IEEE Sigal Processig Magazie, vol. 1, o. 4,. 14-35, July 004. htt://ieeexlore.ieee.org/stam/stam.js?t=&arumber=1311137 Notes ad figures are based o or take from materials i the course textbook: Charles Bocelet,, Probability, Statistics, ad Radom Sigals, Oxford Uiversity Press, February 016. B.J. Bazui, Srig 018 0 of 0 ECE 3800