Convergence of Binomial to Normal: Multiple Proofs

Transcription

1 Iteratioal Mathematical Forum, Vol. 1, 017, o. 9, HIKARI Ltd, Covergece of Biomial to Normal: Multiple Proofs Subhash Bagui 1 ad K. L. Mehra 1 Departmet of Mathematics ad Statistics The Uiversity of West Florida Pesacola, FL 3514, USA Departmet of Mathematical ad Statistical Scieces Uiversity of Alberta Edmoto, T6G G1, Caada Copyright 017 Subhash Bagui ad K.L. Mehra. This article is distributed uder the Creative Commos Attributio Licese, which permits urestricted use, distributio, ad reproductio i ay medium, provided the origial work is properly cited. Abstract This article presets four differet proofs of the covergece of the Biomial B(, p ) distributio to a limitig ormal distributio,. These cotrastig proofs may ot be all foud together i a sigle book or a article i the statistical literature. Readers of this article would fid the presetatio iformative ad especially useful from the pedagogical stadpoit. This review of proofs should be of iterest to teachers ad studets of seior udergraduate courses i probability ad statistics. Keywords: Biomial distributio, Cetral limit theorem, Momet geeratig fuctio, Ratio method, Stirlig s approimatios 1. Itroductio The biomial distributio was first proposed by Jacob Beroulli, a Swiss mathematicia, i his book Ars Coectadi published i 1713 eight years after his death [7]. This distributio is probably most widely used discrete distributio i Statistics. Cosider a series of idepedet trials, each resultig i oe of two possible outcomes, a success with probability p ( 0 p 1) ad failure with probability q1 p. Let deote the umber of successes i these trials.

2 400 Subhash Bagui ad K.L. Mehra The the radom variable (r.v.) is said to have biomial distributio with parameters ad p, b(, p ). The probability mass fuctio (pmf) of is give by (see [6], [7])! p( ) P( ) p q!( )!, 0,1,,, (1.1) with p ( ) 1. The mea ad variace of the biomial r.v. are give, 0 respectively, by p ad pq. Most udergraduate elemetary level statistics books list biomial probability tables (e.g., [6], [7]) for specified values of ( 30) ad p. It is well kow that (see [5]) if both p ad q are greater tha 5, the the biomial probabilities give by the pmf (1.1) ca be satisfactorily approimated by the correspodig ormal probability desity fuctio (pdf). These approimatios (see [5]) tur out to be fairly close for as low as 10 whe p is i a eighborhood of1. The Frech mathematicia Abraham de Moivre (1738) (See Stigler 1986, pp.70-88) was the first to suggest approimatig the biomial distributio with the ormal whe is large. Here i this article, i additio to his proof based o the Stirlig s formula, we shall preset three other methods the Ratio Method, the Method of Momet Geeratig Fuctios, ad that of the Cetral Limit Theorems for demostratig the covergece of biomial to the limitig ormal distributio. Uder the first two methods, this is achieved by showig the covergece, as, of the stadardied pmf of b(, p ) to the stadard ormal probability desity fuctio (pdf). Uder the latter two, this is achieved by showig the covergece, as, of the Laplace or Fourier trasform of the Biomial distributio b(, p) to a Laplace or Fourier trasform, from which the the stadard ormal distributio is idetified as the limitig distributio. The orgaiatio of the paper is described as follows. We state some useful prelimiary results i Sectio. I Sectio 3, we provide the details of various proofs of the covergece of biomial distributio to the limitig ormal. Sectio 4 cotais some cocludig remarks.. Prelimiaries I this sectio we state a few hady formulas, Lemmas, ad Theorems which shall be eeded i describig various methods of proofs preseted i Sectio 3.

3 Covergece of biomial to ormal: multiple proofs 401 Formula.1. Whe is large, the Stirlig s approimatio formula (see [1], [10]) for approimatig factorial fuctio! ( 1)( ) ()(1) is give by! ( ) e (i.e., [! ( e )] 0 as ). (.1) Formula.. (i) For 1 1, 3 4 k k1 l(1 ) ( 1). 3 4 k k1 I this series if we replace by, we get (ii) for 1 1, 3 4 l(1 ) 3 4 k1 (.a) k. (.b) k Defiitio.1. Let be a r.v. with probability mass fuctio (pmf) or probability desity fuctio (pdf) f (),. The the momet geeratig fuctio (mgf) of the r.v. is defied to the fuctio for all t h, h 0. t M ( t) E( e ) t e f ( ), if is discrete t e f ( ) d, if is cotiuous If the mgf eists (i.e., if it is fiite), there is oly oe uique distributio with this mgf. That is, there is a oe-to-oe correspodece betwee the r.v. s ad the mgf s if they eist. Cosequetly, by recogiig the form of the mgf of a r.v, oe ca idetify the distributio of this r.v. Theorem.1. Let { M ( t), 1,, } deote the sequece of mgf s correspodig to the sequece { } of r.v. s 1,, ad M () t the mgf of a r.v., which are all assumed to eist for t h, h 0. If lim M ( t) M ( t) for t h the The otatio d. d meas that, as, the distributio of the r.v. coverges to the distributio of the r.v..

4 40 Subhash Bagui ad K.L. Mehra Lemma.1. Let { ( ), 1} be a sequece of real umbers such that lim ( ) 0. The a ( ) lim 1 e. Theorem.. Let 1 b ab, provided a ad b do ot deped o,, be a sequece of idepedet ad idetically distributed (i.i.d.) r.v. s with fiite mea ad variace 0. Set ad S.. The, S S i1 ( ) d N (0,1) as Theorem. is kow as the Cetral Limit Theorem (CLT). The symbol N (0,1) deotes that the r.v. follows N (0,1), which otatio stads for the ormal distributio with mea 0 ad variace1, referred to as the stadard ormal distributio. 1 Lemma.. Let N (0,1), whose pdf is give by f () e, ; the t M () t e. Proof. Sice N (0,1), it follows that t 1 t t 1 (1 )( t) t. M ( t) E( e ) e e d e e d e Big O ad Small o Notatios: The Big-O equatio g( ) O( f ( )) g f remais bouded, as ; i.e., there eists a positive costat M such that g( ) f ( ) M for all. O the other had, Small-o equatio g( ) o( f ( )) symbolies that ratio ( ) ( ) symbolies that the ratio g( ) f ( ) 0, as ; i.e., give 0, there is a ( ) 0 0 such that g ( ) f ( ) for all 0. Thus, for eample, if f( ) 0, as, g( ) O( f ( )) implies that g ( ) 0 at the same or higher rate tha that of f( ) ; whereas g( ) o( f ( )) implies that g ( ) 0at a higher rate tha that of f( ). For Defiitio.1, Theorem.1, Theorem., ad Lemma.1, see Bai ad Egelhardt 199, pp. 78, 36, 38, ad 34, respectively [3]. i

5 Covergece of biomial to ormal: multiple proofs Proofs of Various Methods I this sectio, we preset four differet proofs of the covergece of biomial b(, p) distributio to a limitig ormal distributio, as Use of Stirlig s Approimatio Formula [4] Usig Stirlig s formula give i Defiitio.1, the biomial pmf (1.1) ca be approimated as ( ) e P( ) p q ( ) ( ) e ( )( ) e p q ( ) ( ) 1 p q pq p q 1 ( ) 1 C p q, (3.1) where C 1 [ pq]. Now takig atural logarithms o both sides of (3.1), we have l P( ) l C ( 1 )l ( 1 )l p q. (3.) Let p ad pq p. The last equatio leads to: p pq pq so that p 1 q p ; ad q pq so that p 1 q q. I view of the precedig equatios ad equatios (.a) ad (.b), we ca rewrite (3.) as l P( ) l C ( p pq 1 )l 1 q p

6 404 Subhash Bagui ad K.L. Mehra p ( q pq 1 )l 1 q :l C I (, p, q) I (, p, q) (say) (3.3) where, usig Taylor series epasios ad detailig of the epressios, we obtai I (,, ) ( 1 ) p q p pq 1 ( 1) 1 q p 3 q q q p p p 3 p 3 4 ( ) 1 3 q 1 p q q q pq p p p 3 ( ) 1 q p 1 q q p p 1 ( ) 1 1 q 1 p pq q 3 q 3 3 p q ( 1) q p 1 ( 3) p q q 3 3 p 3 3 q p q ( 1) q p 1 ( ) p 1 q q p p 1 1 ( ) ( 1) q p pq q O(1 ), (3.3a) for all fied, as, the last equality followig clearly from the precedig epressio, sice the three ifiite sums i this epressio are all (i absolute value) domiated for each fied ad sufficietly large by l 1 (1 q p), which coverges to ero, as, for each fied. Similarly, usig Taylor series

7 Covergece of biomial to ormal: multiple proofs 405 epasios ad appropriate detailig of terms as i (3.3a) (i fact, by ust iter chagig p ad q ad chagig the sig of i (3.3a)), we obtai I(, q, p) pq p O(1 ), (3.3b) for all fied, as. Combiig (3.3), (3.3a), ad (3.3b), we have l P l C O(1 ). (3.4) Hece, from (3.4) we get for large, 1 P( ) Ce e d, (3.4a). The above equatio (3.4a) may also be writte as where d 1 pq ( p) ( pq) 1 P( ) e pq. (3.4b) This completes the proof. Remark 3.1. Uiform Approimatio. From equatios (3.3) (a) - (b), it becomes apparet at oce that the order term O(1 ) i (3.4) holds uiformly i over ay [, ] give real compact iterval 1. Cosequetly, the approimate equality (3.4a) holds uiformly i over ay compact sub-iterval o the real lie. I fact, it follows from these equatios that the precedig approimatio does also hold ( ) ( ) [, ] uiformly, as, over the epadig sequece of compact itervals, where give by [( k p) pq] ad ( ) ( ) ( ) ( k ) 3 [( k p) pq], with ( ) ( ) k mi{ k :[( ) ] 0} ad ( ) ( k ) 3 ( ) k ad ( ) k k ma{ k :[( ) ] 0}. We leave the details of this as a eercise for the readers (cf. Feller [4] Vol I; Theorem 1, pp ) 3.. The Ratio Method [8] The ratio method was itroduced by Proscha (008). The ratio of two successive probability terms of the biomial pmf stated i (1.1) is give by P( 1) P( )!( )! p ( ) p ( 1)!( 1)! q ( 1) q (3.5) Cosider the trasformatio ( p) pq. The p pq, so that substitutig this value of ito (3.5) produces

8 406 Subhash Bagui ad K.L. Mehra P( p pq 1) ( p pq) p. (3.6) P( p pq) ( p pq 1) q The ratio o the left had side of (3.6) ca be epressed as P ( p) pq 1 pq P P ( ) P( ) p pq, (3.7) where ( p) pq ad 1 pq. Note that as, 0. The ratio o the right had side of (3.6) ca be simplified as ( p pq) p ( q ( pq) ) p ( p pq 1) q ( p ( pq) 1 ) q Combiig (3.7) ad (3.8), we get (1 p ( q)) (1 p ) (1 q ( p) 1 p) ( ) (1 ) (1 q q ). (3.8) P( ) (1 p ) P q q. (3.9) Now assume the eistece of a sufficietly smooth pdf f() such that for large, the probability P( ) ca be approimated by the differetial f () d. Hece, we ca take P( ) f ( ) d ad P( ) f ( ) d o the left had side of (3.9), so that the probability ratio becomes equivalet to f ( ) f ( ). This, i couctio with (3.9), implies that f ( ) (1 p ) f p q ( ) (1 ). (3.10) Now takig logarithms o both side of (3.10), we have l f ( ) l f ( ) l(1 p ) l(1 q q ), (3.11)

9 Covergece of biomial to ormal: multiple proofs 407 Dividig both sides of (3.11) by ad takig the limits as, or equivaletly, as 0, the equatio (3.11) takes the form p l 1 q q l f ( ) l ( ) l 1 f lim lim 0. (3.1) The left had side of (3.1) is othig but the derivative of l f( ). Applyig L Hopital s theorem, the limit o the right had side of (3.1) takes the form l 1 lim 0 p l 1 q q p q q lim lim 0 0 (1 p ) (1 q q) Now combiig (3.1) ad (3.13) we may coclude that p q. (3.13) Itegratig both sides of this equatio with respect to, we get l f ( ), where c is the costat of itegratio. This i tur gives f ( ) ke k c e must be equal 1 to make the RHS a valid (0,1) coclude that, as, the r.v. ( ) its distributio i the limit; or equivaletly, that the r.v. a ormal distributio with mea p ad variace d l f ( ). d c, where N desity. Thus, we p pq has the stadard ormal as follows approimately pq whe is large The MGF Method [], [3] Let be a biomial r.v. with parameters ad p. The the mgf of the r.v. t evaluates to M () t e p q ( q pe t ). Let p pq, so 0 that upo settig pq, we ca rewrite as p. Below we derive the mgf of, which is give by

10 408 Subhash Bagui ad K.L. Mehra M t E e E e t t( p ) ( ) ( ) ( ) e pt ( t ) ( E e ) pt pt t e M ( t ) e q pe pt qt. (3.14) qe pe The Taylor series epasio for qt e may be writte as e qt qt q t q t q t ( ) 1 e, (3.15) 3 4 (!) (3!) (4!) where () is a umber betwee 0 ad qt t q p, ad ( ) 0 pt Similarly, the Taylor s series epasio for e may be writte as as. e pt pt p t p t p t ( ) 1 e, (3.16) 3 4 (!) (3!) (4!) where () is a umber betwee 0 ad pt t p q, ad ( ) 0as. Now substitutig these two equatios (3.15) ad (3.16) i the last epressio for M () t i (3.14), we have 3 4 pqt pqt pqt pqt pqt 3 ( ) 3 ( ) M ( t) 1 ( ) ( ) ( ) q p q p q e p e 3 4! 3! 4! pq pq. (3.17) ( ) 3 ( ) ( ) ( t t q p t q e p e ) 1 ( )(3!)( ) 1 ( )(4!)( ) The above equatio (3.17) may be rewritte as M t ( ) ( ) 3 ( ) ( t) 1, where t ( q p) t ( q e p e ) ( ). ( pq)(3!) ( pq)(4!) Sice ( ), ( ) 0 as, it follows that lim ( ) 0 of t. Thus, by Lemma.1, we have for every fied value

11 Covergece of biomial to ormal: multiple proofs 409 lim M ( t) e M ( t), where (0,1) t N, for all real values of t. I view of Theorem.1, thus, we ca coclude that, as, the r.v. p pq has the stadard ormal as its limitig distributio; or equivaletly, that the biomial r.v. approimate ormal distributio with mea p ad variace has, for large, a pq The CLT Method [3], [4] Let Y1, Y,, Y be a sequece of idepedet ad idetical Beroulli r.v. s each with success probability p success, i.e., 1 with prob. p Yi 0 with prob. (1 p) q for i 1,,,. The the sum Y ca be show to have a Biomial i1 distributio with parameters ad p. To verify this assertio, we use the mgf methodology of Sectio : The mgf of each ( q pe t ), k 1,,, t Ee ( ) = ty i i 1 Ee, so that the mgf of i Y is M () t k Y k i1 i E( e k ) pe qe ty t (0) t Y is give by M () t t ( M ( t)) ( q pe ). This is eactly the directly Y k evaluated mgf of a biomial r.v. with parameters ad p i Sectio 3.3. Thus, follows a b(, p ) distributio. Sice is a sum of idepedet ad idetically distributed r.v. s with fiite mea p ad variace pq, by the CLT Theorem., we ca coclude at oce that, ( ) ( ) d N (0,1), as ; or equivaletly, that the r.v. ( p) ( pq) follows approimately the ormal distributio with mea p ad variace pq whe is large. 4. Cocludig Remarks This article presets four methods a ew elemetary method ad three other wellkow oes for derivig the covergece of the stadard biomial b(, p ) distributio to the limitig stadard ormal distributio, as.

12 410 Subhash Bagui ad K.L. Mehra The ew elemetary Ratio Method, of recet origi, is i fact ust a modificatio of oe the other three, vi., of the de Moivre s method based o Stirlig s Approimatio. The Ratio Method, however, circumvets the use of Stirlig s approimatio. The other two methods are, respectively, the Momet Geeratig Fuctios method based o Laplace trasforms ad the last oe that of the Cetral Limit Theorems, based o Fourier trasforms ad comple Aalysis. While the last three methods utilie advaced theoretical tools ad are broader i scope, the Ratio Method requires kowledge of oly elemetary calculus, amely, of derivatives, Taylor s series epasios, simple itegratio etc., alog with that of basic probability cocepts. All four differet types of methods have bee employed here i oe sigle place to demostrate the covergece of the stadardied biomial r.v. to a limitig ormal N (0,1) r.v.. This article may serve as a useful teachig referece paper. Its cotets ca be discussed profitably i seior level probability or mathematical statistics classes. The teachers may assig these differet methods to studets i seior level probability classes as class proects. Ackowledgemets. This article was preseted i abstract ad poster form at Florida Chapter of America Statistical Associatio (ASA) Aual meetig 016, Tallahassee ad ASA Joit Statistical Meetig 016, Chicago. Refereces [1] S.C. Bagui, S.S. Bagui, ad R. Hemasiha, Norigorous proofs of Stirlig s formula, Mathematics ad Computer Educatio, 47 (013), [] S.C. Bagui ad K. L. Mehra, Covergece of biomial, Poisso, egativebiomial, ad gamma to ormal distributio: Momet geeratig fuctios techique, America Joural of Mathematics ad Statistics, 6 (016), [3] L.J. Bai ad M. Egelhardt, Itroductio to Probability ad Mathematical Statistics, Dubury, Belmot, 199. [4] W. Feller, A Itroductio to Probability Theory ad Its Applicatios, Vol. 1, Wiley Easter Limited, New Delhi, [5] A. de Moivre, The Doctrie of Chaces, H. Woodfall Lodo, [6] R. Khaaie, Elemetary Statistics i a World Applicatios, d ed., Scott, Foresma ad Compay, Gleview, [7] J.T. McClave ad T. Sicich, Statistics, Pearso, New Jersey, 006.

13 Covergece of biomial to ormal: multiple proofs 411 [8] M.A. Proscha, The ormal approimatio to the biomial, The America Statisticia, 6 (008), [9] S.M. Stigler, S.M., The History of Statistics: The Measuremet of Ucertaity before 1900, Cambridge, MA: The Belkap Press of Harvard Uiversity Press, [10] J. Stirlig, Methodus Differetialis, Lodo, Received: February 15, 017; Published: March 19, 017