Applying the Moment Generating Functions to the Study of Probability Distributions

Transcription

1 3 Iformaica Ecoomică, r (4)/007 Applyi he Mome Geerai Fucios o he Sudy of Probabiliy Disribuios Silvia SPĂTARU Academy of Ecoomic Sudies, Buchares I his paper, we describe a ool o aid i provi heorems abou radom variables, called he mome eerai fucio, which covers problems abou probabiliies ad expecaios io problems from calculus abou fucio values ad derivaes We show how he mome eerai fucio deermiaes he momes ad how he momes ca be used o recover he mome eerai fucio Usi of mome eerai fucios o fid disribuios of fucios of radom variables is preseed A sadard form of he ceral limi heorem is also saed ad proved Keywords: probabiliy disribuio, probabiliy desiy fucio, mome eerai fucio, ceral limi heorem Iroducio A eerai fucio of a radom variable (rv) is a expeced value of a cerai rasformaio of he variable All eerai fucios have some very impora properies The mos impora propery is ha uder mild codiios, he eerai fucio compleely deermies he disribuio Ofe a radom variable is show o have a cerai disribuio by showi ha he eerai fucio has a cerai ow form There is a process of recoveri he disribuio from a eerai fucio ad his is ow as iversio The secod impora propery is ha he momes of he radom variable ca be deermied from he derivaives of he eerai fucio This propery is useful because ofe obaii momes from he eerai fucio is easier ha compui he momes direcly from heir defiiios Aoher impora propery is ha he eerai fucio of a sum of idepede radom variables is he produc of he eerai fucios This propery is useful because he probabiliy desiy fucio of a sum of idepede variables is he covoluio of he idividual desiy fucios, ad his operaio is much more complicaed The las impora propery is called he coiuiy heorem ad assers ha ordiary coverece of a sequece of eerai fucios correspods o coverece of he correspodi disribuios Ofe i is easier o demosrae he coverece of he eerai fucios ha o demosrae coverece of he disribuios direcly Why is i ecessary o sudy he probabiliy disribuio? The sample observaios are frequely expressed as umerical eves ha correspods o he values of he radom variables Cerai ypes of radom variables occurs frequely i pracice, so i is useful o ow he probabiliy for each value of a radom variable The probabiliy of a observed sample is eeded o mae ifereces abou a populaio Discree ad coiuous disribuios A radom variable is a fucio, whose value is ucerai ad depeds o some radom eve The space or rae of is he se S of possible values of A radom variable is said o be discree if his se has a fiie or couable ifiie umber of disic values (ie ca be lised as a sequece x, x, ) The radom variable is said o have a coiuous disribuio if all values are possible i some real ierval Ofe, here are fucios ha assi probabiliies o all eves i a sample space These fucios are called probabiliy mass fucios if he eves are discreely disribued, or probabiliy desiy fucios if he eves are coiuously disribued All he possible value of a radom variable ad heir associaed probabiliy values cosiue he probabiliy disribuio of he radom variable

2 Iformaica Ecoomică, r (4)/ The discree probabiliy disribuios are specified by he lis of possible values ad he probabiliies aached o hose values, ad he coiuous disribuios are specified by probabiliy desiy fucios The disribuio of a radom variable ca be also described by he cumulaive disribuio fucio F ( x) = P( < x) I he case of discree radom variable, his is o paricularly useful, alhouh i does serve o uify discree ad coiuous variables There are oher ways o characerize disribuios Thus, he probabiliy disribuios ca be also specified by a variey of rasforms, ha is, by fucios ha somehow ecode he properies of he disribuios io a form more coveie for cerai ids of probabiliy calculaio For a discree radom variable wih a probabiliy mass fucio p ( x) = P( = x) we have 0 p( x) for all x ad p( x) = The probabiliy mass fucio or he probabiliy desiy fucio of a radom variable coais all he iformaio ha oe ever eed abou his variable The sequece of momes of a radom variable We ow ha he mea μ = E( ) ad variace σ = E( E( )) = E( ) ( E( )) of a radom variable eer io he fudameal limi heorems of probabiliy, as well as io all sors of pracical calculaios These impora aribues of a radom variable coai impora iformaios abou he disribuio fucio of ha variable Bu he mea ad variace do o coai all he available iformaio abou desiy fucio of a radom variable Besides he wo umerical descripive quaiies μ ad σ ha locae he ceer ad describe he spread of he values of a radom variable, we defie a se of umerical descripive quaiies, called momes, which uiquely deermie he probabiliy disribuio of a radom variable For a discree or coiuous radom variable, he h mome of is a umber defied as μ = E( ), =,,, provided he defii sum or ieral of he expecaio coveres We have a sequece of momes associaed o a radom variable I may cases his sequece deermies he probabiliy disribuio of However, he momes of may o exis I erms of hese momes, he mea μ ad variace σ of are ive simply by μ = μ ad σ = μ μ The hiher momes have more obscure meai as rows The momes ive a lo of useful iformaio abou he disribuio of The owlede of he firs wo momes of ives us is mea ad variace, bu a owlede of all he momes of deermies is probabiliy fucio compleely I ur ou ha differe disribuios ca o have ideical momes Tha is wha maes momes impora Therefore, i seems ha i should always be possible o calculae he expeced value or mea of, E ( ) = μ, he variace, V ( ) = σ or hiher order momes of from is probabiliy desiy fucio, or o calculae he disribuio of, say, sum of wo idepede radom variables ad Y, whose disribuios are ow I pracice, i ur ou ha hese calculaios are ofe very difficul The eerai fucios Rouhly speai, he eerai fucios rasform problems abou sequeces io problems abou fucios I his way we ca use eerai fucios o solve all sors of coui problems Suppose ha a0, a, a, is a fiie or ifiie sequece of real umbers The ordiary eerai fucio of he sequece is he power series = G ( z ) = a + a z + a z + L a z 0 I order o recover he oriial sequece from a = 0 ive ordiary eerai fucio, he followi formula holds:

3 34 Iformaica Ecoomică, r (4)/007 d G a = (0), = 0,,,K! dz Assume ha a0, a, a, is a fiie or ifiie sequece of real umbers The expoeial eerai fucio of his sequece is he power series a = a a G( z) = a0 + z + z + L z!! = 0! For recoveri he oriial sequece of real umbers from he ive expoeial eerai fucio, G(z), he followi formula holds: d G a = ( 0), = 0,,,K dz For a radom variable ai oly oeaive ieer values, wih probabiliies p = P( = ), he probabiliy eerai fucio is defied as: ( ) = G( z) = E z = p z for 0 z The 0 powers of he variable z serves as placeholders for he probabiliies p ha deermie he disribuio We recover he probabiliies p as coefficies i a power series expasio of he probabiliy eerai fucio Expasio of a probabiliy eerai fucio i a power series is jus oe way of exraci iformaio abou he disribuio Repeaed differeiaio iside he expecaio operaor ives d G ( ) ( z) = E( z )= dz = E( ( ) K ( + ) z ), ( ) whece G () = E( ( ) K ( + ) ) for =,,K Thus we ca recover he mome of A exac probabiliy eerai fucio uiquely deermies a disribuio ad a approximaio o he probabiliy eerai fucio approximaely deermies he disribuio The mome eerai fucios The beauy of mome eerai fucios is ha hey ive may resuls wih relaive ease Proofs usi mome eerai fucios are ofe much easier ha showi he same resuls usi desiy fucios (or some oher ways) There is a clever way of oraizi all he momes of a radom variable io oe mahemaical objec This is a fucio of a ew variable, called he mome eerai fucio (mf), which is defied by = E( e ), provided ha he expecaio exiss for i some eihborhood of 0 I he discree case his is equal o e x p(x), ad i he coiuous case o e x f ( x) dx Hece, i is be impora ha he expecaio be fiie for all (, 0 0 ) for some > 0 0 If he expecaio does o exis i a eihbor of 0, we say ha he mome eerai fucio does o exis Sice he expoeial fucio is posiive, E ( e ) always exiss, eiher as a real umber or as a posiive ifiiy The mome eerai fucios may o be defied for all values of, ad some wellow disribuios do o have mome eerai fucio (e he Cauchy disribuio) Observe ha () is a fucio of, o of The mome eerai fucio of a radom variable pacaes all he momes for a radom variable io oe simple expressio Formally, he mome eerai fucio is obaied by subsiui z = e i he probabiliy eerai fucio Noe ha here is a subsiue for mf which is defied for every disribuio, he complex umbers versio of he mf, amely he characerisic fucio Fudameal properies of he momes eerai fucios The mome eerai fucio has may useful properies i he sudy of radom variables, bu we cosider oly a few here Suppose ha is a radom variable wih he mome eerai fucio () Heceforh we assume ha () exiss i some eihbourhood of he orii I his case some useful properies ca be proved

4 Iformaica Ecoomică, r (4)/ If () is he mome eerai fucio of a radom variable he ( 0) = Acually, we have 0 (0) = E( e ) = E() = The momes of he radom variable may be foud by power series expasio The mome eerai fucio of a radom variable is he expoeial eerai fucio of is sequece of momes = μ ( ) =0! Sice he expoeial fucio e has he power series = e, by he series expasio of he fucio e we have he equaliy =0! of radom variables = e The =0! we ae he expecaio of boh sides ad use he fac ha he operaor E commues wih sum o e ( ) = E e E = =0! ( ) = E = E = 0! = 0! 3 Calculai momes We call () he mome eerai fucio because all of he momes of ca be obaied by successively differeiai () ad he evaluai he resul a = 0 The h derivaive of () evaluaed a he poi = 0 is he h mome μ of, ie ( ) ( ) d μ = (0), where ( 0) = ( ) = 0 d I his way, he momes of may also be foud by differeiaio We ca easily see d d d ha = E( e )= E = d d e d = E( e ) (ierchae of E ad differeiaio is valid) Therefore, we obai d = E( ) = μ =0 d I oher words, he mome eerai fucio eeraes all he momes of by differeiaio We ca fid he momes of by calculai he mome eerai fucio ad he differeiai Someimes i is easier o e momes his way ha direcly Toeher, all he momes of a disribuio prey much deermie he disribuio I addiio o produci he momes of, he mf is useful i ideifyi he disribuio of 4 If () exiss i a ierval aroud 0, he owlede of he mf of a rv is equivale o owlede of is probabiliy desiy fucio This meas ha he mf uiquely deermies he probabiliy desiy fucio I eeral, he series defii () will o covere for all Bu i he impora special case where is bouded (ie where he rae of is coaied i a fiie ierval) we ca show ha he series does coveres for all The disribuio fucio is compleely deermied by is momes Theorem Suppose is a coiuous radom variable wih rae coaied i he real ierval [ M, M ] The he series = μ coveres for all o a ifiiely differeiable fucio =0! () ad ( ) (0) = μ Proof: We ow ha μ = x f ( x) dx The we have M M M μ x f ( x) dx M M Hece, for all we have μ ( ) M M e = 0! = 0! This iequaliy shows ha he mome series coveres for all ad ha is sum is ifiiely differeiable, because i is a power series I his way we have show ha he momes μ deermie he fucio () ( ) Coversely, sice μ = (0), we ca see ha he fucio () deermies he momes μ If is a bouded rv, he we ca show ha he mf () of deermiaes he prob-

5 36 Iformaica Ecoomică, r (4)/007 abiliy desiy fucio f (x) of uiquely This is impora sice, o occasio, maipulai eerai fucios is simpler ha maipulai probabiliy desiy fucios 5 Uiqueess heorem assers ha wo radom variables wih he same mf have he same disribuio Le ad Y be wo radom variables wih mome eerai fucios () ad Y () ad wih correspodi disribuio fucios F (x) ad F Y (y) If = Y, he F ( x) = FY ( x) This esures ha he disribuio of a radom variable ca be ideified by is mome eerai fucio A cosequece of he above heorem is ha if all momes of a rv exis, hey characerize compleely he mf (sice he momes are derivaives of he mf i is Taylor series expasio) ad he momes also compleely characerize he disribuio, as well as he cumulaive disribuio fucio, probabiliy desiy fucio ad probabiliy mass fucio Whe a mome eerai fucio exiss, here is a uique disribuio correspodi o ha mome eerai fucio Hece, here is a ijecive mappi bewee mome eerai fucios ad probabiliy disribuios This allows us o use mome eerai fucios o fid disriuios of rasformed radom variables i some cases This echique is mos commoly used for liear combiaios of idepede radom variables 6 Whe he mf exiss, i characerizes a ifiie se of momes The obvious quesio ha he arises is if ca wo differe disribuios have he same ifiie se of mome The aswer is ha, whe he mf exiss i a eihborhood aroud 0, he ifiie sequece of momes does uiquely deermie he disribuio This owlede allows us o deermie he limii disribuio of a sequece of radom variable by examii he associaed mome eerai fucios Theorem Suppose,, is a sequece of radom variables, each havi mf () ad lim = is fiie for all i a eihborhood of 0 The here is a uique cumulaive disribuio fucio F (x) whose momes are deermied by (), ad we have lim F ( x) F ( x) wheever x = F is a coiuiy poi of Thus, coverece of mome eerai fucios o a mome eerai fucio i a eihborhood aroud 0 implies coverece of he associaed cumulaive disribuio fucios 7 Sums of idepede radom variables Mome eerai fucios are useful i esablishi disribuios of sums of idepede radom variables i) If Y = a + b, where a ad b are wo real b cosas, he we have Y = e ( a) ( a + b) b a We have Y = E( e ) = E( e e ) b a b = e E( e ) = e (a) ii) The fucio which eeraes ceral momes of a radom variable wih mea μ is μ ive by μ = e This resul is udersood by cosideri he ( μ ) = E e followi ideiy: μ ( ) = μ μ e E( e ) = e () iii) If ad Y are wo idepede radom variables wih probabiliy desiy fucios f (x) ad f Y (y) ad wih correspodi mome eerai fucios () ad Y (), he heir sum + Y has he mf + Y = Y ( + Y ) Acually, we have + Y = E( e ) = Y E( e ) E( e ) = Y, because ad Y Y bei idepede, so are also e ad e Noe ha his is a very useful propery of mf s, bu he above formula would o be of ay use if we did o ow ha he mf deermies he probabiliy disribuio Usi his resul i is also possible o obai, i a very simple way, he mf of a fiie sequece of idepede ideically disribued radom variables A his poi, he mf s may seem o be a paacea whe i comes o calculai he disribuio of sums of idepede ideically disribued radom vari-

6 Iformaica Ecoomică, r (4)/ ables Someimes we cao wrie dow he disribuio i closed form bu, because here are may umerical mehods for iveri rasforms, we ca calculae probabiliies from a mf 8 There are various reasos for sudyi mome eerai fucios, ad oe of hem is ha hey ca be used o prove he ceral limi heorem Ceral limi heorem Le,, be a sequece of idepede ideically disribued radom variables, each havi mea μ ad variace σ If S = + + L+ S μ ad Z =, he Z has a limii σ disribuio N (0,) as Proof: Le Yi = i μ, for i =,, I his case he variables Y, Y, are idepede ideically disribued ad we have S μ = + L+ μ = Y + Y + L+ Y We ow ha S μ = Y LY = ( Y ) The we ca wrie Z ( S μ )( σ ) = E e = E e ( ) ( )= Z = S = Y μ σ σ Now we use he power series expasio 3 3 Y = + E( Y ) + E( Y ) + E( Y ) +K!! 3! = + σ + o( ) We used he fac ha E( Y ) = 0 ad E( Y ) = σ, ad we deoed ( h( ) o ) a fucio h() such ha 0 as 0 The, for fixed, we obai Z = + σ σ = + o + + o Here o deoes a fucio h() such h( ) ha 0 as 0 We deduce ha i= Z e as ad his is precisely he mf of a variable N (0,) Thus, we have proved ha he disribuio of Z coveres o he sadard ormal disribuio N (0,) as Corollary: If we cosider he sample mea = i, he has a limii disri- σ buio N μ, as The heorem ca be eeralised o idepede radom variables havi differe meas ad variaces Refereces [] Cuculescu, I, (998), Teoria Probabiliailor, Ed All [] Grisead, C, Sell, L, (998), Iroducio o Probabiliy [3] Kalleber, O, (00) Foudaios of Moder Probabiliy, Sprier [4] Perov, VV, (975), Sums of Idepede Radom Variables, Sprier [5] Perov, VV, (995), Limi Theorems of Probabiliy Theory: Sequeces of Idepede Radom Variables, Oxford Uiversiy Press [6] Roussas, G, (005), A Iroducio o Measure-Theoreic Probabiliy, Elsevier Academic Press