Univariate Normal distribution. whereaandbareconstants. Theprobabilitydensityfunction(PDFfromnowon)ofZ andx is. ) 2π.

Transcription

1 Uivariate Normal distributio I geeral, it has two parameters, µ ad σ mea ad stadard deviatio. A special case is stadardized Normal distributio, with the mea of 0 ad stadard deviatio equal to. Ay geeral X ca be coverted to stadardized Z by Z= X µ σ ad reverse X=σZ+µ ItisusuallyaloteasiertodealwithZ,adthecoverttheresultstoX. Weshouldrecallthatigeeral,ifX Nµ,σ,the ax+b Naµ+b, a σ whereaadbarecostats. TheprobabilitydesityfuctioPDFfromowoofZ adx is f Z z= exp z π f X x= exp x µ σ πσ respectively. Similarly, the momet geeratig fuctiomgf is M z t = exp t M x t = e µt M z σt=exp σ t +µt Bivariate Normal distributio Agai, we cosider two versios, the geeral X ad Y ad stadardized Z adz.thegeeraldistributioisdefiedby5parameterstheidividual meas ad variaces, plus the correlatio coefficiet ρ, the stadardized versio hasolyoe,amelyρ. The two joitbivariate PDF s are f zz z,z = exp z +z ρzz ρ π ρ exp x µ σ + y µ σ ρ x µ σ y µ ρ f xy x,y= πσ σ ρ for the stadardized ad geeral case, respectively. σ

2 Similarly, the joit MGFs are M zz t,t =exp t +t +ρt t M xy t,t =e µ t +µ t M zz σ t,σ t = σ exp t +σ t +ρσ σ t t +µ t +µ t WeshouldrememberthatajoitMGFeablesustofidjoitsimplemomets of the distributio by EX Y m = +m M xy t,t t tm t =t =0 Also,wecaeasilyfidtheMGFofamargialdistributioofXbysettig t =0.ThistellsusimmediatelythatbothZ adz arestadardizednormal. But ow, there is oe extra issue to ivestigate: Coditioal distributioofz Z =z usigboldfaceimpliesthatz is o loger variable, but is assumed to have oe specific observed value. To fid the correspodiguivariate! PDF, we have to do this: fzzz,z f z z = exp z +z ρzz ρ π ρ exp z = π exp z ρz ρ π ρ bysimplealgebra. TheresultcabeidetifiedasNρz, ρ,i.e. Normal, withmeaofρz adstadarddeviatioequalto ρ smallerthawhat itwasmargially,i.e. beforeweobservedz. Notethatmaytextbooksuse this otatio, but put variace i place of stadard deviatio. HowdoweutilizethisresulttofidthecoditioaldistributioofX give that Y has bee observed to have a value of y. Well, we could use the same procedure,butthealgebrawouldgetalotmoremessier,orwecadothis: Wealreadykowthatthecoditioaldistributioof X µ σ Y µ σ = y µ σ isnρ y µ σ, ρ.sowehavethecoditioaldistributioof X µ σ Y =y which is clearly the same thig. Now, usig, which holds coditioally as well,wefidthatthecoditioaldistributioofx Y =yis N µ +σ ρ y µ,σ ρ σ Multivariate Normal Distributio Cosider N idepedet, stadardized, Normally distributed radom vari-

3 ables. Their joit PDF is N fz,z,...,z N = π N/ exp i= z i Correspodig MGF N exp i= π N/ exp zt z t i t T exp t The followig liear trasformatio X=BZ+µ wherebisaarbitraryregularnbynmatrix,defiesaewsetofnradom variables havig a geeral Normal distributio. The correspodig PDF is detb exp x µt B T B x µ π N πn detv exp x µt V x µ sicez=b X µ,whichfurtherimpliesthat X µ T B T B X µ = X µ T BB T X µ = X µ T V X µ whichmustthushavetheχ N distributio. The correspodig MGF is E { exp [ t T BZ+µ ]} =exp t T µ exp t T BB T t = exp t T µ exp t T Vt wherev BB T isthecorrespodigvariace-covariacematrixmustbesymmetric ad positive defiite. This shows each margial distributio remais Normal, without a chage i the correspodig µ ad V elemets. 3

4 NotethattherearemaydifferetB sresultigithesamev. To geerate a set of ormally distributed radom variables havig a give variace-covariace matrix V requires us to solve for the correspodig BMaple provides us with Z oly, whe typig: stats[radom,ormald]0. There is ifiitely may such B matrices, oe of them easy to costruct is lower triagular. Partial correlatio coefficiet The variace-covariace matrix ca be coverted ito the correlatio matrix: C ij V ij Vii V jj ThemaidiagoalelemetsofCareallequaltothecorrelatioofX i with itself. Suppose we have three ormally distributed radom variables with a give variace-covariace matrix. The coditioal distributio of X ad X 3 give thatx =x hasacorrelatiocoefficietidepedetofthevalueofx.itis calledthepartialcorrelatiocoefficiet,addeotedρ 3.Letusfiditsvalue i terms of the ordiary correlatio coefficiets.. Ay correlatio coefficiet is idepedet of scalig. We ca thus choose the three X s to be stadardizedbut ot idepedet, havig the followig 3-D PDF: detc exp zt C z π3 where C= SicethemargialPDFofz is ρ ρ 3 ρ ρ 3 ρ 3 ρ 3 π exp z thecoditioalpdfweeedis detc exp zt C z z π The iformatio about the five parameters of the correspodig bi-variate dis- 4

5 tributio is i z T C z z= z ρ z z 3 ρ + 3 z ρ ρ 3 ρ 3 ρ ρ ρ 3 z ρ z z 3 ρ 3 z ρ 3 ρ ρ 3 ρ 3 ρ ρ ρ 3 ρ 3 which, i terms of the two coditioal meas ad stadard deviatios agrees with what we kow from MATH F8. The extra parameter is our partial correlatio coefficiet ρ 3 = ρ 3 ρ ρ 3 ρ ρ 3 or ρ ij k = ρ ij ρ ik ρ jk ρ ik ρ jk i geeral. To get the coditioal mea, stadard deviatio ad correlatio coefficiet give more tha oe X has bee observed, oe ca iterate i the followig maer: µ i Kl = µ i K +σ i K ρ il K x l µ l K σ l K σ i Kl = σ i K ρ il K ρ ij Kl = ρ ij K ρ il K ρ jl K ρ il K ρ jl K etc., where K ow represets ay umber of idicescorrespodig to the already observed X s. Amoredirectwaytofidtheseispresetedithefollowigsectio. Geeral coditioal distributio: WhetheN variablesarepartitioeditotwosubsets,sayx adx, withmeasµ adµ,adthevariace-covariacematrix [ V V whose iverse is [ A= V V ] V V V V V V V V V V V V V V V V V V V V 5 ]

6 ThecoditioalPDFofX givex =x isobviously πn detv exp x µt V x µ π N detv exp x µ T V x µ i.e. still Normal. To get the resultigcoditioal variace-covariace matrix, allweeedtodoistoivertthecorrespodigblockofa,gettig V, V V V V Similarly,thecoditioalmeasayµ isfoudbasedo It equals Proof: x T V µ = x T V V V V µ x T V V V V V V x µ µ =µ +V V x µ [x µ V V x µ ] T V V V V [x µ V V x µ ] Sice = x µ T V V V V x µ x µ T V V V V V V x µ x µ T V V V V V V x µ +x µ T V V V V V V V V x µ V V V V V V V V V V V V the last matrix equals V V V V V V V V = V V V V V V V V +V V = V V V V V i full agreemet with the coditioal PDF quoted above. Fially,toshowthatdetV detv =detv V V V,takethe determiat of each side of [ I V V ][ ] [ V V V V O V = V V ] O V V V V I 6

7 Estimatig µ,ρadρ The stadard method of fidig good estimators of distributio parameters is called Maximum LikelihoodML techique. We will demostrate it o the geeral Normal bivariate case. First, take the atural logarithm of the PDF of a radom idepedet sample ofpairsofx ady observatiosproductofidividualpdfs,amely exp x i µ σ + y i µ σ ρ x i µ σ y i µ σ ρ l πσ σ ρ = i= ] i=[ xi µ σ + yi µ σ ρ xi µ σ yi µ σ ρ lπ lσ lσ l ρ The, replace the x i ad y i variables by the actual sample values switch to boldface,admaximizethisexpressiowithrespecttotheµ,µ,σ,σ ad ρ parametersby settig the correspodig derivatives equal to zero: σ 3 x i µ ρ y i µ =0 σ σ i= i= y i µ ρ x i µ =0 σ σ i= i= i= σ 3 i µ i=x ρ σ x i µ y i µ = ρ σ i= σ y i µ x i µ = ρ σ σ i µ i=y ρ σ ρ i= i= x i µ σ y i µ σ ρ ρ [ x i µ + y i µ ρ x i µ y ] i µ σ σ σ σ = ρ ρ The first two equatios are solved by makig both i= x i µ ad i= y i µ equaltozero,whichmeas i= ˆµ = x i x 7

8 i= ˆµ = y i y the usual sample meas. The ext three equatios ca be re-writte as x i µ ρ x i µ y i µ = ρ σ σ σ i= i= y i µ ρ x i µ y i µ = ρ σ σ σ i= i= ρ x i µ y i µ σ i= σ [ ρ x i µ + y i µ ρ x i µ y ] i µ σ σ σ σ i= = ρ ρ Usigthefirsttwoequatios,thesecodtermofthelastequatiocabesimplifiedto ρ ρ.thelastequatiothusreads ρ x i µ y i µ =ρ ρ σ σ i= which implies i= ρ= x i µ y i µ σ σ Substitutig this for the secod term of the first two equatios makes the secod term read ρ. Cacellig with the correspodig term o the right had side yields x i µ = σ i= y i µ = σ i= ad the followig fial estimators: i= ˆσ = x i x ad ˆρ= ˆσ = i= y i y i= x i xy i y i= x i x i= y i y 8

9 sample correlatio coefficiet, more commoly deoted r. i= Xi X i= Yi Y We kow that X, Y, ad have simple distributios,thedistributioofrisalotmorecomplicated;thisisitspdf: σ σ Γ ρ π Γ r ρr 3 where F is the hypergeometric fuctio defied by F, ; ; +ρ r Fa,b;c;x=+ a b x+aa+ bb+ x c! cc+! + aa+a+ bb+b+ cc+c+ 3! This result is due to Hotellig953. The correspodig expected value of r is x ργ Γ Γ+ F, ; + ;ρ sotheestimatorisclearlybiased. Expadedipowersof,thisbecomes ρ ρ ρ 3ρ ρ +3ρ 8... Oe fids that the distributio of arctahr coverges to Normal distributioalotfasterithasasmallerbias,adalotsmallerskewess. To derive this PDF, we first eed some formulas cocerig dimesioal sphere ofradiusrisdefiedasthefollowigset x +x +...+x r ItsvolumeV r=r V adsurfaceareas rareclearlyrelatedby S r= dv r dr =r V =r S ItiseasiertofidS from π / = exp x x... x dx dx...dx = S S 0 0 r exp r dr = y / exp ydy = S Γ 9

10 xi x =s which implies that ad S = π/ Γ S r= π/ r Γ V r= π/ r Γ Sample from stadardized bi-variate Normal distributio hasapdfgiveby where π ρ / exp π ρ / exp x i +y i ρx i y i ρ s + x +s +ȳ ρrs s ρ xȳ ρ x = s = r = xi yi ȳ= xi x s yi ȳ = xi xy i ȳ s s dx...dy = dx...dy Allwehavetodoowistofigureoutthevolumeofthedimesioalregio filledbytakigeachofthefivequatitiess,s, x,ȳadr,adicreasigthem byds, ds,d x,dȳaddrrespectively. Wekowthat xi = x is a plae i the first dimesios the x-space, goig through x, x,... x adhavig,,... asormal. Similarly is a dimestioal sphere cetered o x, x,... x, with the radius of s. The plae cuts the sphere i the middle, thus creatigthe correspodig cross sectioa dimesioalsphere,whosesurfaceis π / s Γ / 0

11 Thiseedstobefurthermultipliedsicethe ds ad d xdirectiosare perpedicularbyd xds. Similarargumetcabemadeitheyspace,exceptowweeedtokeep they ȳ,y ȳ,...y ȳvectoratafixedagletox x,x x,...x x, aaglewhosecosisr.thisreducesthe dimesioalshpericalsurfaceof radiuss toa dimesioalsphericalsurfaceofradiuss r. This the cotributes π / s 3 3/ r 3/ Γ s dȳds dr r Soowwehave r 4/ s s π 3/ ρ / Γ Γ exp s + x +s +ȳ ρrs s ρ xȳ ρ d xdȳds ds dr R. A. Fisher, 95. The d xdȳ itegratio separable, from to each ca be carried out easily, yieldig r 4/ s s π ρ / 3! exp s +s ρrs s ρ ds ds dr wehavealsoreplaced 3 Γ Γ by 3! π. Toitegrateovers ads,werealizethatthes >s ads <s regios mustcotributethesameamoutthefuctioiss s symmetric, sowe multiply the itegrad by ad itegrate over the latter. Itroducig the followig oe-to-oe trasformatioξ goes from 0 to, η from to ifiity: whose Jacobia is results i ξ = s s η = s + s s s det [ s s s s s s s s s = η r 4/ ξ π ρ / 3! exp = s +s s s s s s η ρrξ ρ ] = dξdη η dr

12 which ca be easily itegrated over ξ, gettig r 4/ π ρ / Ad the last substitutio is ρ η ρr η= ρrz z zgoigfrom0toreplacesη ρrby dηby ad η by gettig Maple tells us that ρr z ρr z dz z ρr +ρr z z dη η dr r 4/ z / +ρr z / π ρ / z ρr 3/ dzdr whichiourcasemeas Thefialasweristhus 0 z k z m az l dz= Γk+Γm+ F l,k+;m+k+;a Γk+m+ Γ Γ Γ F, ; ; +ρr! ρ / πγ r / ρr 3/ F, ; ; +ρr dr