Statistics 3858 : Likelihood Ratio for Multinomial Models

Transcription

1 Statistics 3858 : Likelihood Ratio for Multiomial Models Suppose X is multiomial o M categories, that is X Multiomial, p), where p p 1, p 2,..., p M ) A, ad the parameter space is A {p : p j 0, p j 1 } The dimesio of this parameter space is M 1. It is a simplex of dimesio M 1. The likelihood fuctio is M Lp) c, X 1,..., X M ) where the data is X X 1, X 2,..., X M ). Notice that X j 0 ad X 1 + X X M ad )! c, x 1,..., x M ) x 1... x M x 1!x 2!... x M! is the multiomial coefficiet. The MLE is easily foud usig the log-likelihood ad Lagrage multipliers ad is X1 ˆp,..., X ) M A special multiomial model i certai models is of the form p Xj j p pθ) p 1 θ),..., p M θ)) where the compoets are of a fuctioal form of some other parameter. For example i the Hardy- Weiberg model with M 3 p 1 θ, 2θ1 θ), θ 2 ) where θ Θ 0, 1). We ca view this as a particular 1 dimesioal subset or sub-maifold, say A 0 of the M 1 dimesioal simplex A that is the geeral parameter space for multiomials o M categoriess. This sectio costructs the geeralized likelihood ratio GLR) statistic for H 0 : p A 0 1

2 versus H A : p A A A \ A 0 We ofte write the alterative as H A : p is ot i A 0 or simply refer to is as the geeral alterative i this cotext). Let ˆθ be the MLE of θ. The the MLE of pθ) is give by pˆθ). Sice A A A 0 A, the deomiator for the GLR is the likelihood evaluated at the geeral or urestricted MLE of p. Thus the GLR is M ΛX) p jˆθ) Xj M ˆpXj j The rejectio regio is of the form where c is determied by Λx) < c α P 0 ΛX) < c) By Theorem 9.4A c is obtaied as c 1 2 logc) where α P 0 2 logλx)) > c 1 2 logc) ) The costat c 1 is approximately the upper 1 α quatile of a χ 2 d) freedom is d M 1 dima 0 ). distributio where the degrees of For the Hardy-Weiberg model this is M 1 dima 0 ) A size α.05 test will have c ad c e 3.84/2 e Cosider the fuctio g : R + R give by gy) y logy/y 0 ) where y 0 is a give umber. The first two derivatives are ad g y) logy/y 0 ) + y 1 y logy/y 0 ) + 1 g y) 1 y gy 0 ) y 0 logy 0 /y 0 ) 0 g y 0 ) logy 0 /y 0 ) g y 0 ) 1 y 0 2

3 Whe we take the egative 1 times the log of the GLR ΛX) we see, after gatherig up some commo terms, that it cotais ) ˆp j logp j ˆθ)) + logˆp j ) log. p j ˆθ) Aside : We are iterested i a egative umber times the log GLR, sice the GLR 1, ad this will result i the egative log beig positive. If oe were to defie the GLR with the ratio reversed this would ot be the case, but by covetio GLR is defied as this ratio. Some text books ufortuately do ot follow this covetio. Thus for a give j, takig y 0 p j ˆθ) ad y ˆp j gˆp j ) gp j ˆθ)) + g p j ˆθ)) ) ˆp j p j ˆθ) + ) ˆp j p j ˆθ) ˆp j p j ˆθ) 2p j ˆθ) g p j ˆθ)) ˆp j p j ˆθ) Below cosider g j to be the fuctio g above with y 0 p j ˆθ). It the follows that 2 log ΛX)) X j logp j ˆθ)/ˆp j ) X j logˆp j/p j ˆθ)) ˆp j logˆp j /p j ˆθ)) g j ˆp j ) ) 2 ˆp j p j ˆθ) ) + { } ˆp j p j ˆθ) + ˆp j p j ˆθ) ˆp j p j ˆθ) p j ˆθ) p j ˆθ) ˆp 2 j p j ˆθ) p j ˆθ) 3

4 ˆp j p j ˆθ) p j ˆθ) This last expressio is ofte writte as ˆp j X j O j where O j is the observed couts i the j-th category, ad p j ˆθ) Êj or sometimes E j ) as the expected couts for the best fit for the statistical model with parameter θ, that is the restricted multiomial model that correspods to the ull hypothesis. Whe doig this we obtai χ 2 ˆp j p j ˆθ) p j ˆθ) O j Êj Ê j This last formula is called the Pearso s chi-squared statistic. Thus i this multiomial settig the Pearso s chi-squared statistic is equivalet to the geeralized likelihood ratio test. It also has a very atural property of comparig the observed ad fitted model. We reject if the GLR Λ is very small, or equivaletly whe 2 logλ) χ 2 is very large. This of course is a measure which is large if O j is far from the expected couts for the best fitted model i the ull hypothesis. I order to assess whe the observed value of χ 2 is large, we eed to compute for a give α the critical value so that α P 0 χ 2 > c) By Theorem 9.4A Rice) whe the statistical model is oe accordig to the ull hypothesis, the samplig distributio of χ 2 coverges to χ 2 d) where the degrees of freedom is d M 1 dima 0). I the Hardy-Weiberg example, M 3 ad the ull hypothesis is that p A 0 i the otatio at the begiig this hadout, thus the degrees of freedom is The size α.05 critical value to determie the rejectio regio is thus c The decisio rule is thus to reject if ΛX) e 3.84/ or equivaletly if χ 2 3 O j Êj > 3.84 Ê j 4

5 Alteratively we could observe the correspodig statistic ad calculate the p-value. If we observe χ 2 Obs the the p-value is p-value P Y > χ 2 Obs ). Remark : This is of course the value of the critical costat c so that χ 2 Obs falls o the boudary of the rejectio regio. 5