( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2

Size: px

Start display at page:

Download "( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2"

Wendy Bryant
5 years ago
Views:

1 82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1, give the observed value x, rejects the NH at α -level of sigificace wheever θ Λ x sup θ H 0 f X x θ sup θ Θ f X x θ < c where c is such that Thus the g.l.r.t. has rejectio regio R sup Pr Λ X < c α θ H 0 x : Λ x sup θ H 0 f X x θ sup θ Θ f X x θ θ < c Example: Cosider the last example agai i which we wish to test whether productio i two plats is uiform i.e. test NH : θ 1 θ 2 agaist AH : θ 1 θ 2 where θ i is the parameter of the expoetial distributio of the compoet failure times produced i plat i, i 1, 2. Fid the g.l.r.t. procedure i this case. Solutio: We have see that the likelihood is θ f X x θ θ 1e θ 1T θ m 2 e θ 2S where T z i ad S m y i, ad the log-likelihood is l θ log θ 1 θ 1 T + log θ 2 θ 2 S Hece l θ θ 1 0 l θ θ2 0 θ 1 T 0 m θ 2 S 0 θ1 T θ 2 m S

2 4.4. RESTRICTED M..E AND THE G..R.T. 83 i.e. θ /T m/s ad e m m T T e m S S T S m m T S m e +m We have already see that whe the NH is true the restricted m.l.e. s are θ 1 θ 2 +m ad hece T+S m + m θ e +m T+S + m T e +m T+S S T + S T + S +m + m e +m T + S The g.l.r.t. statistic is therefore Λ x θ +m + m e +m T + S m m T S m e +m ad the rejectio regio of the g.l.r.t. is T S m costat T + S +m R {x :Λ x < c} { } T S m x :costat T + S +m < c { } T S m x : +m < c T + S { T/S } x : +m < c 1 + T/S { } U x : +m < c 1 + U where U T S z i m y i

3 84 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Plot of the statistic U /1+U +m U /1+U +m c 0 0 u 1 u 2 U Figure 4.8: The coditio U /1 + U +m < c is equivalet to U < u 1 ad U > u 2. From the figure above we see that the rejectio regio ca also be writte as R {x :U < u 1 or U > u 2 } { x :U m < v 1 or U m } > v 2 {x :V < v 1 or V > v 2 } where V U m T/ S/m z average of the z-values ȳ average of the y-values. To fid v 1 ad v 2 we must kow the distributio of V whe the NH is true. Sice the Z i s are expoetially distributed with parameter θ 1 the T Z i Gamma,θ 1 i.e. 2θ 1 T χ 2 2. Similarly 2θ 2 S χ 2 2m. Sice the Z i s ad Y i s are idepedet 2θ 1 T/ 2θ 2 S/m χ2 2/2 χ 2 2m/2m F 2,2m Whe the NH is true θ 1 θ 2 ad hece V T/ S/m 2θ 1T/ 2θ 2 S/m F 2,2m

4 4.4. RESTRICTED M..E AND THE G..R.T. 85 The values v 1 ad v 2 determiig the rejectio regio ca therefore be obtaied from the F 2,2m tables such that Pr v 1 F 2,2m v 2 1 α Example: et X be a radom sample of idepedet observatio from the Bi m,θ distributio. Fid the g.l.r.t. of NH : θ θ agaist the alterative AH : θ > θ for some fixed ad kow value θ. Solutio: The likelihood is θ;x where t x i. Cosequetly ad Thus sup θ;x θ Θ sup θ;x θ H 0 Λ x Λ t m x i θ x i 1 θ m x i θ t 1 θ m t sup θ;x θ H 0 sup θ 0,1 θ t 1 θ m t t t 1 t m t m m sup θ t 1 θ m t θ θ t t m 1 t m t t if m m θ θ t 1 θ m t if t m > θ θ;x t 1 if m θ θ t 1 θ m t sup t t θ Θ m 1 t m 1 if t mθ mθ t t m mθ m t m t if t > mθ m t if t m > θ Figure 4.9 gives the plot of Λ t from which it ca be see that it is a o-decreasig fuctio of t ad cosequetly the rejectio regio is give by R {x : Λ t < c α } {x : t > k α }

5 86 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION ikelihood fuctio G.l.r.t. statistic 1 likelihood θ g.l.r.t. Λt c α 0 0 t/m 1 θ 0 0 mθ t k α m Figure 4.9: The likelihood of the biomial data ad the g.l.r.t statistic where k α is such that sup Pr X R sup Pr X i > k α α level of sigificace θ θ θ θ But X i Bi m,θ. Further, sice Pr Y > k θ is icreasig i θ whe Y Bi r, θ, it follows that ad k α satisfies sup Pr θ θ m jk a+1 X i > k α Pr m j θ j 1 θ m j α X i > k α θ θ Note that there may ot exist iteger k α which satisfies the above equatio. If it does ot, we the choose k α such that m jk a+1 m j θ j 1 θ m j < α ad m jk a m j θ j 1 θ m j > α

6 4.5. ASYMPTOTIC FORM OF THE G..R.T. 87 i.e. k α is the smallest iteger for which the is less tha α probability that a Bi m,θ radom variable will exceed it. If is large, we ca use the Normal approximatio to the Bi m,θ distributio ad hece obtai k α as the solutio to Pr N mθ,mθ 1 θ > k α α i.e. i.e. Pr N 0, 1 > k α mθ mθ 1 θ α k α mθ mθ 1 θ z α or k α mθ + z α mθ 1 θ where z α is such that Φ z α 1 α, Φ. beig the Stadard Normal distributio fuctio. Cosequetly the rejectio regio is { } R x : x i mθ mθ 1 θ > z α 4.5 Asymptotic form of the g.l.r.t. Result: I testig the NH : θ H 0 agaist the alterative AH ; θ Θ H 0 where H 0 {θ : h 1 θ 0,h 2 θ 0,...,h r θ 0}, provided the sample size o which the test is based is large, the uder mild regularity coditios 2 log Λ X 2 log θ 2[l θ l] is approximately chi-squared distributed with r degrees of freedom whe the ull hypothesis NH is true. The critical regio of the g.l.r.t ca therefore be take as R { } x : 2 log Λ x χ 2 r,α where χ 2 r,α is take from the chi-squared tables ad is such that if W χ 2 r the Pr W χ 2 r,α α ad α level of sigificace of test. The degrees

7 88 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION of freedom are equal to the umber of idepedet side coditios used to specify the ull hypothesis. Note that 2 log Λ x χ 2 r idepedetly of θ as log as θ H 0. I the above Λ x is the g.l.r.t. statistic, θ is the likelihood fuctio, lθ is the lo-likelihood fuctio, θ the m.l.e. of θ ad θ the restricted m.l.e. i H 0. Sketch of proof: Sice θ Λ x sup θ θ H 0 sup θ θ Θ we have that { } 2 log Λ x 2 log log θ { } 2 l l θ where θ is the m.l.e of θ ad θ is the restricted m.l.e. However, we have see that whe bmθ ad θ emerge as turig poits of the likelihood θ ad hece of the log-likelihood l θ ad whe the ull hypothesis is true, θ ad θ are close to each other with high probability. Hece expadig l θ about θ we get to a secod order approximatio k 2 log Λ x 2 l l θi θ l i θ i 1 k k θi 2 θ i θj θ 2 l j θ j1 i θ j k k θi θ i θj θ 2 l j 4.19 θ i θ j j1 By argumets similar to those that produced 4.8 we have by the law of large umbers 2 l θ 2 l θ E I ij θ as θ i θ j θ i θ j

8 4.5. ASYMPTOTIC FORM OF THE G..R.T. 89 with I ij θ the i,jth elemet of the Fisher Iformatio matrix I θ. Hece for large 4.19 is further approximated by k k 2 log Λ x θi θ i θj θ j I ij j1 T θ θ I θ θ 4.20 Fially sice θ ad θ 0 are close to each other with high probability we ca itroduce the further approximatio T 2 log Λ x θ θ I θ0 θ θ But sice, as we have see, whe the ull hypothesis is true both θ ad θ are multivariate Normally distributed ad hece so is θ θ it follows that 2 log Λ x is approximately a quadratic i a Normally distributed radom vector of zero mea vector. It ca be show that such a quadratic is chisquared distributed. This is a extesio of the result that if W N 0,σ 2 the W 2 /σ 2 χ 2 1 Example et X ij, j 1, 2,..., i be the failure times of a radom sample of i electroic compoet produced selected from the productio lie of the ith maufacturer i 1, 2, 3 ad assume that they are idepedetly ad expoetially distributed with meas which may differ from maufacturer to maufacturer. Assumig that 1, 2 ad 3 are large, costruct a approximate test to test the ull hypothesis that the mea times to failure are the same for the three compaies. If ad X 1j 3106, X 2j 5620 X 3j 3912 j j carry out the test ad report your coclusios. Solutio: {X ij } i j1 is a radom sample from the expoetial distributio with parameter θ i, mea 1/θ i i 1, 2, 3 ad we wish to test the hypothesis NH : θ 1 θ 2 θ 3 agaist AH : θ 1 θ 2 θ 3 Here θ θ 1,θ 2,θ 3 ad the likelihood of the observed values is θ 1 j1 2 3 θ 1 e θx 1j θ 2 e θx 2j θ 3 e θx 3j j1 j j1 θ 1 1 e θ 1t 1 θ 2 2 e θ 2t 2 θ 3 3 e θ 3t 3

9 90 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION where t 1 1 j1 x 1j, t 2 2 j1 x 2j t 3 3 j1 x 3j. The log-likelihood is Hece l θ 1 log θ log θ log θ 3 θ 1 t 1 θ 2 t 2 θ 3 t 3 l θ θ i 0 i θ i t i θ i i t i i 1, 2, 3 Whe NH is true i.e. whe θ 1 θ 2 θ 3 θ say the likelihood fuctio reduces to θ θ e θt 1+t 2 +t 3 ad Hece i.e. Cosequetly l θ log θ log θ θ t 1 + t 2 + t 3 l θ θ θ θ t 1 + t 2 + t 3 0 θ t 1 + t 2 + t 3 θ 1 θ 2 θ e t 1 + t 2 + t 3 ad 1 t 1 1 e 1 2 t 2 2 e e t 1 1 t 2 2 t t 3 3 e 3 Hece g.l.r.t. statistic is θ Λ x t 1 1 t 2 2 t t 1 + t 2 + t t1 t2 t3 t 1 2 3

10 4.5. ASYMPTOTIC FORM OF THE G..R.T. 91 where ad t t 1 + t 2 + t 3. The asymptotic form of the g.l.r.t. statistic is [ ] 2 log Λ x 2 l l θ [ ] t t1 t2 t3 2 log 1 log 2 log 3 log χ 2 2 whe the ull hypothesis is true Note that NH is specified i terms of two coditios sice θ 1 θ 2 θ 3 { θ1 θ 2 0 θ 1 θ 3 0 distributio. For the data give { log Λ 2 60 log 20 log } log 20 log Sice < χ 2 2; there is o evidece, at the 5% level, to reject the ull hypothesis. A importat example: Observatios fall idepedetly i oe of four categories C 1,C 2,c 3 ad C 4 with respective probabilities θ 1,θ 2,θ 3,θ 4 with θ 1 + θ 2 + θ 3 + θ 4 1. the followig hypothesis is put forward NH : θ 1 β 2, θ β1 β, θ β1 β, θ 4 1 β 2 with β 0.6 I a radom sample of 100 such observatios the umbers fallig i the four categories were x 1 10, x 2 13, x 3 37 ad x 4 40 Perform a test of approximate 5% level of sigificace to test this hypothesis agaist the alterative that at least oe of the equalities i NH does ot hold; show that there is evidece to reject NH.

11 92 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Solutio: Note that NH really says θ , θ , θ , θ Note also that sice the θ i s add up to 1, the last oe is determied as soo as we kow the first three θ i s; cosequetly NH ivolves oly 3 idepedet equatios/coditios. The likelihood of θ θ 1,θ 2,θ 3,θ 4 T for the result x i observatios i the ith category i 1, 2, 3, 4 whe are sampled is give by the multiomial probability where x 1 + x 2 + x 3 + x 4 ad! θ x 1!x 2!x 3!x 4! θx 1 1 θ x 2 2 θ x 3 3 θ x 4 4 θ x 1 1 θ x 2 2 θ x θ 1 θ 2 θ 3 x 4 lθ x 1 log θ 1 + x 2 log θ 2 + x 3 log θ 3 + x 4 log θ 1 θ 2 θ 3 + cost] Hece differetiatig w.r.t. θ i i 1, 2, 3 ad equatig to zero to obtai the m.l.e. we get i.e. lθ θ i 0 x i x 4 θ i 1 θ 1 θ 2 θ i 1, 2, 3 3 θ i x i x 4 θ 4 i 1, 2, 3, Sice 4 θ i 1 addig the four equatios above gives 1 x 1 + x 2 + x 3 + x 4 θ4 θ4 x 4 x θ 4 x 4 4 Replacig this i 4.21 we get the m.l.e. θ i x i i 1, 2, 3, 4 ad hece the maximised log-likelihood fuctio l x i log θ i + cost x i log xi + cost

12 4.5. ASYMPTOTIC FORM OF THE G..R.T. 93 Whe NH is true H 0 is a oe poit set, amely H 0 {θ 0.36, 0.12, 0.36, 0.16}. Hece θ 0.36, 0.12, 0.36, 0.16 π 1,π 2, π 3,π 5 ad the maximized log-likelihood uder the NH is l θ x i log π i + cost Therefore [ 2 log Λx 2[l l θ] 2 x i log Note that 2 x i log xi xi π i ] x i log π i χ 2 3 π i expected umber of observatios out of to fall i the ith category whe NH is true e i x i observed umber of observatios out of that fall i the ith category o i Thus the test statistic has the form 2 log Λx 2 o i log oi e i. whe NH is true For the give results 2 log Λx 2 10 log log + 37 log + 40 log > χ 2 3, There is, therefore, evidece at the 5% level of sigificace agaist the ull hypothesis which is therefore rejected.

13 94 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION b We ow chage the ull hypothesis to NH : θ 1 β 2, θ β1 β, θ β1 β, θ 4 1 β 2 with β uspecified i 0,1. Notice that this ull hypothesis is specified i terms of oly two idepedet equatios/coditios. As before the equatio for theta 4 is redudat; further, sice β θ 1, the oly idepedet equatios are θ θ1 1 θ 1, θ θ1 1 θ 1 Thus whe the ull hypothesis NH is true the g.l.r.t. statistic 2 log Λx χ 2 2. Now as i part a whe NH is ot true l x i log xi + cost Whe NH is true the likelihood is θ β! x 1!x 2!x 3!x 4! β2x 1 1 β1 β 2 β 2x 1+x 2 +x 3 1 β x 2+x 3 +2x 4 β N 1 β M x2 3 x3 β1 β 1 β 2x 4 2 where N 2x 1 + x 2 + x 3 ad M x 2 + x 3 + 2x 4. Thus whe NH is true the log-likelihood is ad i.e. the m.l.e. β of β satisfies lβ N log β + M log1 β + costat lβ β 0 N β M 1 β 0 N1 β M β 0 β N N + M N 2 For the give data β N

14 4.5. ASYMPTOTIC FORM OF THE G..R.T. 95 The restricted m.l.e. s of the θ i s whe NH is true are θ 1 β 2 π 1 β, θ2 1 2 β1 β π 2 β, θ β1 β π 3 β, θ4 1 β 2 π 4 β Thus l θ x i log θ i + cost x i log π i β + cost so that 2 log Λx 2[l l θ] x i log xi xi x i log π i β x i log π β Note agai that π i β estimated expected umber fallig i the ith category e i Thus the g.l.r.t. statistic still has the form For the give data 2 log Λx 2 o i log oi e , e , 2 e i e , e ad [ 2 log Λx 2 10 log 10 ] log log log < χ 2 2,

15 96 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Thus there is o evidece at the 5% level of sigificace to reject the ull hypothesis. This example ca be geeralized. Result: The multiomial test Suppose each observatio ca fall i oe of k categories C 1,C 2,...,C k with PrC i θ i, i 1, 2,...,k, k θ i 1, ad that observatios are take idepedetly with x i fallig i C i, i 1, 2,...,k, k x i. Thus the sample joit p.m.f. is f X x θ! x 1!x 2!...x k! θx i 1 θ x2 θ x k k We formulate the ull hypothesis that states that the θ i s follow the model NH : θ i π i β i 1, 2,...,k with the π i s give fuctios ivolvig a ukow s-dimesioal parameter β if o β parameter is ivolved i the model the s 0 ad the π is i NH are give values. For large, the asymptotic form of the g.l.r.t. statistic is 2 log Λx k χ 2 k 1 s xi x i log π i β k o i log if s 0 the π i β π i, the give umerical value i NH where β is the m.l.e. of β uder the ull hypothesis i.e β maximises! Q k x i! k [π iβ] x i. oi e i

Goodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen)

Goodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen) Goodess-of-Fit Tests ad Categorical Data Aalysis (Devore Chapter Fourtee) MATH-252-01: Probability ad Statistics II Sprig 2019 Cotets 1 Chi-Squared Tests with Kow Probabilities 1 1.1 Chi-Squared Testig................