Last Lecture Biostatistics 6 - Statistical Iferece Lecture Uiformly Most Powerful Test Hyu Mi Kag March 8th, 3 What are the typical steps for costructig a likelihood ratio test? Is LRT statistic based o sufficiet statistic idetical to the LRT based o the full data? Whe multiple parameters eed to be estimated, what is the differece i costructig LRT? What is ubiased test? Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 / 3 LRT based o sufficiet statistics Ubiased Test Theorem 84 If TX is a sufficiet statistic for θ, λ t is the LRT statistic based o T, ad λx is the LRT statistic based o x the λ [Tx] λx for every x i the sample space Defiitio If a test always satisfies Prreject H whe H is false Prreject H whe H is true The the test is said to be ubiased Alterative Defiitio Recall that βθ reject H A test is ubiased if βθ βθ for every θ Ω c ad θ Ω Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 4 / 3
Example Example cot d X,, X iid N θ, σ where σ is kow, testig H : θ θ vs H : θ > θ LRT test rejects H if x θ > c X θ βθ > c X θ + θ θ X θ > c + θ θ > c X θ > c θ θ Therefore, for Z N, βθ Z > c + θ θ Because the power fuctio is icreasig fuctio of θ, βθ βθ always holds whe θ θ < θ Therefore the LRTs are ubiased Note that X i N θ, σ, X N θ, σ /, ad X θ σ/ N, Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 5 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 6 / 3 Uiformly Most Powerful Test level α test Defiitio Let C be a class of tests betwee H : θ Ω vs H : θ Ω c A test i C, with power fuctio βθ is uiformly most powerful test i class C if βθ β θ for every θ Ω c ad every β θ, which is a power fuctio of aother test i C Cosider C be the set of all the level α test The test i this class is called a level α test level α test has the smallest type II error probability for ay θ Ω c i this class A test is uiform i the sese that it is most powerful for every θ Ω c For simple hypothesis such as H : θ θ ad H : θ θ, level α test always exists Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 7 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 8 / 3
Lemma Theorem 83 - Lemma Cosider testig H : θ θ vs H : θ θ where the pdf or pmf correspodig the θ i is fx θ i, i,, usig a test with rejectio regio R that satisfies x R if fx θ > kfx θ 83 ad x R c if fx θ < kfx θ 83 For some k ad α X R θ, The, Sufficiecy Ay test that satisfies 83 ad 83 is a level α test Necessity if there exist a test satisfyig 83 ad 83 with k >, the every level α test is a size α test satisfies 83, ad every level α test satisfies 83 except perhaps o a set A satisfyig PrX A θ X A θ Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 9 / 3 Example of Lemma cot d Suppose that 3/4 < k < 9/4, the level α test rejects H if x α reject θ / x θ / 4 If k > 9/4 the level α test always ot reject H, ad α Example of Lemma Let X Biomial, θ, ad cosider testig H : θ θ / vs H : θ θ 3/4 Calculatig the ratios of the pmfs give, f θ f θ 4, f θ f θ 3 4, f θ f θ 9 4 Suppose that k < /4, the the rejectio regio R,,, ad level α test always rejects H Therefore α reject H θ θ / Suppose that /4 < k < 3/4, the R,, ad level α test rejects H if x or x α reject θ / x θ / + Prx θ / 3 4 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 / 3 Example - Normal Distributio iid X i N θ, σ where σ is kow Cosider testig H : θ θ vs H : θ θ where θ > θ [ fx θ πσ exp x i θ ] fx θ fx θ x i θ exp exp x i θ σ [ exp x i θ + x i θ ] [ exp x i θ x i θ ] [ θ exp θ + x ] iθ θ Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 / 3
Example cot d Example cot d level α test rejects if [ θ exp θ + x ] iθ θ > k θ θ + x iθ θ > log k x i > k α X i > k θ Uder H, where Z N, X θ X i N θ, σ X N θ, σ / N, α X i > k θ Z > k / θ Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 4 / 3 Example cot d Lemma o Sufficiet Statistics k / θ σ/ z α k σ θ + z α Thus, the level α test reject if X i > k, or equivaletly, reject H if X > k / θ + z α Corollary 833 Cosider H : θ θ vs H : θ θ Suppose TX is a sufficiet statistic for θ ad gt θ i is the pdf or pmf of T Correspodig θ i, i, The ay test based o T with rejectio regio S is a level α test if it satisfies t S if gt θ > k gt θ ad t S c if gt θ < k gt θ For some k > ad α T S θ Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 5 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 6 / 3
Proof Revisitig the Example of Normal Distributio The rejectio regio i the sample space is R x : Tx t S x : gtx θ > kgtx θ By Factorizatio Theorem: fx θ i hxgtx θ i R x : gtx θ hx > kgtx θ hx x : fx θ > kfx θ By Lemma, this test is the level α test, ad α X R TX S θ X i iid N θ, σ where σ is kow Cosider testig H : θ θ vs H : θ θ where θ > θ T X is a sufficiet statistic for θ, where T N θ, σ / gt θ i gt θ gt θ πσ / exp t θ i / exp t θ σ / exp t θ / exp [ t θ t θ ] / exp [ θ / θ tθ θ ] Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 7 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 8 / 3 Revisitig the Example cot d Revisitig the Example cot d Uder H, X N θ, σ / k satisfies level α test reject if exp [ θ / θ tθ θ ] > k [ θ / θ + tθ θ ] > log k X t > k Prreject H θ α k θ α X > k θ X θ > k θ Z > k θ z α k θ + z α σ Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 9 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 / 3
Mootoe Likelihood Ratio Example of Mootoe Likelihood Ratio Defiitio A family of pdfs or pmfs gt θ : θ Ω for a uivariate radom variable T with real-valued parameter θ have a mootoe likelihood ratio if gt θ gt θ is a icreasig or o-decreasig fuctio of t for every θ > θ o t : gt θ > or gt θ > Note: we may defie MLR usig decreasig fuctio of t But all followig theorems are stated accordig to the defiitio Normal, Poisso, Biomial have the MLR Property Exercise 85 If T is from a expoetial family with the pdf or pmf gt θ htcθ exp[wθ t] The T has a MLR if wθ is a o-decreasig fuctio of θ Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 / 3 Proof Theorem Suppose that θ > θ gt θ gt θ htcθ exp[wθ t] htcθ exp[wθ t] cθ cθ exp[wθ wθ t] If wθ is a o-decreasig fuctio of θ, the wθ wθ ad exp[wθ wθ t] is a icreasig fuctio of t Therefore, gt θ gt θ is a o-decreasig fuctio of t, ad T has MLR if wθ is a o-decreasig fuctio of θ Theorem 87 Suppose TX is a sufficiet statistic for θ ad the family gt θ : θ Ω is a MLR family The For testig H : θ θ vs H : θ > θ, the level α test is give by rejectig H is ad oly if T > t where α T > t θ For testig H : θ θ vs H : θ < θ, the level α test is give by rejectig H if ad oly if T < t where α T < t θ Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 4 / 3
Example Applicatio of Theorem Fidig a level α test Let X i iid N θ, σ where σ is kow, Fid the level α test for H : θ θ vs H : θ > θ TX X is a sufficiet statistic for θ, ad T N θ, σ / gt θ where wθ property πσ / exp t θ / πσ / exp t + θ tθ / πσ / exp htcθ exp[wθt] θ σ / t / exp θ tθ exp / σ / is a icreasig fuctio i θ Therefore T is MLR Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 5 / 3 By, level α test rejects H iff T > t where where Z N, α T > t θ T θ > t θ Z > t t z α t θ + σ z α level α test rejects H if T X > θ + σ z α Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 6 / 3 Testig H : θ θ vs H : θ < θ level α test rejects H if T < t where T θ α T < t θ < t θ Z < t α Z t t σ/ z α t θ + σ z α θ σ z α Therefore, the test rejects H if T < t θ σ z α Normal Example with Kow Mea iid X i N µ, σ where σ is ukow ad µ is kow Fid the level α test for testig H : σ σ vs H : σ > σ Let T X i µ is sufficiet for σ To check whether T has MLR property, we eed to fid gt σ X i µ N, σ Xi µ χ σ Y T/σ f Y y Xi µ χ σ Γ / y e y Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 7 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 8 / 3
Normal Example with Kow Mea cot d Normal Example with Kow Mea cot d f T t t Γ / σ t Γ / σ t σ Γ / htcσ exp[wσ t] dt e t dy e t σ e t where wσ is a icreasig fuctio i σ Therefore, T X i µ has the MLR property Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 9 / 3 By Theorem, level α rejects s H if ad oly if T > t where t is chose such that α T > t σ Note that T χ σ T PrT > t σ > t Pr χ > t T σ σ t σ χ α χ,α σ t σ χ,α where χ,α satisfies χ f χ,α xdx α σ σ Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 3 / 3 Remarks For may problems, level α test does ot exist Example 839 I such cases, we ca restrict our search amog a subset of tests, for example, all ubiased tests Today Uiformly Most Powerful Test Lemma Mootoe Likelihood Ratio Theorem Next Lecture Asymptotics of LRT Wald Test Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture March 8th, 3 3 / 3