of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell : If WX is a ubiased estimator of τθ, ϕt E[WX T] is a better ubiased estimator for a sufficiet statistic Uiqueess of MVUE : Theorem 739 - Best ubiased estimator is uique MVUE ad UE of zeros : Theorem 73 - Best ubiased estimator is ucorrelated with ay ubiased estimators of zero UMVE by complete sufficiet statistics : Theorem 733 - Ay fuctio of complete sufficiet statistic is the best ubiased estimator for its expected value How to get UMVUE Strategies to obtai best ubiased estimators: Coditio a simple ubiased estimator o complete sufficiet statistics Come up with a fuctio of sufficiet statistic whose expected value is τθ Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Bayesia Framework Bayesia Decisio Theory Prior distributio πθ Samplig distributio x θ f X x θ Joit distributio πθfx θ Margial distributio mx πθfx θdθ Posterior distributio πθ x f Xx θπθ mx Bayes Estimator is a posterior mea of θ : E[θ x] Loss Fuctio Lθ, ˆθ eg θ ˆθ Risk Fuctio is the average loss : Rθ, ˆθ E[Lθ, ˆθ θ] For squared error loss L θ ˆθ, the risk fuctio is MSE Bayes Risk is the average risk across all θ : E[Rθ, ˆθ πθ] Bayes Rule Estimator miimizes Bayes risk miimizes posterior expected loss Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 4 / 3
Asymptotics Hypothesis Testig Cosistecy Usig law of large umbers, show variace ad bias coverges to zero, for ay cotiuous mappig fuctio τ Asymptotic Normality Usig cetral limit theorem, Slutsky Theorem, ad Delta Method Asymptotic Relative Efficiecy AREV, W σ W /σ V Asymptotically Efficiet ARE with CR-boud of ubiased estimator of τθ is Asymptotic Efficiecy of MLE Theorem MLE is always asymptotically efficiet uder regularity coditio Type I error PrX R θ whe θ Ω Type II error PrX R θ whe θ Ω c Power fuctio βθ PrX R θ βθ represets Type I error uder H, ad power -Type II error uder H Size α test sup θ Ω βθ α Level α test sup θ Ω βθ α LRT λx Lˆθ x Lˆθ x rejects H whe λx c log λx log c c LRT based o sufficiet statistics LRT based o full data ad sufficiet statistics are idetical Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 5 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 6 / 3 UMP Asymptotic Tests ad p-values Ubiased Test βθ βθ for every θ Ω c ad θ Ω UMP Test βθ β θ for every θ Ω c ad β θ of every other test with a class of test C UMP level α Test UMP test i the class of all the level α test smallest Type II error give the upper boud of Type I error Neyma-Pearso For H : θ θ vs H : θ θ, a test with rejectio regio fx θ /fx θ > k is a UMP level α test for its size MLR gt θ /gt θ is a icreasig fuctio of t for every θ > θ Karli-Rabi If T is sufficiet ad has MLR, the test rejectig R {T : T > t } or R {T : T < t } is a UMP level α test for oe-sided composite hypothesis Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 7 / 3 Asymptotic Distributio of LRT For testig, H : θ θ vs H : θ θ, d log λx χ uder regularity coditio Wald Test If W is a cosistet estimator of θ, ad S is a cosistet estimator of VarW, the Z W θ /S follows a stadard ormal distributio Two-sided test : Z > z α/ Oe-sided test : Z > z α/ or Z < z α/ p-value A p-value px is valid if, PrpX α θ α for every θ Ω ad α Costructig p-value Theorem 837 : If large WX value gives evidece that H is true, px sup θ Ω PrWX Wx θ is a valid p-value p-value give sufficiet statistics For a sufficiet statistic SX, px PrWX Wx SX Sx is also a valid p-value Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 8 / 3
Iterval Estimatio Practice Problem cotiued from last week Coverage probability Prθ [LX, UX] Coverage coefficiet is α if if θ Ω Prθ [LX, UX] α Cofidece iterval [LX, UX] is α if if θ Ω Prθ [LX, UX] α Ivertig a level α test If Aθ is the acceptace regio of a level α test, the CX {θ : X Aθ} is a α cofidece set or iterval Problem Let fx θ be the logistic locatio pdf fx θ e x θ + e x θ < x <, < θ < a Show that this family has a MLR b Based o oe observatio X, fid the most powerful size α test of H : θ versus H : θ c Show that the test i part b is UMP size α for testig H : θ vs H : θ > Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 9 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Solutio for a Solutio for b For θ < θ, fx θ fx θ Let rx + e x θ / + e x θ e x θ +e x θ e x θ +e x θ e θ θ + e x θ + e x θ r x ex θ + e x θ + e x θ e x θ + e x θ ex θ e x θ + e x θ > x θ > x θ Therefore, the family of X has a MLR Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 The UMP test rejects H if ad oly if fx + e x e fx + e x > k + e x + e x > k + e x e + e x > k X > x Because uder H, Fx θ ex +e, the rejectio regio of UMP level x α test satisfies Fx θ + e x α α x log α Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3
Solutio for c Practice Problem Because the family of X has a MLR, UMP size α for testig H : θ vs H : θ > should be a form of X > x PrX > x θ α Therefore, x log α α, which is idetical to the test defied i b Problem Suppose X,, X are iid radom samples with pdf f X x θ θ exp θx, where x, θ > a Show that b Show that distributio x X i x X i is a cosistet estimator for θ is asymptotically ormal ad derive its asymptotic c Derive the Wald asymptotic size α test for H : θ θ vs H : θ θ d Fid a asymptotic α cofidece iterval for θ by ivertig the above test You may use the fact that EX /θ ad VarX /θ Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 4 / 3 Solutio a - Cosistecy Solutio b - Asymptotic Distributio Obtai EX /θ Derive yourself if ot give EX xfx θdx θx exp θxdx [ x exp θx] + exp θxdx [ + ] θ exp θx θ By LLN Law of Large Number, X P EX /θ 3 By Theorem of cotiuous map, / i X i /X P θ Obtai VarX /θ Derive if eeded, omitted here Apply CLTCetral Limit Theorem, X AN θ, θ 3 Apply Delta method Let gy /y, the g y /y Xi /X gx AN g/θ, [g /θ] θ AN θ, θ X θ N, θ Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 5 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 6 / 3
Solutio c - Wald asymptotic size α test Solutio c - Wald Asymptotic size α test cot d Obtai a cosistet estimator of θ : i WX X i Obtai a costat estimator of VarW S AN X i X P VarX θ i i X i X i X i X P θ P θ θ, θ CLT Cotiuous Map Theorem Slutsky s Theorem 3 Costruct a two-sided asymptotic size α Wald test, whose rejectio regio is ZX WX θ S/ i X θ i i X i X X θ X i X z α/ i Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 7 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 8 / 3 Solutio d - Asymptotic α cofidece iterval Practice Problem 3 The acceptace regio is A x : x θ x i x z α/ By ivertig the acceptace regio, the cofidece iterval is CX θ : X θ X i X z α/ which is equivalet to CX θ i i X z α/ i X i X, X + z α/ i X i X Problem The idepedet radom variables X,, X have the followig pdf fx θ, β βxβ θ β < x < θ, β > Fid the MLEs of β ad θ Whe β is a kow costat β, costruct a LRT testig H : θ θ vs H : θ < θ 3 Whe β is a kow costat β, fid the upper cofidece limit for θ with cofidece coefficiet α Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 9 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3
a - MLE b - LRT Lθ, β x β i x i β θ β Ix θ Because L is a decreasig fuctio of θ ad positive oly whe θ x ˆθ x lθ, β x log β + β log x i β log θ l β β + log x i log θ ˆβ log ˆθ log x i x log x i λx sup θ Ω Lˆθ x sup θ Ω Lˆθ x { θ < x Lθ x Lx x θ x θ < x x θ c x β θ β θ x c Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 b - size α LRT c - Upper α cofidece limit x α Pr θ c β c α β c Therefore, the rejectio regio for size α LRT is is } R {x : x θ α β The acceptace regio of size α LRT is } Aθ {x : x > θ α β By isertig the acceptace regio, the α cofidece iterval becomes } CX {θ : X > θα β } {θ : θ < X α β Therefore, the upper α cofidece limit is X α β Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 4 / 3
Practice Problem 4 a - MLE of θ Problem A radom sample X,, X is draw from a populatio N θ, θ where θ > a Fid the ˆθ, the MLE of θ b Fid the asymptotic distributio of ˆθ c Compute AREˆθ, X Determie whether ˆθ is asymptotically more efficiet tha X or ot You may use the followig fact: VarX 4θ 3 + θ Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 5 / 3 [ Lθ x πθ / i exp x i θ ] θ lθ x logπ + log θ i x i θ θ logπ + x log θ i θ + x i θ l θ x x θ + i θ θ x i θ θ θ + θ x i ˆθ + + 4 x i / x i ˆθ + ˆθ Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 6 / 3 b - Asymptotic distributio of MLE b - Asymptotic distributio of MLE cot d By CLT, Let W X i, the W AN EX, VarX The asymptotic distributio of MLE ˆθ ˆθ AN θ, σ θ AN θ + θ, 4θ3 + θ for some fuctio σ θ ad we would like to fid σ θ usig the asymptotic distributio of W Let gy y + y, the g y y + ad gˆθ W The by the Delta Method, the asymptotic distributio of W ca be writte as W gˆθ AN gθ, g θ σ θ AN θ + θ, θ + σ θ AN θ + θ, 4θ3 + θ σ θ 4θ3 + θ θ + θ θ + θ + θ θ + Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 7 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 8 / 3
b - Asymptotic distributio of MLE cot d c - ARE of MLE compared to X The asymptotic distributio of MLE ˆθ ˆθ AN θ, σ θ θ AN θ, θ + Note that you caot use CR-boud for the asymptotic variace of MLE because the regularity coditio does ot hold ope set criteria By CLT, the asymptotic distributio of X is X AN θ, θ The, AREˆθ, X is AREˆθ, X θ θ θ+ θ + + θ θ > Therefore, ˆθ is more efficiet estimator tha X Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 9 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 3 / 3 Wrappig Up May thaks for your attetios ad feedbacks Please complete your teachig evaluatios, which will be very helpful for further improvemet i the ext year 3 Fial exam will be Thursday April 5th, 4:-6:pm 4 The last office hour will be held Wedesday April 4th, 4:-5:pm 5 The grade will be posted durig the weeked 6 Do t forget the materials we have leared, because they are the key topics for your cadidacy exam Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 3 / 3