# Asymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values

Size: px
Start display at page:

Download "Asymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values"

## Transcription

1 of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell : If WX is a ubiased estimator of τθ, ϕt E[WX T] is a better ubiased estimator for a sufficiet statistic Uiqueess of MVUE : Theorem Best ubiased estimator is uique MVUE ad UE of zeros : Theorem 73 - Best ubiased estimator is ucorrelated with ay ubiased estimators of zero UMVE by complete sufficiet statistics : Theorem Ay fuctio of complete sufficiet statistic is the best ubiased estimator for its expected value How to get UMVUE Strategies to obtai best ubiased estimators: Coditio a simple ubiased estimator o complete sufficiet statistics Come up with a fuctio of sufficiet statistic whose expected value is τθ Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Bayesia Framework Bayesia Decisio Theory Prior distributio πθ Samplig distributio x θ f X x θ Joit distributio πθfx θ Margial distributio mx πθfx θdθ Posterior distributio πθ x f Xx θπθ mx Bayes Estimator is a posterior mea of θ : E[θ x] Loss Fuctio Lθ, ˆθ eg θ ˆθ Risk Fuctio is the average loss : Rθ, ˆθ E[Lθ, ˆθ θ] For squared error loss L θ ˆθ, the risk fuctio is MSE Bayes Risk is the average risk across all θ : E[Rθ, ˆθ πθ] Bayes Rule Estimator miimizes Bayes risk miimizes posterior expected loss Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 4 / 3

3 Iterval Estimatio Practice Problem cotiued from last week Coverage probability Prθ [LX, UX] Coverage coefficiet is α if if θ Ω Prθ [LX, UX] α Cofidece iterval [LX, UX] is α if if θ Ω Prθ [LX, UX] α Ivertig a level α test If Aθ is the acceptace regio of a level α test, the CX {θ : X Aθ} is a α cofidece set or iterval Problem Let fx θ be the logistic locatio pdf fx θ e x θ + e x θ < x <, < θ < a Show that this family has a MLR b Based o oe observatio X, fid the most powerful size α test of H : θ versus H : θ c Show that the test i part b is UMP size α for testig H : θ vs H : θ > Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 9 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Solutio for a Solutio for b For θ < θ, fx θ fx θ Let rx + e x θ / + e x θ e x θ +e x θ e x θ +e x θ e θ θ + e x θ + e x θ r x ex θ + e x θ + e x θ e x θ + e x θ ex θ e x θ + e x θ > x θ > x θ Therefore, the family of X has a MLR Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 The UMP test rejects H if ad oly if fx + e x e fx + e x > k + e x + e x > k + e x e + e x > k X > x Because uder H, Fx θ ex +e, the rejectio regio of UMP level x α test satisfies Fx θ + e x α α x log α Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3

4 Solutio for c Practice Problem Because the family of X has a MLR, UMP size α for testig H : θ vs H : θ > should be a form of X > x PrX > x θ α Therefore, x log α α, which is idetical to the test defied i b Problem Suppose X,, X are iid radom samples with pdf f X x θ θ exp θx, where x, θ > a Show that b Show that distributio x X i x X i is a cosistet estimator for θ is asymptotically ormal ad derive its asymptotic c Derive the Wald asymptotic size α test for H : θ θ vs H : θ θ d Fid a asymptotic α cofidece iterval for θ by ivertig the above test You may use the fact that EX /θ ad VarX /θ Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 4 / 3 Solutio a - Cosistecy Solutio b - Asymptotic Distributio Obtai EX /θ Derive yourself if ot give EX xfx θdx θx exp θxdx [ x exp θx] + exp θxdx [ + ] θ exp θx θ By LLN Law of Large Number, X P EX /θ 3 By Theorem of cotiuous map, / i X i /X P θ Obtai VarX /θ Derive if eeded, omitted here Apply CLTCetral Limit Theorem, X AN θ, θ 3 Apply Delta method Let gy /y, the g y /y Xi /X gx AN g/θ, [g /θ] θ AN θ, θ X θ N, θ Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 5 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 6 / 3

5 Solutio c - Wald asymptotic size α test Solutio c - Wald Asymptotic size α test cot d Obtai a cosistet estimator of θ : i WX X i Obtai a costat estimator of VarW S AN X i X P VarX θ i i X i X i X i X P θ P θ θ, θ CLT Cotiuous Map Theorem Slutsky s Theorem 3 Costruct a two-sided asymptotic size α Wald test, whose rejectio regio is ZX WX θ S/ i X θ i i X i X X θ X i X z α/ i Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 7 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 8 / 3 Solutio d - Asymptotic α cofidece iterval Practice Problem 3 The acceptace regio is A x : x θ x i x z α/ By ivertig the acceptace regio, the cofidece iterval is CX θ : X θ X i X z α/ which is equivalet to CX θ i i X z α/ i X i X, X + z α/ i X i X Problem The idepedet radom variables X,, X have the followig pdf fx θ, β βxβ θ β < x < θ, β > Fid the MLEs of β ad θ Whe β is a kow costat β, costruct a LRT testig H : θ θ vs H : θ < θ 3 Whe β is a kow costat β, fid the upper cofidece limit for θ with cofidece coefficiet α Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 9 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3

6 a - MLE b - LRT Lθ, β x β i x i β θ β Ix θ Because L is a decreasig fuctio of θ ad positive oly whe θ x ˆθ x lθ, β x log β + β log x i β log θ l β β + log x i log θ ˆβ log ˆθ log x i x log x i λx sup θ Ω Lˆθ x sup θ Ω Lˆθ x { θ < x Lθ x Lx x θ x θ < x x θ c x β θ β θ x c Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 b - size α LRT c - Upper α cofidece limit x α Pr θ c β c α β c Therefore, the rejectio regio for size α LRT is is } R {x : x θ α β The acceptace regio of size α LRT is } Aθ {x : x > θ α β By isertig the acceptace regio, the α cofidece iterval becomes } CX {θ : X > θα β } {θ : θ < X α β Therefore, the upper α cofidece limit is X α β Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 3 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 4 / 3

7 Practice Problem 4 a - MLE of θ Problem A radom sample X,, X is draw from a populatio N θ, θ where θ > a Fid the ˆθ, the MLE of θ b Fid the asymptotic distributio of ˆθ c Compute AREˆθ, X Determie whether ˆθ is asymptotically more efficiet tha X or ot You may use the followig fact: VarX 4θ 3 + θ Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 5 / 3 [ Lθ x πθ / i exp x i θ ] θ lθ x logπ + log θ i x i θ θ logπ + x log θ i θ + x i θ l θ x x θ + i θ θ x i θ θ θ + θ x i ˆθ x i / x i ˆθ + ˆθ Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 6 / 3 b - Asymptotic distributio of MLE b - Asymptotic distributio of MLE cot d By CLT, Let W X i, the W AN EX, VarX The asymptotic distributio of MLE ˆθ ˆθ AN θ, σ θ AN θ + θ, 4θ3 + θ for some fuctio σ θ ad we would like to fid σ θ usig the asymptotic distributio of W Let gy y + y, the g y y + ad gˆθ W The by the Delta Method, the asymptotic distributio of W ca be writte as W gˆθ AN gθ, g θ σ θ AN θ + θ, θ + σ θ AN θ + θ, 4θ3 + θ σ θ 4θ3 + θ θ + θ θ + θ + θ θ + Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 7 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 8 / 3

8 b - Asymptotic distributio of MLE cot d c - ARE of MLE compared to X The asymptotic distributio of MLE ˆθ ˆθ AN θ, σ θ θ AN θ, θ + Note that you caot use CR-boud for the asymptotic variace of MLE because the regularity coditio does ot hold ope set criteria By CLT, the asymptotic distributio of X is X AN θ, θ The, AREˆθ, X is AREˆθ, X θ θ θ+ θ + + θ θ > Therefore, ˆθ is more efficiet estimator tha X Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 9 / 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 3 / 3 Wrappig Up May thaks for your attetios ad feedbacks Please complete your teachig evaluatios, which will be very helpful for further improvemet i the ext year 3 Fial exam will be Thursday April 5th, 4:-6:pm 4 The last office hour will be held Wedesday April 4th, 4:-5:pm 5 The grade will be posted durig the weeked 6 Do t forget the materials we have leared, because they are the key topics for your cadidacy exam Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 3 / 3

### Summary. Recap ... Last Lecture. Summary. Theorem

Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca

### Last Lecture. Wald Test

Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig

### Summary. Recap. Last Lecture. Let W n = W n (X 1,, X n ) = W n (X) be a sequence of estimators for

Last Lecture Biostatistics 602 - Statistical Iferece Lecture 17 Asymptotic Evaluatio of oit Estimators Hyu Mi Kag March 19th, 2013 What is a Bayes Risk? What is the Bayes rule Estimator miimizig square

### Last Lecture. Biostatistics Statistical Inference Lecture 16 Evaluation of Bayes Estimator. Recap - Example. Recap - Bayes Estimator

Last Lecture Biostatistics 60 - Statistical Iferece Lecture 16 Evaluatio of Bayes Estimator Hyu Mi Kag March 14th, 013 What is a Bayes Estimator? Is a Bayes Estimator the best ubiased estimator? Compared

### Last Lecture. Unbiased Test

Last Lecture Biostatistics 6 - Statistical Iferece Lecture Uiformly Most Powerful Test Hyu Mi Kag March 8th, 3 What are the typical steps for costructig a likelihood ratio test? Is LRT statistic based

### Introductory statistics

CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key

### Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara

Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio

### Lecture Notes 15 Hypothesis Testing (Chapter 10)

1 Itroductio Lecture Notes 15 Hypothesis Testig Chapter 10) Let X 1,..., X p θ x). Suppose we we wat to kow if θ = θ 0 or ot, where θ 0 is a specific value of θ. For example, if we are flippig a coi, we

### Direction: This test is worth 250 points. You are required to complete this test within 50 minutes.

Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely

### Topic 9: Sampling Distributions of Estimators

Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

### LECTURE 14 NOTES. A sequence of α-level tests {ϕ n (x)} is consistent if

LECTURE 14 NOTES 1. Asymptotic power of tests. Defiitio 1.1. A sequece of -level tests {ϕ x)} is cosistet if β θ) := E θ [ ϕ x) ] 1 as, for ay θ Θ 1. Just like cosistecy of a sequece of estimators, Defiitio

### Lecture 11 and 12: Basic estimation theory

Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis

### Problem Set 4 Due Oct, 12

EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

### Topic 9: Sampling Distributions of Estimators

Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

### Topic 9: Sampling Distributions of Estimators

Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

### Lecture 33: Bootstrap

Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece

### Lecture 12: September 27

36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.

### MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4

MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.

### 2. The volume of the solid of revolution generated by revolving the area bounded by the

IIT JAM Mathematical Statistics (MS) Solved Paper. A eigevector of the matrix M= ( ) is (a) ( ) (b) ( ) (c) ( ) (d) ( ) Solutio: (a) Eigevalue of M = ( ) is. x So, let x = ( y) be the eigevector. z (M

### STATISTICAL INFERENCE

STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample

### Stat410 Probability and Statistics II (F16)

Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems

### Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.

Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized

### Stat 319 Theory of Statistics (2) Exercises

Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.

### Introduction to Econometrics (3 rd Updated Edition) Solutions to Odd- Numbered End- of- Chapter Exercises: Chapter 3

Itroductio to Ecoometrics (3 rd Updated Editio) by James H. Stock ad Mark W. Watso Solutios to Odd- Numbered Ed- of- Chapter Exercises: Chapter 3 (This versio August 17, 014) 015 Pearso Educatio, Ic. Stock/Watso

### Direction: This test is worth 150 points. You are required to complete this test within 55 minutes.

Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem

### Exam II Review. CEE 3710 November 15, /16/2017. EXAM II Friday, November 17, in class. Open book and open notes.

Exam II Review CEE 3710 November 15, 017 EXAM II Friday, November 17, i class. Ope book ad ope otes. Focus o material covered i Homeworks #5 #8, Note Packets #10 19 1 Exam II Topics **Will emphasize material

### Random Variables, Sampling and Estimation

Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig

### Unbiased Estimation. February 7-12, 2008

Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom

### This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.

Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the

### Statistical Theory MT 2009 Problems 1: Solution sketches

Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where

### 5. Likelihood Ratio Tests

1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,

### Lecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett

Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets

### Lecture 16: UMVUE: conditioning on sufficient and complete statistics

Lecture 16: UMVUE: coditioig o sufficiet ad complete statistics The 2d method of derivig a UMVUE whe a sufficiet ad complete statistic is available Fid a ubiased estimator of ϑ, say U(X. Coditioig o a

### This section is optional.

4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore

### 1.010 Uncertainty in Engineering Fall 2008

MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval

### Statistical Theory MT 2008 Problems 1: Solution sketches

Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α

### MATH 472 / SPRING 2013 ASSIGNMENT 2: DUE FEBRUARY 4 FINALIZED

MATH 47 / SPRING 013 ASSIGNMENT : DUE FEBRUARY 4 FINALIZED Please iclude a cover sheet that provides a complete setece aswer to each the followig three questios: (a) I your opiio, what were the mai ideas

### Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.

Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator

### ( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2

82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1,

### Estimation for Complete Data

Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of

### x = Pr ( X (n) βx ) =

Exercise 93 / page 45 The desity of a variable X i i 1 is fx α α a For α kow let say equal to α α > fx α α x α Pr X i x < x < Usig a Pivotal Quatity: x α 1 < x < α > x α 1 ad We solve i a similar way as

### Parameter, Statistic and Random Samples

Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,

### of the matrix is =-85, so it is not positive definite. Thus, the first

BOSTON COLLEGE Departmet of Ecoomics EC771: Ecoometrics Sprig 4 Prof. Baum, Ms. Uysal Solutio Key for Problem Set 1 1. Are the followig quadratic forms positive for all values of x? (a) y = x 1 8x 1 x

### First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise

First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >

### Probability and Statistics

ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,

### Lecture 7: Properties of Random Samples

Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ

### Lecture 6 Simple alternatives and the Neyman-Pearson lemma

STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull

### Efficient GMM LECTURE 12 GMM II

DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet

### IIT JAM Mathematical Statistics (MS) 2006 SECTION A

IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim

### EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

### Exponential Families and Bayesian Inference

Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where

### An Introduction to Asymptotic Theory

A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu

### 10-701/ Machine Learning Mid-term Exam Solution

0-70/5-78 Machie Learig Mid-term Exam Solutio Your Name: Your Adrew ID: True or False (Give oe setece explaatio) (20%). (F) For a cotiuous radom variable x ad its probability distributio fuctio p(x), it

### MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND.

XI-1 (1074) MOST PEOPLE WOULD RATHER LIVE WITH A PROBLEM THEY CAN'T SOLVE, THAN ACCEPT A SOLUTION THEY CAN'T UNDERSTAND. R. E. D. WOOLSEY AND H. S. SWANSON XI-2 (1075) STATISTICAL DECISION MAKING Advaced

### Composite Hypotheses

Composite Hypotheses March 25-27, 28 For a composite hypothesis, the parameter space Θ is divided ito two disjoit regios, Θ ad Θ 1. The test is writte H : Θ versus H 1 : Θ 1 with H is called the ull hypothesis

### Sample Size Determination (Two or More Samples)

Sample Sie Determiatio (Two or More Samples) STATGRAPHICS Rev. 963 Summary... Data Iput... Aalysis Summary... 5 Power Curve... 5 Calculatios... 6 Summary This procedure determies a suitable sample sie

### Simulation. Two Rule For Inverting A Distribution Function

Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump

### Large Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution

Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,

### Statistical inference: example 1. Inferential Statistics

Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either

### Stat 421-SP2012 Interval Estimation Section

Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible

### Chapter 6 Principles of Data Reduction

Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a

### Kurskod: TAMS11 Provkod: TENB 21 March 2015, 14:00-18:00. English Version (no Swedish Version)

Kurskod: TAMS Provkod: TENB 2 March 205, 4:00-8:00 Examier: Xiagfeg Yag (Tel: 070 2234765). Please aswer i ENGLISH if you ca. a. You are allowed to use: a calculator; formel -och tabellsamlig i matematisk

### Frequentist Inference

Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for

### TAMS24: Notations and Formulas

TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =

### Homework for 2/3. 1. Determine the values of the following quantities: a. t 0.1,15 b. t 0.05,15 c. t 0.1,25 d. t 0.05,40 e. t 0.

Name: ID: Homework for /3. Determie the values of the followig quatities: a. t 0.5 b. t 0.055 c. t 0.5 d. t 0.0540 e. t 0.00540 f. χ 0.0 g. χ 0.0 h. χ 0.00 i. χ 0.0050 j. χ 0.990 a. t 0.5.34 b. t 0.055.753

### STAT Homework 7 - Solutions

STAT-36700 Homework 7 - Solutios Fall 208 October 28, 208 This cotais solutios for Homework 7. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better

### SDS 321: Introduction to Probability and Statistics

SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/

### Properties and Hypothesis Testing

Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.

### 1. Parameter estimation point estimation and interval estimation. 2. Hypothesis testing methods to help decision making.

Chapter 7 Parameter Estimatio 7.1 Itroductio Statistical Iferece Statistical iferece helps us i estimatig the characteristics of the etire populatio based upo the data collected from (or the evidece 0produced

### Since X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain

Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the

### Lecture 19: Convergence

Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

### MBACATÓLICA. Quantitative Methods. Faculdade de Ciências Económicas e Empresariais UNIVERSIDADE CATÓLICA PORTUGUESA 9. SAMPLING DISTRIBUTIONS

MBACATÓLICA Quatitative Methods Miguel Gouveia Mauel Leite Moteiro Faculdade de Ciêcias Ecoómicas e Empresariais UNIVERSIDADE CATÓLICA PORTUGUESA 9. SAMPLING DISTRIBUTIONS MBACatólica 006/07 Métodos Quatitativos

### Logit regression Logit regression

Logit regressio Logit regressio models the probability of Y= as the cumulative stadard logistic distributio fuctio, evaluated at z = β 0 + β X: Pr(Y = X) = F(β 0 + β X) F is the cumulative logistic distributio

### Solutions: Homework 3

Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe

### Point Estimation: properties of estimators 1 FINITE-SAMPLE PROPERTIES. finite-sample properties (CB 7.3) large-sample properties (CB 10.

Poit Estimatio: properties of estimators fiite-sample properties CB 7.3) large-sample properties CB 10.1) 1 FINITE-SAMPLE PROPERTIES How a estimator performs for fiite umber of observatios. Estimator:

### November 2002 Course 4 solutions

November Course 4 solutios Questio # Aswer: B φ ρ = = 5. φ φ ρ = φ + =. φ Solvig simultaeously gives: φ = 8. φ = 6. Questio # Aswer: C g = [(.45)] = [5.4] = 5; h= 5.4 5 =.4. ˆ π =.6 x +.4 x =.6(36) +.4(4)

### Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.

Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,

### Lecture 8: Convergence of transformations and law of large numbers

Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges

### EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY

EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA, 016 MODULE : Statistical Iferece Time allowed: Three hours Cadidates should aswer FIVE questios. All questios carry equal marks. The umber

### Mathematical Statistics - MS

Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios

### Machine Learning Brett Bernstein

Machie Learig Brett Berstei Week Lecture: Cocept Check Exercises Starred problems are optioal. Statistical Learig Theory. Suppose A = Y = R ad X is some other set. Furthermore, assume P X Y is a discrete

### The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.

SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample

### STA 4032 Final Exam Formula Sheet

Chapter 2. Probability STA 4032 Fial Eam Formula Sheet Some Baic Probability Formula: (1) P (A B) = P (A) + P (B) P (A B). (2) P (A ) = 1 P (A) ( A i the complemet of A). (3) If S i a fiite ample pace

### LECTURE 8: ASYMPTOTICS I

LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece

### Bayesian Methods: Introduction to Multi-parameter Models

Bayesia Methods: Itroductio to Multi-parameter Models Parameter: θ = ( θ, θ) Give Likelihood p(y θ) ad prior p(θ ), the posterior p proportioal to p(y θ) x p(θ ) Margial posterior ( θ, θ y) is Iterested

### Lecture 2: Monte Carlo Simulation

STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?

### Math 152. Rumbos Fall Solutions to Review Problems for Exam #2. Number of Heads Frequency

Math 152. Rumbos Fall 2009 1 Solutios to Review Problems for Exam #2 1. I the book Experimetatio ad Measuremet, by W. J. Youde ad published by the by the Natioal Sciece Teachers Associatio i 1962, the

### EFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS

EFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS Ryszard Zieliński Ist Math Polish Acad Sc POBox 21, 00-956 Warszawa 10, Polad e-mail: rziel@impagovpl ABSTRACT Weak laws of large umbers (W LLN), strog

### 6. Sufficient, Complete, and Ancillary Statistics

Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary

### 2 1. The r.s., of size n2, from population 2 will be. 2 and 2. 2) The two populations are independent. This implies that all of the n1 n2

Chapter 8 Comparig Two Treatmets Iferece about Two Populatio Meas We wat to compare the meas of two populatios to see whether they differ. There are two situatios to cosider, as show i the followig examples:

### Topic 18: Composite Hypotheses

Toc 18: November, 211 Simple hypotheses limit us to a decisio betwee oe of two possible states of ature. This limitatio does ot allow us, uder the procedures of hypothesis testig to address the basic questio:

### ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors

ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic

### HYPOTHESIS TESTS FOR ONE POPULATION MEAN WORKSHEET MTH 1210, FALL 2018

HYPOTHESIS TESTS FOR ONE POPULATION MEAN WORKSHEET MTH 1210, FALL 2018 We are resposible for 2 types of hypothesis tests that produce ifereces about the ukow populatio mea, µ, each of which has 3 possible

### Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator

Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday

### Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].

Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x

### Hypothesis Testing. Evaluation of Performance of Learned h. Issues. Trade-off Between Bias and Variance

Hypothesis Testig Empirically evaluatig accuracy of hypotheses: importat activity i ML. Three questios: Give observed accuracy over a sample set, how well does this estimate apply over additioal samples?

### Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d

Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y

### 22S:194 Statistical Inference II Homework Assignments. Luke Tierney

S:194 Statistical Iferece II Homework Assigmets Luke Sprig 003 Assigmet 1 Problem 6.3 Problem 6.6 Due Friday, Jauary 31, 003. Problem 6.9 Problem 6.10 Due Friday, Jauary 31, 003. Problem 6.14 Problem 6.0

### Testing Statistical Hypotheses for Compare. Means with Vague Data

Iteratioal Mathematical Forum 5 o. 3 65-6 Testig Statistical Hypotheses for Compare Meas with Vague Data E. Baloui Jamkhaeh ad A. adi Ghara Departmet of Statistics Islamic Azad iversity Ghaemshahr Brach

### Common Large/Small Sample Tests 1/55

Commo Large/Small Sample Tests 1/55 Test of Hypothesis for the Mea (σ Kow) Covert sample result ( x) to a z value Hypothesis Tests for µ Cosider the test H :μ = μ H 1 :μ > μ σ Kow (Assume the populatio