2.6.3 Generalized likelihood ratio tests

Size: px
Start display at page:

Download "2.6.3 Generalized likelihood ratio tests"

Transcription

1 26 HYPOTHESIS TESTING Generalized likelihood ratio tests When a UMP test does not exist, we usually use a generalized likelihood ratio test to verify H 0 : θ Θ against H 1 : θ Θ\Θ It can be used when H 0 is composite, which none of the above methods can The generalized likelihood ratio test has critical region R = y : λy a}, where λy = max θ Θ Lθ y max θ Θ Lθ y is the generalized likelihood ratio and a is a constant chosen to give significance level α, that is such that PλY a H 0 = α If we let ˆθ denote the maximum likelihood estimate of θ and let θ 0 denote the value of θ which maximises the likelihood over all values of θ in Θ, then we may write λy = L θ 0 y L θ y The quantity θ 0 is called the restricted maximum likelihood estimate of θ under H 0 Example 238 Suppose that Y i iid Nµ, σ 2 and consider testing H 0 : µ = µ 0 against H 1 : µ µ 0 Then the likelihood is Lµ, σ 2 y = 2πσ 2 n 2 exp 1 2σ 2 } y i µ 2 The maximum likelihood estimate of θ = µ, σ 2 T is θ = µ, σ 2 T, where µ = y and σ 2 = n y i y 2 /n Similarly, the restricted maximum likelihood estimate of θ = µ, σ 2 T under H 0 is θ 0 = µ 0, σ 2 0 T, where µ 0 = µ 0 and σ 2 0 = n y i µ 0 2 /n

2 114 CHAPTER 2 ELEMENTS OF STATISTICAL INFERENCE Thus, the generalized likelihood ratio is λy = Lµ 0, ˆσ 2 0 y Lˆµ, ˆσ 2 y = = } 2πˆσ 0 2 n 2 exp 1 n 2ˆσ 0 2 y i µ 0 2 2πˆσ 2 n 2 exp 1 n 2ˆσ 2 y i y 2} n 2 2 ˆσ n n exp y i y 2 ˆσ n y i y n n y } i µ n y i µ 0 2 = n y } i y 2 n 2 n y i µ 0 2 Since the critical region is R = y : λy a}, we reject H 0 if n y } i y 2 n 2 n n a y i y y 2 i µ 0 2 n y i µ 0 b, 2 where a and b are constants chosen to give significance level α Now, we may write y i µ 0 2 = y i y + y µ 0 } 2 So we reject H 0 if = = y i y 2 + 2y µ 0 y i y + ny µ 0 2 y i y 2 + ny µ 0 2 Thus, rearranging, we reject H 0 if n y i y 2 n y i y 2 + ny µ 0 2 b 1 + ny µ 0 2 n y i y c ny µ n y i y d, 2 1 n 1 where c and d are constants chosen to give significance level α, that is we can write ny µ0 2 α = PλY a H 0 = P d H S 2 0, where S 2 = 1 n 1 n Y i Y 2

3 26 HYPOTHESIS TESTING 115 To get d we need to work out the distribution of ny µ 0 2 under the null hypothesis S 2 For Y i Nµ, σ 2 we have iid Y N µ, σ2 n and so, under H 0, This gives Also Y N µ 0, σ2 n and n Y µ0 σ 2 ny µ 0 2 σ 2 χ 2 1 n 1S 2 σ 2 χ 2 n 1 N0, 1 Now we may use the fact that if U and V are independent rvs such that U χ 2 ν 1 and V χ 2 ν 2, then U/ν 1 V/ν 2 F ν1,ν 2 Here, U = ny µ 0 2 σ 2 and V = n 1S2 Hence, if H σ 2 0 is true, we have F = U/1 V/n 1 = ny µ 0 2 S 2 F 1,n 1 Therefore, we reject H 0 at a significance level α if ny µ 0 2 F s 2 1,n 1,α, where F 1,n 1,α is such that PF F 1,n 1,α = α Equivalently, we reject H 0 if ny µ0 2 that is, if s 2 t n 1, α 2, y µ 0 s2 /n t n 1, α 2 Of course, this is the usual two-sided t test It can be shown that all of the standard tests in situations with normal distributions are generalized likelihood ratio tests

4 116 CHAPTER 2 ELEMENTS OF STATISTICAL INFERENCE 264 Wilks theorem In more complex cases, we have to use the following approximation to find the critical region The result is stated without proof Theorem 29 Wilks theorem Assume that the joint distribution of Y 1,,Y n depends on p unknown parameters and that, under H 0, the joint distribution depends on p 0 unknown parameters Let ν = p p 0 Then, under some regularity conditions, when the null hypothesis is true, the distribution of the statistic 2 logλy } converges to a χ 2 ν distribution as the sample size n, ie, when H 0 is true and n is large, 2 logλy } approx χ2 ν Thus, for large n, the critical region for a test with approximate significance level α is R = y : 2 logλy} χ 2 ν,α } Example 239 Suppose that Y i iid Poissonλ and consider testing H 0 : λ = λ 0 against H 1 : λ λ 0 Then we have seen that no UMP test exists in this case Now, the likelihood is y i e nλ Lλ y = λ n n y i! The maximum likelihood estimate of λ is ˆλ = y and the restricted maximum likelihood estimate of λ under H 0 is ˆλ 0 = λ 0 Thus, the generalized likelihood ratio is n λy = Lλ 0 y = λ y i 0 e nλ 0 n y i! n Lˆλ y y i! y n y i e ny n λ0 y i = e ny λ0 y It follows that 2 logλy} = 2 ny log y = 2n y log λ0 λ 0 } + ny λ 0 y + λ 0 y }

5 26 HYPOTHESIS TESTING 117 Here, p = 1 and p 0 = 0, and so ν = 1 Therefore, by Wilks theorem, when H 0 is true and n is large, } Y 2n Y log + λ 0 Y χ 2 1 λ 0 Hence, for a test with approximate significance level α, we reject H 0 if and only if } y 2n y log + λ 0 y χ 2 1,α λ 0 Example 240 Suppose that Y i, i = 1,,n, are iid random variables with the probability mass function given by θj, if y = j, j = 1, 2, 3; PY = y = 0, otherwise, where θ j are unknown parameters such that θ 1 +θ 2 +θ 3 = 1 and θ j 0 Consider testing H 0 : θ 1 = θ 2 = θ 3 against H 1 : H 0 is not true We will use the Wilks theorem to derive the critical region for testing this hypothesis at an approximate significance level α Here the full parameter space Θ is two-dimensional because there are only two free parameters, ie, θ 3 = 1 θ 1 θ 2 and Hence p = 2 Θ = θ = θ 1, θ 2, θ 3 T : θ 3 = 1 θ 1 θ 2, θ j 0} The restricted parameter space is zero-dimensional, because under the null hypothesis all the parameters are equal and as they sum up to 1, they all must be equal to 1/3 Hence 1 Θ = θ = 3, 1 3, 1 } T 3 and so p 0 = 0 zero unknown parameters That is the number of degrees of freedom of the χ 2 distribution is ν = p p 0 = 2 To calculate λy we need to find the MLEθ in Θ and in the restricted space Θ

6 118 CHAPTER 2 ELEMENTS OF STATISTICAL INFERENCE MLEθ in Θ: The likelihood function is Lθ; y = θ n 1 1 θn θ 1 θ 2 n 3, Then, the log- where n j is the number of responses equal to j, j = 1, 2, 3 likelihood is For j = 1, 2 we have, lθ; y = n 1 logθ 1 + n 2 logθ 2 + n 3 log1 θ 1 θ 2 When compared to zero, this gives l = n j n θ j θ j 1 θ 1 θ 2 θ j n j = θ 3 n 3 = γ Now, since θ 1 + θ 2 + θ 3 = 1 we obtain γ = 1 n, where n = n 1 + n 2 + n 3 Then the estimates of the parameters are θ j = n j /n, j = 1, 2, 3 The second derivatives are 2 l θ 2 j = n j θ 2 j 2 l θ j θ i = n 3 θ 2 3 n 3 θ 2 3 The determinant of the second derivatives is positive for all θ and the element 2 l θ 2 1 is negative for all θ, so also for θ Hence, there is a maximum at θ and MLEθ in Θ : Lθ; y = MLEθ j = θ j = n j n 1 3 n1 1 3 n2 n3 1 3 That is θ 0 = 1, 1, 1 T Now, we can calculate λy: λy = L θ 0 ; y 1 n1 1 n2 1 n3 n1 n2 n3 L θ; y = n n n n1 n1 n2 n2 n3 n3 = n n n 3n 1 3n 2 3n 3

7 26 HYPOTHESIS TESTING 119 That is 2 logλy} = 2 3 n n j log 3n j We reject H 0 at an approximate significance level α if the observed sample belongs to the critical region R, where 3 } 3nj R = y : 2 n j log χ 2 2;α n 265 Contingency tables We now show that the usual test for association in contingency tables is a generalized likelihood ratio test and that its asymptotic distribution is an example of Wilks theorem Suppose that we have an r c contingency table in which there are Y ij individuals classified in row i and column j, when N individuals are classified independently Let θ ij be the corresponding probability that an individual is classified in row i and column j, so that θ ij 0 and r c θ ij = 1 Then the variables Y ij have a multinomial distribution with parameters N and θ ij for i = 1, 2,, r and j = 1, 2,, c Example 241 In an experiment 150 patients were allocated to three groups of 45, 45 and 60 patients each Two groups were given a new drug at different dose levels and the third group received placebo The responses are as follows: Improved No difference Worse r i = c y ij Placebo Half dose Full dose c j = r y ij N = 150 We are interested in testing the hypothesis that the response to the drug does not depend on the dose level

8 120 CHAPTER 2 ELEMENTS OF STATISTICAL INFERENCE The null hypothesis is that the row and column classifications are independent, and the alternative is that they are dependent More precisely, the null hypothesis is H 0 : θ ij = a i b j for some a i > 0 and b j > 0, with r a i = 1 and c b j = 1, and the alternative is H 1 : θ ij a i b j for at least one pair i, j The usual test statistic is given by X 2 = Y ij E ij 2 E ij, where E ij = R ic j N, R i = c Y ij and C j = r Y ij For large N, X 2 χ 2 r 1c 1 approximately when H 0 is true, and so we reject H 0 at the level of significance α if X 2 χ 2 r 1c 1,α Now consider the generalized likelihood ratio test Then the likelihood is Lθ y = r N! c y ij! r c θ y ij ij = A r c θ y ij ij, where the coefficient A is the number of ways that N subjects can be divided in rc groups with y ij in th-th group The log-likelihood is lθ; y = loga + y ij logθ ij We have to maximize this, subject to the constraint r c θ ij = 1 Let Σ represent the sum over all pairs i, j except r, c Then we may write lθ y = loga + Σ y ij logθ ij + y rc log 1 Σ θ ij Thus, for i, j r, c, solving the equation l = y ij y rc = y ij y rc = 0 θ ij θ ij 1 Σ θ ij θ ij θ rc

9 26 HYPOTHESIS TESTING 121 yields ˆθ ij /y ij = ˆθ rc /y rc = γ, say Since r c θ ij = r c y ijγ = 1, we have γ = 1/N, so that the maximum likelihood estimate of θ ij is ˆθ ij = y ij /N for i = 1, 2,, r and j = 1, 2,, c It follows that l θ y = loga + y ij log yij N Now, under H 0, we have θ ij = a i b j and so lθ y = loga + = loga + y ij loga i + logb j } r i loga i + c j logb j Now, we maximize this subject to the constraints r a i = 1 and c b j = 1 That is, we maximize lθ y = loga + Σ r i loga i + r r loga r + Σ c j logb j + c c logb c = loga + Σ r i loga i + r r log1 Σ a i + Σ c j logb j + c c log1 Σ b j Then, l = r i r r a i a i a r which, when compared to zero gives or However, Hence, γ = 1/N and so 1 = r i â i = r r â r â i r i = âr r r = γ â i = r i γ = Nγ â i = r i N Similarly, we obtain the ML estimates for b j as b j = c j /N

10 122 CHAPTER 2 ELEMENTS OF STATISTICAL INFERENCE Thus, the restricted maximum likelihood estimate of θ ij under H 0 is for i = 1, 2,, r and j = 1, 2,, c It follows that θ ij 0 = â i bj = r i C j N N = /N l θ 0 y = loga + y ij log eij N Hence, we obtain } L θ 0 y 2 logλy} = 2 log = 2l θ 0 y l θ y} = 2 L θ y y ij log yij Here, p = rc 1 and p 0 = r 1+c 1 = r+c 2, and so ν = r 1c 1 Therefore, by Wilks theorem, when H 0 is true and N is large, 2 Yij Y ij log E ij approx χ2 r 1c 1 The test statistics X 2 and 2 logλy} are asymptotically equivalent, that is, they differ by quantities that tend to zero as N To see this, first note that y ij is small relative to y ij and when N is large Thus, since y ij = + y ij, we may write by Taylor series expansion of log1 + x around x, cut after second order term log yij = log 1 + y ij y ij 1 2 y ij 2 e 2 ij Hence, we have y ij log yij yij + y ij } 1 2 y ij + 1 y ij 2 2 } y ij 2 e 2 ij

11 26 HYPOTHESIS TESTING 123 Since r c y ij = 0, it follows that 2 logλy} y ij 2 So we now see why we use the X 2 test statistic Example 242 continued Now we will test the hypothesis from the previous example The table of values is following Improved No difference Worse Placebo Half dose Full dose This gives X 2 obs = y ij 2 = 1417 The critical value is χ 2 4;005 = 9488, hence there is no evidence to reject the null hypothesis saying that the different responses are independent of the levels of the new drug and placebo We obtain the same conclusion using the critical region derived from the Wilks theorem, which is } yij R = y : 2 y ij log χ 2 ν;α Here 2 y ij log yij = 142

2.5.3 Generalized likelihood ratio tests

2.5.3 Generalized likelihood ratio tests 25 HYPOTHESIS TESTING 127 253 Generalized likelihood ratio tests When a UMP test does not exist, we usually use a generalized likelihood ratio test to verify H 0 : ϑ Θ against H 1 : ϑ Θ\Θ It can be used

More information

Notes on the Multivariate Normal and Related Topics

Notes on the Multivariate Normal and Related Topics Version: July 10, 2013 Notes on the Multivariate Normal and Related Topics Let me refresh your memory about the distinctions between population and sample; parameters and statistics; population distributions

More information

Lecture 21: October 19

Lecture 21: October 19 36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 21: October 19 21.1 Likelihood Ratio Test (LRT) To test composite versus composite hypotheses the general method is to use

More information

Lecture 15. Hypothesis testing in the linear model

Lecture 15. Hypothesis testing in the linear model 14. Lecture 15. Hypothesis testing in the linear model Lecture 15. Hypothesis testing in the linear model 1 (1 1) Preliminary lemma 15. Hypothesis testing in the linear model 15.1. Preliminary lemma Lemma

More information

2.3 Methods of Estimation

2.3 Methods of Estimation 96 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE 2.3 Methods of Estimation 2.3. Method of Moments The Method of Moments is a simple technique based on the idea that the sample moments are natural estimators

More information

MATH5745 Multivariate Methods Lecture 07

MATH5745 Multivariate Methods Lecture 07 MATH5745 Multivariate Methods Lecture 07 Tests of hypothesis on covariance matrix March 16, 2018 MATH5745 Multivariate Methods Lecture 07 March 16, 2018 1 / 39 Test on covariance matrices: Introduction

More information

Summary of Chapters 7-9

Summary of Chapters 7-9 Summary of Chapters 7-9 Chapter 7. Interval Estimation 7.2. Confidence Intervals for Difference of Two Means Let X 1,, X n and Y 1, Y 2,, Y m be two independent random samples of sizes n and m from two

More information

Math 152. Rumbos Fall Solutions to Assignment #12

Math 152. Rumbos Fall Solutions to Assignment #12 Math 52. umbos Fall 2009 Solutions to Assignment #2. Suppose that you observe n iid Bernoulli(p) random variables, denoted by X, X 2,..., X n. Find the LT rejection region for the test of H o : p p o versus

More information

2.5 Hypothesis Testing

2.5 Hypothesis Testing 118 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE 2.5 Hypothesis Testing We assume that Y 1,...,Y n have a joint distribution which depends on the unknown parametersϑ = ϑ 1,...,ϑ p ) T. The set of all possible

More information

I i=1 1 I(J 1) j=1 (Y ij Ȳi ) 2. j=1 (Y j Ȳ )2 ] = 2n( is the two-sample t-test statistic.

I i=1 1 I(J 1) j=1 (Y ij Ȳi ) 2. j=1 (Y j Ȳ )2 ] = 2n( is the two-sample t-test statistic. Serik Sagitov, Chalmers and GU, February, 08 Solutions chapter Matlab commands: x = data matrix boxplot(x) anova(x) anova(x) Problem.3 Consider one-way ANOVA test statistic For I = and = n, put F = MS

More information

Likelihood Ratio tests

Likelihood Ratio tests Likelihood Ratio tests For general composite hypotheses optimality theory is not usually successful in producing an optimal test. instead we look for heuristics to guide our choices. The simplest approach

More information

Hypothesis Testing. Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA

Hypothesis Testing. Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA Hypothesis Testing Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA An Example Mardia et al. (979, p. ) reprint data from Frets (9) giving the length and breadth (in

More information

Topic 19 Extensions on the Likelihood Ratio

Topic 19 Extensions on the Likelihood Ratio Topic 19 Extensions on the Likelihood Ratio Two-Sided Tests 1 / 12 Outline Overview Normal Observations Power Analysis 2 / 12 Overview The likelihood ratio test is a popular choice for composite hypothesis

More information

Exercises and Answers to Chapter 1

Exercises and Answers to Chapter 1 Exercises and Answers to Chapter The continuous type of random variable X has the following density function: a x, if < x < a, f (x), otherwise. Answer the following questions. () Find a. () Obtain mean

More information

Introductory Econometrics

Introductory Econometrics Session 4 - Testing hypotheses Roland Sciences Po July 2011 Motivation After estimation, delivering information involves testing hypotheses Did this drug had any effect on the survival rate? Is this drug

More information

Diagnostics can identify two possible areas of failure of assumptions when fitting linear models.

Diagnostics can identify two possible areas of failure of assumptions when fitting linear models. 1 Transformations 1.1 Introduction Diagnostics can identify two possible areas of failure of assumptions when fitting linear models. (i) lack of Normality (ii) heterogeneity of variances It is important

More information

Testing Hypothesis. Maura Mezzetti. Department of Economics and Finance Università Tor Vergata

Testing Hypothesis. Maura Mezzetti. Department of Economics and Finance Università Tor Vergata Maura Department of Economics and Finance Università Tor Vergata Hypothesis Testing Outline It is a mistake to confound strangeness with mystery Sherlock Holmes A Study in Scarlet Outline 1 The Power Function

More information

Statistics 3858 : Maximum Likelihood Estimators

Statistics 3858 : Maximum Likelihood Estimators Statistics 3858 : Maximum Likelihood Estimators 1 Method of Maximum Likelihood In this method we construct the so called likelihood function, that is L(θ) = L(θ; X 1, X 2,..., X n ) = f n (X 1, X 2,...,

More information

Chapter 7. Hypothesis Testing

Chapter 7. Hypothesis Testing Chapter 7. Hypothesis Testing Joonpyo Kim June 24, 2017 Joonpyo Kim Ch7 June 24, 2017 1 / 63 Basic Concepts of Testing Suppose that our interest centers on a random variable X which has density function

More information

Likelihood-based inference with missing data under missing-at-random

Likelihood-based inference with missing data under missing-at-random Likelihood-based inference with missing data under missing-at-random Jae-kwang Kim Joint work with Shu Yang Department of Statistics, Iowa State University May 4, 014 Outline 1. Introduction. Parametric

More information

Statistical Inference

Statistical Inference Statistical Inference Classical and Bayesian Methods Revision Class for Midterm Exam AMS-UCSC Th Feb 9, 2012 Winter 2012. Session 1 (Revision Class) AMS-132/206 Th Feb 9, 2012 1 / 23 Topics Topics We will

More information

Hypothesis Testing: The Generalized Likelihood Ratio Test

Hypothesis Testing: The Generalized Likelihood Ratio Test Hypothesis Testing: The Generalized Likelihood Ratio Test Consider testing the hypotheses H 0 : θ Θ 0 H 1 : θ Θ \ Θ 0 Definition: The Generalized Likelihood Ratio (GLR Let L(θ be a likelihood for a random

More information

MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30

MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30 MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD Copyright c 2012 (Iowa State University) Statistics 511 1 / 30 INFORMATION CRITERIA Akaike s Information criterion is given by AIC = 2l(ˆθ) + 2k, where l(ˆθ)

More information

Topic 21 Goodness of Fit

Topic 21 Goodness of Fit Topic 21 Goodness of Fit Contingency Tables 1 / 11 Introduction Two-way Table Smoking Habits The Hypothesis The Test Statistic Degrees of Freedom Outline 2 / 11 Introduction Contingency tables, also known

More information

Stat 579: Generalized Linear Models and Extensions

Stat 579: Generalized Linear Models and Extensions Stat 579: Generalized Linear Models and Extensions Mixed models Yan Lu March, 2018, week 8 1 / 32 Restricted Maximum Likelihood (REML) REML: uses a likelihood function calculated from the transformed set

More information

Likelihoods. P (Y = y) = f(y). For example, suppose Y has a geometric distribution on 1, 2,... with parameter p. Then the pmf is

Likelihoods. P (Y = y) = f(y). For example, suppose Y has a geometric distribution on 1, 2,... with parameter p. Then the pmf is Likelihoods The distribution of a random variable Y with a discrete sample space (e.g. a finite sample space or the integers) can be characterized by its probability mass function (pmf): P (Y = y) = f(y).

More information

Comparing two independent samples

Comparing two independent samples In many applications it is necessary to compare two competing methods (for example, to compare treatment effects of a standard drug and an experimental drug). To compare two methods from statistical point

More information

Math 181B Homework 1 Solution

Math 181B Homework 1 Solution Math 181B Homework 1 Solution 1. Write down the likelihood: L(λ = n λ X i e λ X i! (a One-sided test: H 0 : λ = 1 vs H 1 : λ = 0.1 The likelihood ratio: where LR = L(1 L(0.1 = 1 X i e n 1 = λ n X i e nλ

More information

Classical Inference for Gaussian Linear Models

Classical Inference for Gaussian Linear Models Classical Inference for Gaussian Linear Models Surya Tokdar Gaussian linear model Data (z i, Y i ), i = 1,, n, dim(z i ) = p Model Y i = z T i Parameters: β R p, σ 2 > 0 Inference needed on η = a T β IID

More information

Statistics 3858 : Contingency Tables

Statistics 3858 : Contingency Tables Statistics 3858 : Contingency Tables 1 Introduction Before proceeding with this topic the student should review generalized likelihood ratios ΛX) for multinomial distributions, its relation to Pearson

More information

Hypothesis Testing One Sample Tests

Hypothesis Testing One Sample Tests STATISTICS Lecture no. 13 Department of Econometrics FEM UO Brno office 69a, tel. 973 442029 email:jiri.neubauer@unob.cz 12. 1. 2010 Tests on Mean of a Normal distribution Tests on Variance of a Normal

More information

Lecture 23 Maximum Likelihood Estimation and Bayesian Inference

Lecture 23 Maximum Likelihood Estimation and Bayesian Inference Lecture 23 Maximum Likelihood Estimation and Bayesian Inference Thais Paiva STA 111 - Summer 2013 Term II August 7, 2013 1 / 31 Thais Paiva STA 111 - Summer 2013 Term II Lecture 23, 08/07/2013 Lecture

More information

Lecture 3. Inference about multivariate normal distribution

Lecture 3. Inference about multivariate normal distribution Lecture 3. Inference about multivariate normal distribution 3.1 Point and Interval Estimation Let X 1,..., X n be i.i.d. N p (µ, Σ). We are interested in evaluation of the maximum likelihood estimates

More information

2.3 Analysis of Categorical Data

2.3 Analysis of Categorical Data 90 CHAPTER 2. ESTIMATION AND HYPOTHESIS TESTING 2.3 Analysis of Categorical Data 2.3.1 The Multinomial Probability Distribution A mulinomial random variable is a generalization of the binomial rv. It results

More information

A Very Brief Summary of Statistical Inference, and Examples

A Very Brief Summary of Statistical Inference, and Examples A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random

More information

MAS3301 Bayesian Statistics

MAS3301 Bayesian Statistics MAS3301 Bayesian Statistics M. Farrow School of Mathematics and Statistics Newcastle University Semester 2, 2008-9 1 11 Conjugate Priors IV: The Dirichlet distribution and multinomial observations 11.1

More information

Nuisance parameters and their treatment

Nuisance parameters and their treatment BS2 Statistical Inference, Lecture 2, Hilary Term 2008 April 2, 2008 Ancillarity Inference principles Completeness A statistic A = a(x ) is said to be ancillary if (i) The distribution of A does not depend

More information

Math 494: Mathematical Statistics

Math 494: Mathematical Statistics Math 494: Mathematical Statistics Instructor: Jimin Ding jmding@wustl.edu Department of Mathematics Washington University in St. Louis Class materials are available on course website (www.math.wustl.edu/

More information

Hypothesis Test. The opposite of the null hypothesis, called an alternative hypothesis, becomes

Hypothesis Test. The opposite of the null hypothesis, called an alternative hypothesis, becomes Neyman-Pearson paradigm. Suppose that a researcher is interested in whether the new drug works. The process of determining whether the outcome of the experiment points to yes or no is called hypothesis

More information

Answer Key for STAT 200B HW No. 7

Answer Key for STAT 200B HW No. 7 Answer Key for STAT 200B HW No. 7 May 5, 2007 Problem 2.2 p. 649 Assuming binomial 2-sample model ˆπ =.75, ˆπ 2 =.6. a ˆτ = ˆπ 2 ˆπ =.5. From Ex. 2.5a on page 644: ˆπ ˆπ + ˆπ 2 ˆπ 2.75.25.6.4 = + =.087;

More information

Math 494: Mathematical Statistics

Math 494: Mathematical Statistics Math 494: Mathematical Statistics Instructor: Jimin Ding jmding@wustl.edu Department of Mathematics Washington University in St. Louis Class materials are available on course website (www.math.wustl.edu/

More information

Cherry Blossom run (1) The credit union Cherry Blossom Run is a 10 mile race that takes place every year in D.C. In 2009 there were participants

Cherry Blossom run (1) The credit union Cherry Blossom Run is a 10 mile race that takes place every year in D.C. In 2009 there were participants 18.650 Statistics for Applications Chapter 5: Parametric hypothesis testing 1/37 Cherry Blossom run (1) The credit union Cherry Blossom Run is a 10 mile race that takes place every year in D.C. In 2009

More information

Estimation MLE-Pandemic data MLE-Financial crisis data Evaluating estimators. Estimation. September 24, STAT 151 Class 6 Slide 1

Estimation MLE-Pandemic data MLE-Financial crisis data Evaluating estimators. Estimation. September 24, STAT 151 Class 6 Slide 1 Estimation September 24, 2018 STAT 151 Class 6 Slide 1 Pandemic data Treatment outcome, X, from n = 100 patients in a pandemic: 1 = recovered and 0 = not recovered 1 1 1 0 0 0 1 1 1 0 0 1 0 1 0 0 1 1 1

More information

Ch. 5 Hypothesis Testing

Ch. 5 Hypothesis Testing Ch. 5 Hypothesis Testing The current framework of hypothesis testing is largely due to the work of Neyman and Pearson in the late 1920s, early 30s, complementing Fisher s work on estimation. As in estimation,

More information

Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach

Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score

More information

Answer Key for STAT 200B HW No. 8

Answer Key for STAT 200B HW No. 8 Answer Key for STAT 200B HW No. 8 May 8, 2007 Problem 3.42 p. 708 The values of Ȳ for x 00, 0, 20, 30 are 5/40, 0, 20/50, and, respectively. From Corollary 3.5 it follows that MLE exists i G is identiable

More information

2014/2015 Smester II ST5224 Final Exam Solution

2014/2015 Smester II ST5224 Final Exam Solution 014/015 Smester II ST54 Final Exam Solution 1 Suppose that (X 1,, X n ) is a random sample from a distribution with probability density function f(x; θ) = e (x θ) I [θ, ) (x) (i) Show that the family of

More information

Statistics. Statistics

Statistics. Statistics The main aims of statistics 1 1 Choosing a model 2 Estimating its parameter(s) 1 point estimates 2 interval estimates 3 Testing hypotheses Distributions used in statistics: χ 2 n-distribution 2 Let X 1,

More information

BIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation

BIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation BIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation Yujin Chung November 29th, 2016 Fall 2016 Yujin Chung Lec13: MLE Fall 2016 1/24 Previous Parametric tests Mean comparisons (normality assumption)

More information

F & B Approaches to a simple model

F & B Approaches to a simple model A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 215 http://www.astro.cornell.edu/~cordes/a6523 Lecture 11 Applications: Model comparison Challenges in large-scale surveys

More information

Lecture 10: Generalized likelihood ratio test

Lecture 10: Generalized likelihood ratio test Stat 200: Introduction to Statistical Inference Autumn 2018/19 Lecture 10: Generalized likelihood ratio test Lecturer: Art B. Owen October 25 Disclaimer: These notes have not been subjected to the usual

More information

BIOS 2083 Linear Models c Abdus S. Wahed

BIOS 2083 Linear Models c Abdus S. Wahed Chapter 5 206 Chapter 6 General Linear Model: Statistical Inference 6.1 Introduction So far we have discussed formulation of linear models (Chapter 1), estimability of parameters in a linear model (Chapter

More information

Introduction to Estimation Methods for Time Series models Lecture 2

Introduction to Estimation Methods for Time Series models Lecture 2 Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:

More information

4.5.1 The use of 2 log Λ when θ is scalar

4.5.1 The use of 2 log Λ when θ is scalar 4.5. ASYMPTOTIC FORM OF THE G.L.R.T. 97 4.5.1 The use of 2 log Λ when θ is scalar Suppose we wish to test the hypothesis NH : θ = θ where θ is a given value against the alternative AH : θ θ on the basis

More information

simple if it completely specifies the density of x

simple if it completely specifies the density of x 3. Hypothesis Testing Pure significance tests Data x = (x 1,..., x n ) from f(x, θ) Hypothesis H 0 : restricts f(x, θ) Are the data consistent with H 0? H 0 is called the null hypothesis simple if it completely

More information

Economics 520. Lecture Note 19: Hypothesis Testing via the Neyman-Pearson Lemma CB 8.1,

Economics 520. Lecture Note 19: Hypothesis Testing via the Neyman-Pearson Lemma CB 8.1, Economics 520 Lecture Note 9: Hypothesis Testing via the Neyman-Pearson Lemma CB 8., 8.3.-8.3.3 Uniformly Most Powerful Tests and the Neyman-Pearson Lemma Let s return to the hypothesis testing problem

More information

Hypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3

Hypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3 Hypothesis Testing CB: chapter 8; section 0.3 Hypothesis: statement about an unknown population parameter Examples: The average age of males in Sweden is 7. (statement about population mean) The lowest

More information

Foundations of Statistical Inference

Foundations of Statistical Inference Foundations of Statistical Inference Julien Berestycki Department of Statistics University of Oxford MT 2015 Julien Berestycki (University of Oxford) SB2a MT 2015 1 / 16 Lecture 16 : Bayesian analysis

More information

STAT 514 Solutions to Assignment #6

STAT 514 Solutions to Assignment #6 STAT 514 Solutions to Assignment #6 Question 1: Suppose that X 1,..., X n are a simple random sample from a Weibull distribution with density function f θ x) = θcx c 1 exp{ θx c }I{x > 0} for some fixed

More information

Composite Hypotheses and Generalized Likelihood Ratio Tests

Composite Hypotheses and Generalized Likelihood Ratio Tests Composite Hypotheses and Generalized Likelihood Ratio Tests Rebecca Willett, 06 In many real world problems, it is difficult to precisely specify probability distributions. Our models for data may involve

More information

Homework 7: Solutions. P3.1 from Lehmann, Romano, Testing Statistical Hypotheses.

Homework 7: Solutions. P3.1 from Lehmann, Romano, Testing Statistical Hypotheses. Stat 300A Theory of Statistics Homework 7: Solutions Nikos Ignatiadis Due on November 28, 208 Solutions should be complete and concisely written. Please, use a separate sheet or set of sheets for each

More information

(a) (3 points) Construct a 95% confidence interval for β 2 in Equation 1.

(a) (3 points) Construct a 95% confidence interval for β 2 in Equation 1. Problem 1 (21 points) An economist runs the regression y i = β 0 + x 1i β 1 + x 2i β 2 + x 3i β 3 + ε i (1) The results are summarized in the following table: Equation 1. Variable Coefficient Std. Error

More information

Lecture 26: Likelihood ratio tests

Lecture 26: Likelihood ratio tests Lecture 26: Likelihood ratio tests Likelihood ratio When both H 0 and H 1 are simple (i.e., Θ 0 = {θ 0 } and Θ 1 = {θ 1 }), Theorem 6.1 applies and a UMP test rejects H 0 when f θ1 (X) f θ0 (X) > c 0 for

More information

Categorical Variables and Contingency Tables: Description and Inference

Categorical Variables and Contingency Tables: Description and Inference Categorical Variables and Contingency Tables: Description and Inference STAT 526 Professor Olga Vitek March 3, 2011 Reading: Agresti Ch. 1, 2 and 3 Faraway Ch. 4 3 Univariate Binomial and Multinomial Measurements

More information

Final Exam. 1. (6 points) True/False. Please read the statements carefully, as no partial credit will be given.

Final Exam. 1. (6 points) True/False. Please read the statements carefully, as no partial credit will be given. 1. (6 points) True/False. Please read the statements carefully, as no partial credit will be given. (a) If X and Y are independent, Corr(X, Y ) = 0. (b) (c) (d) (e) A consistent estimator must be asymptotically

More information

Central Limit Theorem ( 5.3)

Central Limit Theorem ( 5.3) Central Limit Theorem ( 5.3) Let X 1, X 2,... be a sequence of independent random variables, each having n mean µ and variance σ 2. Then the distribution of the partial sum S n = X i i=1 becomes approximately

More information

Mathematical statistics

Mathematical statistics October 1 st, 2018 Lecture 11: Sufficient statistic Where are we? Week 1 Week 2 Week 4 Week 7 Week 10 Week 14 Probability reviews Chapter 6: Statistics and Sampling Distributions Chapter 7: Point Estimation

More information

2017 Financial Mathematics Orientation - Statistics

2017 Financial Mathematics Orientation - Statistics 2017 Financial Mathematics Orientation - Statistics Written by Long Wang Edited by Joshua Agterberg August 21, 2018 Contents 1 Preliminaries 5 1.1 Samples and Population............................. 5

More information

STT 843 Key to Homework 1 Spring 2018

STT 843 Key to Homework 1 Spring 2018 STT 843 Key to Homework Spring 208 Due date: Feb 4, 208 42 (a Because σ = 2, σ 22 = and ρ 2 = 05, we have σ 2 = ρ 2 σ σ22 = 2/2 Then, the mean and covariance of the bivariate normal is µ = ( 0 2 and Σ

More information

General Linear Model: Statistical Inference

General Linear Model: Statistical Inference Chapter 6 General Linear Model: Statistical Inference 6.1 Introduction So far we have discussed formulation of linear models (Chapter 1), estimability of parameters in a linear model (Chapter 4), least

More information

Part 1.) We know that the probability of any specific x only given p ij = p i p j is just multinomial(n, p) where p k1 k 2

Part 1.) We know that the probability of any specific x only given p ij = p i p j is just multinomial(n, p) where p k1 k 2 Problem.) I will break this into two parts: () Proving w (m) = p( x (m) X i = x i, X j = x j, p ij = p i p j ). In other words, the probability of a specific table in T x given the row and column counts

More information

GARCH Models Estimation and Inference

GARCH Models Estimation and Inference GARCH Models Estimation and Inference Eduardo Rossi University of Pavia December 013 Rossi GARCH Financial Econometrics - 013 1 / 1 Likelihood function The procedure most often used in estimating θ 0 in

More information

Maximum Likelihood Large Sample Theory

Maximum Likelihood Large Sample Theory Maximum Likelihood Large Sample Theory MIT 18.443 Dr. Kempthorne Spring 2015 1 Outline 1 Large Sample Theory of Maximum Likelihood Estimates 2 Asymptotic Results: Overview Asymptotic Framework Data Model

More information

Estimation Theory. as Θ = (Θ 1,Θ 2,...,Θ m ) T. An estimator

Estimation Theory. as Θ = (Θ 1,Θ 2,...,Θ m ) T. An estimator Estimation Theory Estimation theory deals with finding numerical values of interesting parameters from given set of data. We start with formulating a family of models that could describe how the data were

More information

Regression Estimation - Least Squares and Maximum Likelihood. Dr. Frank Wood

Regression Estimation - Least Squares and Maximum Likelihood. Dr. Frank Wood Regression Estimation - Least Squares and Maximum Likelihood Dr. Frank Wood Least Squares Max(min)imization Function to minimize w.r.t. β 0, β 1 Q = n (Y i (β 0 + β 1 X i )) 2 i=1 Minimize this by maximizing

More information

Summary of Chapter 7 (Sections ) and Chapter 8 (Section 8.1)

Summary of Chapter 7 (Sections ) and Chapter 8 (Section 8.1) Summary of Chapter 7 (Sections 7.2-7.5) and Chapter 8 (Section 8.1) Chapter 7. Tests of Statistical Hypotheses 7.2. Tests about One Mean (1) Test about One Mean Case 1: σ is known. Assume that X N(µ, σ

More information

Lecture 2: Basic Concepts and Simple Comparative Experiments Montgomery: Chapter 2

Lecture 2: Basic Concepts and Simple Comparative Experiments Montgomery: Chapter 2 Lecture 2: Basic Concepts and Simple Comparative Experiments Montgomery: Chapter 2 Fall, 2013 Page 1 Random Variable and Probability Distribution Discrete random variable Y : Finite possible values {y

More information

McGill University. Faculty of Science. Department of Mathematics and Statistics. Part A Examination. Statistics: Theory Paper

McGill University. Faculty of Science. Department of Mathematics and Statistics. Part A Examination. Statistics: Theory Paper McGill University Faculty of Science Department of Mathematics and Statistics Part A Examination Statistics: Theory Paper Date: 10th May 2015 Instructions Time: 1pm-5pm Answer only two questions from Section

More information

Hypothesis Testing. A rule for making the required choice can be described in two ways: called the rejection or critical region of the test.

Hypothesis Testing. A rule for making the required choice can be described in two ways: called the rejection or critical region of the test. Hypothesis Testing Hypothesis testing is a statistical problem where you must choose, on the basis of data X, between two alternatives. We formalize this as the problem of choosing between two hypotheses:

More information

Minimum Message Length Inference and Mixture Modelling of Inverse Gaussian Distributions

Minimum Message Length Inference and Mixture Modelling of Inverse Gaussian Distributions Minimum Message Length Inference and Mixture Modelling of Inverse Gaussian Distributions Daniel F. Schmidt Enes Makalic Centre for Molecular, Environmental, Genetic & Analytic (MEGA) Epidemiology School

More information

[y i α βx i ] 2 (2) Q = i=1

[y i α βx i ] 2 (2) Q = i=1 Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation

More information

STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots. March 8, 2015

STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots. March 8, 2015 STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots March 8, 2015 The duality between CI and hypothesis testing The duality between CI and hypothesis

More information

Statistical Data Analysis Stat 3: p-values, parameter estimation

Statistical Data Analysis Stat 3: p-values, parameter estimation Statistical Data Analysis Stat 3: p-values, parameter estimation London Postgraduate Lectures on Particle Physics; University of London MSci course PH4515 Glen Cowan Physics Department Royal Holloway,

More information

Generalized Linear Models Introduction

Generalized Linear Models Introduction Generalized Linear Models Introduction Statistics 135 Autumn 2005 Copyright c 2005 by Mark E. Irwin Generalized Linear Models For many problems, standard linear regression approaches don t work. Sometimes,

More information

Categorical Data Analysis Chapter 3

Categorical Data Analysis Chapter 3 Categorical Data Analysis Chapter 3 The actual coverage probability is usually a bit higher than the nominal level. Confidence intervals for association parameteres Consider the odds ratio in the 2x2 table,

More information

Lecture 1 October 9, 2013

Lecture 1 October 9, 2013 Probabilistic Graphical Models Fall 2013 Lecture 1 October 9, 2013 Lecturer: Guillaume Obozinski Scribe: Huu Dien Khue Le, Robin Bénesse The web page of the course: http://www.di.ens.fr/~fbach/courses/fall2013/

More information

Problems ( ) 1 exp. 2. n! e λ and

Problems ( ) 1 exp. 2. n! e λ and Problems The expressions for the probability mass function of the Poisson(λ) distribution, and the density function of the Normal distribution with mean µ and variance σ 2, may be useful: ( ) 1 exp. 2πσ

More information

Definition 3.1 A statistical hypothesis is a statement about the unknown values of the parameters of the population distribution.

Definition 3.1 A statistical hypothesis is a statement about the unknown values of the parameters of the population distribution. Hypothesis Testing Definition 3.1 A statistical hypothesis is a statement about the unknown values of the parameters of the population distribution. Suppose the family of population distributions is indexed

More information

Statistics 135 Fall 2008 Final Exam

Statistics 135 Fall 2008 Final Exam Name: SID: Statistics 135 Fall 2008 Final Exam Show your work. The number of points each question is worth is shown at the beginning of the question. There are 10 problems. 1. [2] The normal equations

More information

2.3.3 Method of Least Squares

2.3.3 Method of Least Squares .3. METHODS OF ESTIMATION 105.3.3 Method of Least Squares IfY 1,...,Y n are independent random variables, which have the same variance and higher-order moments, and, if each E(Y i ) is a linear function

More information

ST495: Survival Analysis: Hypothesis testing and confidence intervals

ST495: Survival Analysis: Hypothesis testing and confidence intervals ST495: Survival Analysis: Hypothesis testing and confidence intervals Eric B. Laber Department of Statistics, North Carolina State University April 3, 2014 I remember that one fateful day when Coach took

More information

STAT 512 sp 2018 Summary Sheet

STAT 512 sp 2018 Summary Sheet STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}

More information

Epidemiology Principles of Biostatistics Chapter 10 - Inferences about two populations. John Koval

Epidemiology Principles of Biostatistics Chapter 10 - Inferences about two populations. John Koval Epidemiology 9509 Principles of Biostatistics Chapter 10 - Inferences about John Koval Department of Epidemiology and Biostatistics University of Western Ontario What is being covered 1. differences in

More information

Accurate directional inference for vector parameters

Accurate directional inference for vector parameters Accurate directional inference for vector parameters Nancy Reid October 28, 2016 with Don Fraser, Nicola Sartori, Anthony Davison Parametric models and likelihood model f (y; θ), θ R p data y = (y 1,...,

More information

ST3241 Categorical Data Analysis I Two-way Contingency Tables. 2 2 Tables, Relative Risks and Odds Ratios

ST3241 Categorical Data Analysis I Two-way Contingency Tables. 2 2 Tables, Relative Risks and Odds Ratios ST3241 Categorical Data Analysis I Two-way Contingency Tables 2 2 Tables, Relative Risks and Odds Ratios 1 What Is A Contingency Table (p.16) Suppose X and Y are two categorical variables X has I categories

More information

Likelihood inference in the presence of nuisance parameters

Likelihood inference in the presence of nuisance parameters Likelihood inference in the presence of nuisance parameters Nancy Reid, University of Toronto www.utstat.utoronto.ca/reid/research 1. Notation, Fisher information, orthogonal parameters 2. Likelihood inference

More information

Discrete Multivariate Statistics

Discrete Multivariate Statistics Discrete Multivariate Statistics Univariate Discrete Random variables Let X be a discrete random variable which, in this module, will be assumed to take a finite number of t different values which are

More information

Methods for Statistical Prediction Financial Time Series I. Topic 1: Review on Hypothesis Testing

Methods for Statistical Prediction Financial Time Series I. Topic 1: Review on Hypothesis Testing Methods for Statistical Prediction Financial Time Series I Topic 1: Review on Hypothesis Testing Hung Chen Department of Mathematics National Taiwan University 9/26/2002 OUTLINE 1. Fundamental Concepts

More information

Hypothesis Testing Chap 10p460

Hypothesis Testing Chap 10p460 Hypothesis Testing Chap 1p46 Elements of a statistical test p462 - Null hypothesis - Alternative hypothesis - Test Statistic - Rejection region Rejection Region p462 The rejection region (RR) specifies

More information

Mathematics Qualifying Examination January 2015 STAT Mathematical Statistics

Mathematics Qualifying Examination January 2015 STAT Mathematical Statistics Mathematics Qualifying Examination January 2015 STAT 52800 - Mathematical Statistics NOTE: Answer all questions completely and justify your derivations and steps. A calculator and statistical tables (normal,

More information