2.6.3 Generalized likelihood ratio tests
|
|
- Laurel Lang
- 6 years ago
- Views:
Transcription
1 26 HYPOTHESIS TESTING Generalized likelihood ratio tests When a UMP test does not exist, we usually use a generalized likelihood ratio test to verify H 0 : θ Θ against H 1 : θ Θ\Θ It can be used when H 0 is composite, which none of the above methods can The generalized likelihood ratio test has critical region R = y : λy a}, where λy = max θ Θ Lθ y max θ Θ Lθ y is the generalized likelihood ratio and a is a constant chosen to give significance level α, that is such that PλY a H 0 = α If we let ˆθ denote the maximum likelihood estimate of θ and let θ 0 denote the value of θ which maximises the likelihood over all values of θ in Θ, then we may write λy = L θ 0 y L θ y The quantity θ 0 is called the restricted maximum likelihood estimate of θ under H 0 Example 238 Suppose that Y i iid Nµ, σ 2 and consider testing H 0 : µ = µ 0 against H 1 : µ µ 0 Then the likelihood is Lµ, σ 2 y = 2πσ 2 n 2 exp 1 2σ 2 } y i µ 2 The maximum likelihood estimate of θ = µ, σ 2 T is θ = µ, σ 2 T, where µ = y and σ 2 = n y i y 2 /n Similarly, the restricted maximum likelihood estimate of θ = µ, σ 2 T under H 0 is θ 0 = µ 0, σ 2 0 T, where µ 0 = µ 0 and σ 2 0 = n y i µ 0 2 /n
2 114 CHAPTER 2 ELEMENTS OF STATISTICAL INFERENCE Thus, the generalized likelihood ratio is λy = Lµ 0, ˆσ 2 0 y Lˆµ, ˆσ 2 y = = } 2πˆσ 0 2 n 2 exp 1 n 2ˆσ 0 2 y i µ 0 2 2πˆσ 2 n 2 exp 1 n 2ˆσ 2 y i y 2} n 2 2 ˆσ n n exp y i y 2 ˆσ n y i y n n y } i µ n y i µ 0 2 = n y } i y 2 n 2 n y i µ 0 2 Since the critical region is R = y : λy a}, we reject H 0 if n y } i y 2 n 2 n n a y i y y 2 i µ 0 2 n y i µ 0 b, 2 where a and b are constants chosen to give significance level α Now, we may write y i µ 0 2 = y i y + y µ 0 } 2 So we reject H 0 if = = y i y 2 + 2y µ 0 y i y + ny µ 0 2 y i y 2 + ny µ 0 2 Thus, rearranging, we reject H 0 if n y i y 2 n y i y 2 + ny µ 0 2 b 1 + ny µ 0 2 n y i y c ny µ n y i y d, 2 1 n 1 where c and d are constants chosen to give significance level α, that is we can write ny µ0 2 α = PλY a H 0 = P d H S 2 0, where S 2 = 1 n 1 n Y i Y 2
3 26 HYPOTHESIS TESTING 115 To get d we need to work out the distribution of ny µ 0 2 under the null hypothesis S 2 For Y i Nµ, σ 2 we have iid Y N µ, σ2 n and so, under H 0, This gives Also Y N µ 0, σ2 n and n Y µ0 σ 2 ny µ 0 2 σ 2 χ 2 1 n 1S 2 σ 2 χ 2 n 1 N0, 1 Now we may use the fact that if U and V are independent rvs such that U χ 2 ν 1 and V χ 2 ν 2, then U/ν 1 V/ν 2 F ν1,ν 2 Here, U = ny µ 0 2 σ 2 and V = n 1S2 Hence, if H σ 2 0 is true, we have F = U/1 V/n 1 = ny µ 0 2 S 2 F 1,n 1 Therefore, we reject H 0 at a significance level α if ny µ 0 2 F s 2 1,n 1,α, where F 1,n 1,α is such that PF F 1,n 1,α = α Equivalently, we reject H 0 if ny µ0 2 that is, if s 2 t n 1, α 2, y µ 0 s2 /n t n 1, α 2 Of course, this is the usual two-sided t test It can be shown that all of the standard tests in situations with normal distributions are generalized likelihood ratio tests
4 116 CHAPTER 2 ELEMENTS OF STATISTICAL INFERENCE 264 Wilks theorem In more complex cases, we have to use the following approximation to find the critical region The result is stated without proof Theorem 29 Wilks theorem Assume that the joint distribution of Y 1,,Y n depends on p unknown parameters and that, under H 0, the joint distribution depends on p 0 unknown parameters Let ν = p p 0 Then, under some regularity conditions, when the null hypothesis is true, the distribution of the statistic 2 logλy } converges to a χ 2 ν distribution as the sample size n, ie, when H 0 is true and n is large, 2 logλy } approx χ2 ν Thus, for large n, the critical region for a test with approximate significance level α is R = y : 2 logλy} χ 2 ν,α } Example 239 Suppose that Y i iid Poissonλ and consider testing H 0 : λ = λ 0 against H 1 : λ λ 0 Then we have seen that no UMP test exists in this case Now, the likelihood is y i e nλ Lλ y = λ n n y i! The maximum likelihood estimate of λ is ˆλ = y and the restricted maximum likelihood estimate of λ under H 0 is ˆλ 0 = λ 0 Thus, the generalized likelihood ratio is n λy = Lλ 0 y = λ y i 0 e nλ 0 n y i! n Lˆλ y y i! y n y i e ny n λ0 y i = e ny λ0 y It follows that 2 logλy} = 2 ny log y = 2n y log λ0 λ 0 } + ny λ 0 y + λ 0 y }
5 26 HYPOTHESIS TESTING 117 Here, p = 1 and p 0 = 0, and so ν = 1 Therefore, by Wilks theorem, when H 0 is true and n is large, } Y 2n Y log + λ 0 Y χ 2 1 λ 0 Hence, for a test with approximate significance level α, we reject H 0 if and only if } y 2n y log + λ 0 y χ 2 1,α λ 0 Example 240 Suppose that Y i, i = 1,,n, are iid random variables with the probability mass function given by θj, if y = j, j = 1, 2, 3; PY = y = 0, otherwise, where θ j are unknown parameters such that θ 1 +θ 2 +θ 3 = 1 and θ j 0 Consider testing H 0 : θ 1 = θ 2 = θ 3 against H 1 : H 0 is not true We will use the Wilks theorem to derive the critical region for testing this hypothesis at an approximate significance level α Here the full parameter space Θ is two-dimensional because there are only two free parameters, ie, θ 3 = 1 θ 1 θ 2 and Hence p = 2 Θ = θ = θ 1, θ 2, θ 3 T : θ 3 = 1 θ 1 θ 2, θ j 0} The restricted parameter space is zero-dimensional, because under the null hypothesis all the parameters are equal and as they sum up to 1, they all must be equal to 1/3 Hence 1 Θ = θ = 3, 1 3, 1 } T 3 and so p 0 = 0 zero unknown parameters That is the number of degrees of freedom of the χ 2 distribution is ν = p p 0 = 2 To calculate λy we need to find the MLEθ in Θ and in the restricted space Θ
6 118 CHAPTER 2 ELEMENTS OF STATISTICAL INFERENCE MLEθ in Θ: The likelihood function is Lθ; y = θ n 1 1 θn θ 1 θ 2 n 3, Then, the log- where n j is the number of responses equal to j, j = 1, 2, 3 likelihood is For j = 1, 2 we have, lθ; y = n 1 logθ 1 + n 2 logθ 2 + n 3 log1 θ 1 θ 2 When compared to zero, this gives l = n j n θ j θ j 1 θ 1 θ 2 θ j n j = θ 3 n 3 = γ Now, since θ 1 + θ 2 + θ 3 = 1 we obtain γ = 1 n, where n = n 1 + n 2 + n 3 Then the estimates of the parameters are θ j = n j /n, j = 1, 2, 3 The second derivatives are 2 l θ 2 j = n j θ 2 j 2 l θ j θ i = n 3 θ 2 3 n 3 θ 2 3 The determinant of the second derivatives is positive for all θ and the element 2 l θ 2 1 is negative for all θ, so also for θ Hence, there is a maximum at θ and MLEθ in Θ : Lθ; y = MLEθ j = θ j = n j n 1 3 n1 1 3 n2 n3 1 3 That is θ 0 = 1, 1, 1 T Now, we can calculate λy: λy = L θ 0 ; y 1 n1 1 n2 1 n3 n1 n2 n3 L θ; y = n n n n1 n1 n2 n2 n3 n3 = n n n 3n 1 3n 2 3n 3
7 26 HYPOTHESIS TESTING 119 That is 2 logλy} = 2 3 n n j log 3n j We reject H 0 at an approximate significance level α if the observed sample belongs to the critical region R, where 3 } 3nj R = y : 2 n j log χ 2 2;α n 265 Contingency tables We now show that the usual test for association in contingency tables is a generalized likelihood ratio test and that its asymptotic distribution is an example of Wilks theorem Suppose that we have an r c contingency table in which there are Y ij individuals classified in row i and column j, when N individuals are classified independently Let θ ij be the corresponding probability that an individual is classified in row i and column j, so that θ ij 0 and r c θ ij = 1 Then the variables Y ij have a multinomial distribution with parameters N and θ ij for i = 1, 2,, r and j = 1, 2,, c Example 241 In an experiment 150 patients were allocated to three groups of 45, 45 and 60 patients each Two groups were given a new drug at different dose levels and the third group received placebo The responses are as follows: Improved No difference Worse r i = c y ij Placebo Half dose Full dose c j = r y ij N = 150 We are interested in testing the hypothesis that the response to the drug does not depend on the dose level
8 120 CHAPTER 2 ELEMENTS OF STATISTICAL INFERENCE The null hypothesis is that the row and column classifications are independent, and the alternative is that they are dependent More precisely, the null hypothesis is H 0 : θ ij = a i b j for some a i > 0 and b j > 0, with r a i = 1 and c b j = 1, and the alternative is H 1 : θ ij a i b j for at least one pair i, j The usual test statistic is given by X 2 = Y ij E ij 2 E ij, where E ij = R ic j N, R i = c Y ij and C j = r Y ij For large N, X 2 χ 2 r 1c 1 approximately when H 0 is true, and so we reject H 0 at the level of significance α if X 2 χ 2 r 1c 1,α Now consider the generalized likelihood ratio test Then the likelihood is Lθ y = r N! c y ij! r c θ y ij ij = A r c θ y ij ij, where the coefficient A is the number of ways that N subjects can be divided in rc groups with y ij in th-th group The log-likelihood is lθ; y = loga + y ij logθ ij We have to maximize this, subject to the constraint r c θ ij = 1 Let Σ represent the sum over all pairs i, j except r, c Then we may write lθ y = loga + Σ y ij logθ ij + y rc log 1 Σ θ ij Thus, for i, j r, c, solving the equation l = y ij y rc = y ij y rc = 0 θ ij θ ij 1 Σ θ ij θ ij θ rc
9 26 HYPOTHESIS TESTING 121 yields ˆθ ij /y ij = ˆθ rc /y rc = γ, say Since r c θ ij = r c y ijγ = 1, we have γ = 1/N, so that the maximum likelihood estimate of θ ij is ˆθ ij = y ij /N for i = 1, 2,, r and j = 1, 2,, c It follows that l θ y = loga + y ij log yij N Now, under H 0, we have θ ij = a i b j and so lθ y = loga + = loga + y ij loga i + logb j } r i loga i + c j logb j Now, we maximize this subject to the constraints r a i = 1 and c b j = 1 That is, we maximize lθ y = loga + Σ r i loga i + r r loga r + Σ c j logb j + c c logb c = loga + Σ r i loga i + r r log1 Σ a i + Σ c j logb j + c c log1 Σ b j Then, l = r i r r a i a i a r which, when compared to zero gives or However, Hence, γ = 1/N and so 1 = r i â i = r r â r â i r i = âr r r = γ â i = r i γ = Nγ â i = r i N Similarly, we obtain the ML estimates for b j as b j = c j /N
10 122 CHAPTER 2 ELEMENTS OF STATISTICAL INFERENCE Thus, the restricted maximum likelihood estimate of θ ij under H 0 is for i = 1, 2,, r and j = 1, 2,, c It follows that θ ij 0 = â i bj = r i C j N N = /N l θ 0 y = loga + y ij log eij N Hence, we obtain } L θ 0 y 2 logλy} = 2 log = 2l θ 0 y l θ y} = 2 L θ y y ij log yij Here, p = rc 1 and p 0 = r 1+c 1 = r+c 2, and so ν = r 1c 1 Therefore, by Wilks theorem, when H 0 is true and N is large, 2 Yij Y ij log E ij approx χ2 r 1c 1 The test statistics X 2 and 2 logλy} are asymptotically equivalent, that is, they differ by quantities that tend to zero as N To see this, first note that y ij is small relative to y ij and when N is large Thus, since y ij = + y ij, we may write by Taylor series expansion of log1 + x around x, cut after second order term log yij = log 1 + y ij y ij 1 2 y ij 2 e 2 ij Hence, we have y ij log yij yij + y ij } 1 2 y ij + 1 y ij 2 2 } y ij 2 e 2 ij
11 26 HYPOTHESIS TESTING 123 Since r c y ij = 0, it follows that 2 logλy} y ij 2 So we now see why we use the X 2 test statistic Example 242 continued Now we will test the hypothesis from the previous example The table of values is following Improved No difference Worse Placebo Half dose Full dose This gives X 2 obs = y ij 2 = 1417 The critical value is χ 2 4;005 = 9488, hence there is no evidence to reject the null hypothesis saying that the different responses are independent of the levels of the new drug and placebo We obtain the same conclusion using the critical region derived from the Wilks theorem, which is } yij R = y : 2 y ij log χ 2 ν;α Here 2 y ij log yij = 142
2.5.3 Generalized likelihood ratio tests
25 HYPOTHESIS TESTING 127 253 Generalized likelihood ratio tests When a UMP test does not exist, we usually use a generalized likelihood ratio test to verify H 0 : ϑ Θ against H 1 : ϑ Θ\Θ It can be used
More informationNotes on the Multivariate Normal and Related Topics
Version: July 10, 2013 Notes on the Multivariate Normal and Related Topics Let me refresh your memory about the distinctions between population and sample; parameters and statistics; population distributions
More informationLecture 21: October 19
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 21: October 19 21.1 Likelihood Ratio Test (LRT) To test composite versus composite hypotheses the general method is to use
More informationLecture 15. Hypothesis testing in the linear model
14. Lecture 15. Hypothesis testing in the linear model Lecture 15. Hypothesis testing in the linear model 1 (1 1) Preliminary lemma 15. Hypothesis testing in the linear model 15.1. Preliminary lemma Lemma
More information2.3 Methods of Estimation
96 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE 2.3 Methods of Estimation 2.3. Method of Moments The Method of Moments is a simple technique based on the idea that the sample moments are natural estimators
More informationMATH5745 Multivariate Methods Lecture 07
MATH5745 Multivariate Methods Lecture 07 Tests of hypothesis on covariance matrix March 16, 2018 MATH5745 Multivariate Methods Lecture 07 March 16, 2018 1 / 39 Test on covariance matrices: Introduction
More informationSummary of Chapters 7-9
Summary of Chapters 7-9 Chapter 7. Interval Estimation 7.2. Confidence Intervals for Difference of Two Means Let X 1,, X n and Y 1, Y 2,, Y m be two independent random samples of sizes n and m from two
More informationMath 152. Rumbos Fall Solutions to Assignment #12
Math 52. umbos Fall 2009 Solutions to Assignment #2. Suppose that you observe n iid Bernoulli(p) random variables, denoted by X, X 2,..., X n. Find the LT rejection region for the test of H o : p p o versus
More information2.5 Hypothesis Testing
118 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE 2.5 Hypothesis Testing We assume that Y 1,...,Y n have a joint distribution which depends on the unknown parametersϑ = ϑ 1,...,ϑ p ) T. The set of all possible
More informationI i=1 1 I(J 1) j=1 (Y ij Ȳi ) 2. j=1 (Y j Ȳ )2 ] = 2n( is the two-sample t-test statistic.
Serik Sagitov, Chalmers and GU, February, 08 Solutions chapter Matlab commands: x = data matrix boxplot(x) anova(x) anova(x) Problem.3 Consider one-way ANOVA test statistic For I = and = n, put F = MS
More informationLikelihood Ratio tests
Likelihood Ratio tests For general composite hypotheses optimality theory is not usually successful in producing an optimal test. instead we look for heuristics to guide our choices. The simplest approach
More informationHypothesis Testing. Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA
Hypothesis Testing Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA An Example Mardia et al. (979, p. ) reprint data from Frets (9) giving the length and breadth (in
More informationTopic 19 Extensions on the Likelihood Ratio
Topic 19 Extensions on the Likelihood Ratio Two-Sided Tests 1 / 12 Outline Overview Normal Observations Power Analysis 2 / 12 Overview The likelihood ratio test is a popular choice for composite hypothesis
More informationExercises and Answers to Chapter 1
Exercises and Answers to Chapter The continuous type of random variable X has the following density function: a x, if < x < a, f (x), otherwise. Answer the following questions. () Find a. () Obtain mean
More informationIntroductory Econometrics
Session 4 - Testing hypotheses Roland Sciences Po July 2011 Motivation After estimation, delivering information involves testing hypotheses Did this drug had any effect on the survival rate? Is this drug
More informationDiagnostics can identify two possible areas of failure of assumptions when fitting linear models.
1 Transformations 1.1 Introduction Diagnostics can identify two possible areas of failure of assumptions when fitting linear models. (i) lack of Normality (ii) heterogeneity of variances It is important
More informationTesting Hypothesis. Maura Mezzetti. Department of Economics and Finance Università Tor Vergata
Maura Department of Economics and Finance Università Tor Vergata Hypothesis Testing Outline It is a mistake to confound strangeness with mystery Sherlock Holmes A Study in Scarlet Outline 1 The Power Function
More informationStatistics 3858 : Maximum Likelihood Estimators
Statistics 3858 : Maximum Likelihood Estimators 1 Method of Maximum Likelihood In this method we construct the so called likelihood function, that is L(θ) = L(θ; X 1, X 2,..., X n ) = f n (X 1, X 2,...,
More informationChapter 7. Hypothesis Testing
Chapter 7. Hypothesis Testing Joonpyo Kim June 24, 2017 Joonpyo Kim Ch7 June 24, 2017 1 / 63 Basic Concepts of Testing Suppose that our interest centers on a random variable X which has density function
More informationLikelihood-based inference with missing data under missing-at-random
Likelihood-based inference with missing data under missing-at-random Jae-kwang Kim Joint work with Shu Yang Department of Statistics, Iowa State University May 4, 014 Outline 1. Introduction. Parametric
More informationStatistical Inference
Statistical Inference Classical and Bayesian Methods Revision Class for Midterm Exam AMS-UCSC Th Feb 9, 2012 Winter 2012. Session 1 (Revision Class) AMS-132/206 Th Feb 9, 2012 1 / 23 Topics Topics We will
More informationHypothesis Testing: The Generalized Likelihood Ratio Test
Hypothesis Testing: The Generalized Likelihood Ratio Test Consider testing the hypotheses H 0 : θ Θ 0 H 1 : θ Θ \ Θ 0 Definition: The Generalized Likelihood Ratio (GLR Let L(θ be a likelihood for a random
More informationMISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30
MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD Copyright c 2012 (Iowa State University) Statistics 511 1 / 30 INFORMATION CRITERIA Akaike s Information criterion is given by AIC = 2l(ˆθ) + 2k, where l(ˆθ)
More informationTopic 21 Goodness of Fit
Topic 21 Goodness of Fit Contingency Tables 1 / 11 Introduction Two-way Table Smoking Habits The Hypothesis The Test Statistic Degrees of Freedom Outline 2 / 11 Introduction Contingency tables, also known
More informationStat 579: Generalized Linear Models and Extensions
Stat 579: Generalized Linear Models and Extensions Mixed models Yan Lu March, 2018, week 8 1 / 32 Restricted Maximum Likelihood (REML) REML: uses a likelihood function calculated from the transformed set
More informationLikelihoods. P (Y = y) = f(y). For example, suppose Y has a geometric distribution on 1, 2,... with parameter p. Then the pmf is
Likelihoods The distribution of a random variable Y with a discrete sample space (e.g. a finite sample space or the integers) can be characterized by its probability mass function (pmf): P (Y = y) = f(y).
More informationComparing two independent samples
In many applications it is necessary to compare two competing methods (for example, to compare treatment effects of a standard drug and an experimental drug). To compare two methods from statistical point
More informationMath 181B Homework 1 Solution
Math 181B Homework 1 Solution 1. Write down the likelihood: L(λ = n λ X i e λ X i! (a One-sided test: H 0 : λ = 1 vs H 1 : λ = 0.1 The likelihood ratio: where LR = L(1 L(0.1 = 1 X i e n 1 = λ n X i e nλ
More informationClassical Inference for Gaussian Linear Models
Classical Inference for Gaussian Linear Models Surya Tokdar Gaussian linear model Data (z i, Y i ), i = 1,, n, dim(z i ) = p Model Y i = z T i Parameters: β R p, σ 2 > 0 Inference needed on η = a T β IID
More informationStatistics 3858 : Contingency Tables
Statistics 3858 : Contingency Tables 1 Introduction Before proceeding with this topic the student should review generalized likelihood ratios ΛX) for multinomial distributions, its relation to Pearson
More informationHypothesis Testing One Sample Tests
STATISTICS Lecture no. 13 Department of Econometrics FEM UO Brno office 69a, tel. 973 442029 email:jiri.neubauer@unob.cz 12. 1. 2010 Tests on Mean of a Normal distribution Tests on Variance of a Normal
More informationLecture 23 Maximum Likelihood Estimation and Bayesian Inference
Lecture 23 Maximum Likelihood Estimation and Bayesian Inference Thais Paiva STA 111 - Summer 2013 Term II August 7, 2013 1 / 31 Thais Paiva STA 111 - Summer 2013 Term II Lecture 23, 08/07/2013 Lecture
More informationLecture 3. Inference about multivariate normal distribution
Lecture 3. Inference about multivariate normal distribution 3.1 Point and Interval Estimation Let X 1,..., X n be i.i.d. N p (µ, Σ). We are interested in evaluation of the maximum likelihood estimates
More information2.3 Analysis of Categorical Data
90 CHAPTER 2. ESTIMATION AND HYPOTHESIS TESTING 2.3 Analysis of Categorical Data 2.3.1 The Multinomial Probability Distribution A mulinomial random variable is a generalization of the binomial rv. It results
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
More informationMAS3301 Bayesian Statistics
MAS3301 Bayesian Statistics M. Farrow School of Mathematics and Statistics Newcastle University Semester 2, 2008-9 1 11 Conjugate Priors IV: The Dirichlet distribution and multinomial observations 11.1
More informationNuisance parameters and their treatment
BS2 Statistical Inference, Lecture 2, Hilary Term 2008 April 2, 2008 Ancillarity Inference principles Completeness A statistic A = a(x ) is said to be ancillary if (i) The distribution of A does not depend
More informationMath 494: Mathematical Statistics
Math 494: Mathematical Statistics Instructor: Jimin Ding jmding@wustl.edu Department of Mathematics Washington University in St. Louis Class materials are available on course website (www.math.wustl.edu/
More informationHypothesis Test. The opposite of the null hypothesis, called an alternative hypothesis, becomes
Neyman-Pearson paradigm. Suppose that a researcher is interested in whether the new drug works. The process of determining whether the outcome of the experiment points to yes or no is called hypothesis
More informationAnswer Key for STAT 200B HW No. 7
Answer Key for STAT 200B HW No. 7 May 5, 2007 Problem 2.2 p. 649 Assuming binomial 2-sample model ˆπ =.75, ˆπ 2 =.6. a ˆτ = ˆπ 2 ˆπ =.5. From Ex. 2.5a on page 644: ˆπ ˆπ + ˆπ 2 ˆπ 2.75.25.6.4 = + =.087;
More informationMath 494: Mathematical Statistics
Math 494: Mathematical Statistics Instructor: Jimin Ding jmding@wustl.edu Department of Mathematics Washington University in St. Louis Class materials are available on course website (www.math.wustl.edu/
More informationCherry Blossom run (1) The credit union Cherry Blossom Run is a 10 mile race that takes place every year in D.C. In 2009 there were participants
18.650 Statistics for Applications Chapter 5: Parametric hypothesis testing 1/37 Cherry Blossom run (1) The credit union Cherry Blossom Run is a 10 mile race that takes place every year in D.C. In 2009
More informationEstimation MLE-Pandemic data MLE-Financial crisis data Evaluating estimators. Estimation. September 24, STAT 151 Class 6 Slide 1
Estimation September 24, 2018 STAT 151 Class 6 Slide 1 Pandemic data Treatment outcome, X, from n = 100 patients in a pandemic: 1 = recovered and 0 = not recovered 1 1 1 0 0 0 1 1 1 0 0 1 0 1 0 0 1 1 1
More informationCh. 5 Hypothesis Testing
Ch. 5 Hypothesis Testing The current framework of hypothesis testing is largely due to the work of Neyman and Pearson in the late 1920s, early 30s, complementing Fisher s work on estimation. As in estimation,
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationAnswer Key for STAT 200B HW No. 8
Answer Key for STAT 200B HW No. 8 May 8, 2007 Problem 3.42 p. 708 The values of Ȳ for x 00, 0, 20, 30 are 5/40, 0, 20/50, and, respectively. From Corollary 3.5 it follows that MLE exists i G is identiable
More information2014/2015 Smester II ST5224 Final Exam Solution
014/015 Smester II ST54 Final Exam Solution 1 Suppose that (X 1,, X n ) is a random sample from a distribution with probability density function f(x; θ) = e (x θ) I [θ, ) (x) (i) Show that the family of
More informationStatistics. Statistics
The main aims of statistics 1 1 Choosing a model 2 Estimating its parameter(s) 1 point estimates 2 interval estimates 3 Testing hypotheses Distributions used in statistics: χ 2 n-distribution 2 Let X 1,
More informationBIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation
BIO5312 Biostatistics Lecture 13: Maximum Likelihood Estimation Yujin Chung November 29th, 2016 Fall 2016 Yujin Chung Lec13: MLE Fall 2016 1/24 Previous Parametric tests Mean comparisons (normality assumption)
More informationF & B Approaches to a simple model
A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 215 http://www.astro.cornell.edu/~cordes/a6523 Lecture 11 Applications: Model comparison Challenges in large-scale surveys
More informationLecture 10: Generalized likelihood ratio test
Stat 200: Introduction to Statistical Inference Autumn 2018/19 Lecture 10: Generalized likelihood ratio test Lecturer: Art B. Owen October 25 Disclaimer: These notes have not been subjected to the usual
More informationBIOS 2083 Linear Models c Abdus S. Wahed
Chapter 5 206 Chapter 6 General Linear Model: Statistical Inference 6.1 Introduction So far we have discussed formulation of linear models (Chapter 1), estimability of parameters in a linear model (Chapter
More informationIntroduction to Estimation Methods for Time Series models Lecture 2
Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:
More information4.5.1 The use of 2 log Λ when θ is scalar
4.5. ASYMPTOTIC FORM OF THE G.L.R.T. 97 4.5.1 The use of 2 log Λ when θ is scalar Suppose we wish to test the hypothesis NH : θ = θ where θ is a given value against the alternative AH : θ θ on the basis
More informationsimple if it completely specifies the density of x
3. Hypothesis Testing Pure significance tests Data x = (x 1,..., x n ) from f(x, θ) Hypothesis H 0 : restricts f(x, θ) Are the data consistent with H 0? H 0 is called the null hypothesis simple if it completely
More informationEconomics 520. Lecture Note 19: Hypothesis Testing via the Neyman-Pearson Lemma CB 8.1,
Economics 520 Lecture Note 9: Hypothesis Testing via the Neyman-Pearson Lemma CB 8., 8.3.-8.3.3 Uniformly Most Powerful Tests and the Neyman-Pearson Lemma Let s return to the hypothesis testing problem
More informationHypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3
Hypothesis Testing CB: chapter 8; section 0.3 Hypothesis: statement about an unknown population parameter Examples: The average age of males in Sweden is 7. (statement about population mean) The lowest
More informationFoundations of Statistical Inference
Foundations of Statistical Inference Julien Berestycki Department of Statistics University of Oxford MT 2015 Julien Berestycki (University of Oxford) SB2a MT 2015 1 / 16 Lecture 16 : Bayesian analysis
More informationSTAT 514 Solutions to Assignment #6
STAT 514 Solutions to Assignment #6 Question 1: Suppose that X 1,..., X n are a simple random sample from a Weibull distribution with density function f θ x) = θcx c 1 exp{ θx c }I{x > 0} for some fixed
More informationComposite Hypotheses and Generalized Likelihood Ratio Tests
Composite Hypotheses and Generalized Likelihood Ratio Tests Rebecca Willett, 06 In many real world problems, it is difficult to precisely specify probability distributions. Our models for data may involve
More informationHomework 7: Solutions. P3.1 from Lehmann, Romano, Testing Statistical Hypotheses.
Stat 300A Theory of Statistics Homework 7: Solutions Nikos Ignatiadis Due on November 28, 208 Solutions should be complete and concisely written. Please, use a separate sheet or set of sheets for each
More information(a) (3 points) Construct a 95% confidence interval for β 2 in Equation 1.
Problem 1 (21 points) An economist runs the regression y i = β 0 + x 1i β 1 + x 2i β 2 + x 3i β 3 + ε i (1) The results are summarized in the following table: Equation 1. Variable Coefficient Std. Error
More informationLecture 26: Likelihood ratio tests
Lecture 26: Likelihood ratio tests Likelihood ratio When both H 0 and H 1 are simple (i.e., Θ 0 = {θ 0 } and Θ 1 = {θ 1 }), Theorem 6.1 applies and a UMP test rejects H 0 when f θ1 (X) f θ0 (X) > c 0 for
More informationCategorical Variables and Contingency Tables: Description and Inference
Categorical Variables and Contingency Tables: Description and Inference STAT 526 Professor Olga Vitek March 3, 2011 Reading: Agresti Ch. 1, 2 and 3 Faraway Ch. 4 3 Univariate Binomial and Multinomial Measurements
More informationFinal Exam. 1. (6 points) True/False. Please read the statements carefully, as no partial credit will be given.
1. (6 points) True/False. Please read the statements carefully, as no partial credit will be given. (a) If X and Y are independent, Corr(X, Y ) = 0. (b) (c) (d) (e) A consistent estimator must be asymptotically
More informationCentral Limit Theorem ( 5.3)
Central Limit Theorem ( 5.3) Let X 1, X 2,... be a sequence of independent random variables, each having n mean µ and variance σ 2. Then the distribution of the partial sum S n = X i i=1 becomes approximately
More informationMathematical statistics
October 1 st, 2018 Lecture 11: Sufficient statistic Where are we? Week 1 Week 2 Week 4 Week 7 Week 10 Week 14 Probability reviews Chapter 6: Statistics and Sampling Distributions Chapter 7: Point Estimation
More information2017 Financial Mathematics Orientation - Statistics
2017 Financial Mathematics Orientation - Statistics Written by Long Wang Edited by Joshua Agterberg August 21, 2018 Contents 1 Preliminaries 5 1.1 Samples and Population............................. 5
More informationSTT 843 Key to Homework 1 Spring 2018
STT 843 Key to Homework Spring 208 Due date: Feb 4, 208 42 (a Because σ = 2, σ 22 = and ρ 2 = 05, we have σ 2 = ρ 2 σ σ22 = 2/2 Then, the mean and covariance of the bivariate normal is µ = ( 0 2 and Σ
More informationGeneral Linear Model: Statistical Inference
Chapter 6 General Linear Model: Statistical Inference 6.1 Introduction So far we have discussed formulation of linear models (Chapter 1), estimability of parameters in a linear model (Chapter 4), least
More informationPart 1.) We know that the probability of any specific x only given p ij = p i p j is just multinomial(n, p) where p k1 k 2
Problem.) I will break this into two parts: () Proving w (m) = p( x (m) X i = x i, X j = x j, p ij = p i p j ). In other words, the probability of a specific table in T x given the row and column counts
More informationGARCH Models Estimation and Inference
GARCH Models Estimation and Inference Eduardo Rossi University of Pavia December 013 Rossi GARCH Financial Econometrics - 013 1 / 1 Likelihood function The procedure most often used in estimating θ 0 in
More informationMaximum Likelihood Large Sample Theory
Maximum Likelihood Large Sample Theory MIT 18.443 Dr. Kempthorne Spring 2015 1 Outline 1 Large Sample Theory of Maximum Likelihood Estimates 2 Asymptotic Results: Overview Asymptotic Framework Data Model
More informationEstimation Theory. as Θ = (Θ 1,Θ 2,...,Θ m ) T. An estimator
Estimation Theory Estimation theory deals with finding numerical values of interesting parameters from given set of data. We start with formulating a family of models that could describe how the data were
More informationRegression Estimation - Least Squares and Maximum Likelihood. Dr. Frank Wood
Regression Estimation - Least Squares and Maximum Likelihood Dr. Frank Wood Least Squares Max(min)imization Function to minimize w.r.t. β 0, β 1 Q = n (Y i (β 0 + β 1 X i )) 2 i=1 Minimize this by maximizing
More informationSummary of Chapter 7 (Sections ) and Chapter 8 (Section 8.1)
Summary of Chapter 7 (Sections 7.2-7.5) and Chapter 8 (Section 8.1) Chapter 7. Tests of Statistical Hypotheses 7.2. Tests about One Mean (1) Test about One Mean Case 1: σ is known. Assume that X N(µ, σ
More informationLecture 2: Basic Concepts and Simple Comparative Experiments Montgomery: Chapter 2
Lecture 2: Basic Concepts and Simple Comparative Experiments Montgomery: Chapter 2 Fall, 2013 Page 1 Random Variable and Probability Distribution Discrete random variable Y : Finite possible values {y
More informationMcGill University. Faculty of Science. Department of Mathematics and Statistics. Part A Examination. Statistics: Theory Paper
McGill University Faculty of Science Department of Mathematics and Statistics Part A Examination Statistics: Theory Paper Date: 10th May 2015 Instructions Time: 1pm-5pm Answer only two questions from Section
More informationHypothesis Testing. A rule for making the required choice can be described in two ways: called the rejection or critical region of the test.
Hypothesis Testing Hypothesis testing is a statistical problem where you must choose, on the basis of data X, between two alternatives. We formalize this as the problem of choosing between two hypotheses:
More informationMinimum Message Length Inference and Mixture Modelling of Inverse Gaussian Distributions
Minimum Message Length Inference and Mixture Modelling of Inverse Gaussian Distributions Daniel F. Schmidt Enes Makalic Centre for Molecular, Environmental, Genetic & Analytic (MEGA) Epidemiology School
More information[y i α βx i ] 2 (2) Q = i=1
Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation
More informationSTAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots. March 8, 2015
STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots March 8, 2015 The duality between CI and hypothesis testing The duality between CI and hypothesis
More informationStatistical Data Analysis Stat 3: p-values, parameter estimation
Statistical Data Analysis Stat 3: p-values, parameter estimation London Postgraduate Lectures on Particle Physics; University of London MSci course PH4515 Glen Cowan Physics Department Royal Holloway,
More informationGeneralized Linear Models Introduction
Generalized Linear Models Introduction Statistics 135 Autumn 2005 Copyright c 2005 by Mark E. Irwin Generalized Linear Models For many problems, standard linear regression approaches don t work. Sometimes,
More informationCategorical Data Analysis Chapter 3
Categorical Data Analysis Chapter 3 The actual coverage probability is usually a bit higher than the nominal level. Confidence intervals for association parameteres Consider the odds ratio in the 2x2 table,
More informationLecture 1 October 9, 2013
Probabilistic Graphical Models Fall 2013 Lecture 1 October 9, 2013 Lecturer: Guillaume Obozinski Scribe: Huu Dien Khue Le, Robin Bénesse The web page of the course: http://www.di.ens.fr/~fbach/courses/fall2013/
More informationProblems ( ) 1 exp. 2. n! e λ and
Problems The expressions for the probability mass function of the Poisson(λ) distribution, and the density function of the Normal distribution with mean µ and variance σ 2, may be useful: ( ) 1 exp. 2πσ
More informationDefinition 3.1 A statistical hypothesis is a statement about the unknown values of the parameters of the population distribution.
Hypothesis Testing Definition 3.1 A statistical hypothesis is a statement about the unknown values of the parameters of the population distribution. Suppose the family of population distributions is indexed
More informationStatistics 135 Fall 2008 Final Exam
Name: SID: Statistics 135 Fall 2008 Final Exam Show your work. The number of points each question is worth is shown at the beginning of the question. There are 10 problems. 1. [2] The normal equations
More information2.3.3 Method of Least Squares
.3. METHODS OF ESTIMATION 105.3.3 Method of Least Squares IfY 1,...,Y n are independent random variables, which have the same variance and higher-order moments, and, if each E(Y i ) is a linear function
More informationST495: Survival Analysis: Hypothesis testing and confidence intervals
ST495: Survival Analysis: Hypothesis testing and confidence intervals Eric B. Laber Department of Statistics, North Carolina State University April 3, 2014 I remember that one fateful day when Coach took
More informationSTAT 512 sp 2018 Summary Sheet
STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}
More informationEpidemiology Principles of Biostatistics Chapter 10 - Inferences about two populations. John Koval
Epidemiology 9509 Principles of Biostatistics Chapter 10 - Inferences about John Koval Department of Epidemiology and Biostatistics University of Western Ontario What is being covered 1. differences in
More informationAccurate directional inference for vector parameters
Accurate directional inference for vector parameters Nancy Reid October 28, 2016 with Don Fraser, Nicola Sartori, Anthony Davison Parametric models and likelihood model f (y; θ), θ R p data y = (y 1,...,
More informationST3241 Categorical Data Analysis I Two-way Contingency Tables. 2 2 Tables, Relative Risks and Odds Ratios
ST3241 Categorical Data Analysis I Two-way Contingency Tables 2 2 Tables, Relative Risks and Odds Ratios 1 What Is A Contingency Table (p.16) Suppose X and Y are two categorical variables X has I categories
More informationLikelihood inference in the presence of nuisance parameters
Likelihood inference in the presence of nuisance parameters Nancy Reid, University of Toronto www.utstat.utoronto.ca/reid/research 1. Notation, Fisher information, orthogonal parameters 2. Likelihood inference
More informationDiscrete Multivariate Statistics
Discrete Multivariate Statistics Univariate Discrete Random variables Let X be a discrete random variable which, in this module, will be assumed to take a finite number of t different values which are
More informationMethods for Statistical Prediction Financial Time Series I. Topic 1: Review on Hypothesis Testing
Methods for Statistical Prediction Financial Time Series I Topic 1: Review on Hypothesis Testing Hung Chen Department of Mathematics National Taiwan University 9/26/2002 OUTLINE 1. Fundamental Concepts
More informationHypothesis Testing Chap 10p460
Hypothesis Testing Chap 1p46 Elements of a statistical test p462 - Null hypothesis - Alternative hypothesis - Test Statistic - Rejection region Rejection Region p462 The rejection region (RR) specifies
More informationMathematics Qualifying Examination January 2015 STAT Mathematical Statistics
Mathematics Qualifying Examination January 2015 STAT 52800 - Mathematical Statistics NOTE: Answer all questions completely and justify your derivations and steps. A calculator and statistical tables (normal,
More information