parameter space Θ, depending only on X, such that Note: it is not θ that is random, but the set C(X).

Similar documents
Lecture 28: Asymptotic confidence sets

Final Exam. 1. (6 points) True/False. Please read the statements carefully, as no partial credit will be given.

simple if it completely specifies the density of x

Spring 2012 Math 541B Exam 1

A Very Brief Summary of Statistical Inference, and Examples

Testing Hypothesis. Maura Mezzetti. Department of Economics and Finance Università Tor Vergata

Chapter 8 - Statistical intervals for a single sample

2014/2015 Smester II ST5224 Final Exam Solution

MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30

(a) (3 points) Construct a 95% confidence interval for β 2 in Equation 1.

STAT 135 Lab 5 Bootstrapping and Hypothesis Testing

Hypothesis Test. The opposite of the null hypothesis, called an alternative hypothesis, becomes

Statistics Ph.D. Qualifying Exam: Part I October 18, 2003

Chapters 9. Properties of Point Estimators

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.

Lecture 32: Asymptotic confidence sets and likelihoods

DA Freedman Notes on the MLE Fall 2003

McGill University. Faculty of Science. Department of Mathematics and Statistics. Part A Examination. Statistics: Theory Paper

STAT 461/561- Assignments, Year 2015

STATS 200: Introduction to Statistical Inference. Lecture 29: Course review

Statistical Inference

Interval Estimation. Chapter 9

Mathematics Ph.D. Qualifying Examination Stat Probability, January 2018

A Very Brief Summary of Statistical Inference, and Examples


Comparing two independent samples

Review: General Approach to Hypothesis Testing. 1. Define the research question and formulate the appropriate null and alternative hypotheses.

Lecture 3. Inference about multivariate normal distribution

Mathematics Qualifying Examination January 2015 STAT Mathematical Statistics

Statistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation

STATISTICS SYLLABUS UNIT I

Statistics Ph.D. Qualifying Exam

Economics 583: Econometric Theory I A Primer on Asymptotics: Hypothesis Testing

Review Quiz. 1. Prove that in a one-dimensional canonical exponential family, the complete and sufficient statistic achieves the

Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Hypothesis testing. Anna Wegloop Niels Landwehr/Tobias Scheffer

A union of Bayesian, frequentist and fiducial inferences by confidence distribution and artificial data sampling

Probability and Statistics qualifying exam, May 2015

Chapter 9: Hypothesis Testing Sections

Stat 5102 Final Exam May 14, 2015

Loglikelihood and Confidence Intervals

ST495: Survival Analysis: Hypothesis testing and confidence intervals

Two hours. To be supplied by the Examinations Office: Mathematical Formula Tables THE UNIVERSITY OF MANCHESTER. 21 June :45 11:45

For iid Y i the stronger conclusion holds; for our heuristics ignore differences between these notions.

Review. December 4 th, Review

Spring 2012 Math 541A Exam 1. X i, S 2 = 1 n. n 1. X i I(X i < c), T n =

Visual interpretation with normal approximation

Hypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3

Mathematical statistics

f(x θ)dx with respect to θ. Assuming certain smoothness conditions concern differentiating under the integral the integral sign, we first obtain

Lecture 13: p-values and union intersection tests

Exercises Chapter 4 Statistical Hypothesis Testing

Institute of Actuaries of India

Statement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.

STAT 135 Lab 2 Confidence Intervals, MLE and the Delta Method

Problem Selected Scores

Cherry Blossom run (1) The credit union Cherry Blossom Run is a 10 mile race that takes place every year in D.C. In 2009 there were participants

Statistics Ph.D. Qualifying Exam: Part II November 9, 2002

1 Likelihood. 1.1 Likelihood function. Likelihood & Maximum Likelihood Estimators

Chapter 3 sections. SKIP: 3.10 Markov Chains. SKIP: pages Chapter 3 - continued

STAT 135 Lab 3 Asymptotic MLE and the Method of Moments

40.530: Statistics. Professor Chen Zehua. Singapore University of Design and Technology

Likelihood-based inference with missing data under missing-at-random

Math 494: Mathematical Statistics

Stat 5101 Lecture Notes

Ch. 5 Hypothesis Testing

Chapter 7. Confidence Sets Lecture 30: Pivotal quantities and confidence sets

Modelling of Dependent Credit Rating Transitions

HT Introduction. P(X i = x i ) = e λ λ x i

Statistics and econometrics

MAS223 Statistical Inference and Modelling Exercises

Lecture 16 November Application of MoUM to our 2-sided testing problem

BTRY 4090: Spring 2009 Theory of Statistics

Master s Written Examination

Let us first identify some classes of hypotheses. simple versus simple. H 0 : θ = θ 0 versus H 1 : θ = θ 1. (1) one-sided

The University of Hong Kong Department of Statistics and Actuarial Science STAT2802 Statistical Models Tutorial Solutions Solutions to Problems 71-80

Some General Types of Tests

Greene, Econometric Analysis (6th ed, 2008)

Statistics. Statistics

Part III. A Decision-Theoretic Approach and Bayesian testing

Pivotal Quantities. Mathematics 47: Lecture 16. Dan Sloughter. Furman University. March 30, 2006

You can compute the maximum likelihood estimate for the correlation

STAT 512 sp 2018 Summary Sheet

SYSM 6303: Quantitative Introduction to Risk and Uncertainty in Business Lecture 4: Fitting Data to Distributions

Chapter 6. Hypothesis Tests Lecture 20: UMP tests and Neyman-Pearson lemma

Sampling distribution of GLM regression coefficients

MS&E 226: Small Data

Political Science 236 Hypothesis Testing: Review and Bootstrapping

Problem 1 (20) Log-normal. f(x) Cauchy

Lecture 17: Likelihood ratio and asymptotic tests

Math 494: Mathematical Statistics

Lecture 8 Inequality Testing and Moment Inequality Models

Semester , Example Exam 1

Chapter 3 sections. SKIP: 3.10 Markov Chains. SKIP: pages Chapter 3 - continued

Econ 583 Homework 7 Suggested Solutions: Wald, LM and LR based on GMM and MLE

Mathematical statistics

Statistics GIDP Ph.D. Qualifying Exam Theory Jan 11, 2016, 9:00am-1:00pm

Lecture 26: Likelihood ratio tests

Statistics. Lecture 2 August 7, 2000 Frank Porter Caltech. The Fundamentals; Point Estimation. Maximum Likelihood, Least Squares and All That

This does not cover everything on the final. Look at the posted practice problems for other topics.

TOLERANCE INTERVALS FOR DISCRETE DISTRIBUTIONS IN EXPONENTIAL FAMILIES

Transcription:

4. Interval estimation The goal for interval estimation is to specify the accurary of an estimate. A 1 α confidence set for a parameter θ is a set C(X) in the parameter space Θ, depending only on X, such that P θ (θ C(X)) = 1 α. Note: it is not θ that is random, but the set C(X). 1

For a scalar θ we would usually like to find an interval C(X) = [l(x), u(x)] so that P θ (θ [l(x), u(x)]) = 1 α. Then [l(x), u(x)] is an interval estimator or confidence interval for θ; and the observed interval [l(x), u(x)] is an interval estimate. If l is or if u is +, then we have a one-sided estimator/estimate. If l is, we have an upper confidence interval, if u is +, we have an lower confidence interval. 2

Example: Normal, unknown mean and variance. Let X 1,..., X n be a random sample from N (µ, σ 2 ), where both µ and σ 2 are unknown. Then (X µ)/(s/ n) t n 1 and so 1 α = P µ,σ 2 X µ S/ n t n 1,1 α/2 = P µ,σ 2(X t n 1,1 α/2 S/ n µ X + t n 1,1 α/2 S/ n), and so the (familiar) interval with end points X ± t n 1,1 α/2 S/ n is a 1 α confidence interval for µ. 3

Construction of confidence sets Pivotal quantities A pivotal quantity (or pivot) is a random variable t(x, θ) whose distribution is independent of all parameters, and so it has the same distribution for all θ. Example: (X µ)/(s/ n) in the example above has t n 1 -distribution if the random sample comes from N (µ, σ 2 ). 4

We use pivotal quantities to construct confidence sets, as follows. Suppose θ is a scalar. Choose a, b such that P θ (a t(x, θ) b) = 1 α. Manipulate this equation to give P θ (l(x) θ u(x)) = 1 α (if t is a monotonic function of θ); then [l(x), u(x)] is a 1 α confidence interval for θ. 5

Example: Exponential random sample. Let X 1,..., X n be a random sample from an exponential distribution with unknown mean µ. Then we know that nx/µ Gamma(n, 1). If the α-quantile of Gamma(n, 1) is denoted by g n,α then 1 α = P µ (nx/µ g n,α ) = P µ (µ nx/g n,α ). Hence [0, nx/g n,α ] is a 1 α confidence interval for µ. Alternatively, we say that nx/g n,α is the upper 1 α confidence limit for µ. 6

Confidence sets derived from point estimators Suppose ˆθ(X) is an estimator for a scalar θ, from a known distribution. Then we can take our confidence interval as [ˆθ a 1 α, ˆθ + b 1 α ] where a 1 α and b 1 α are chosen suitably. If ˆθ N(θ, v), perhaps approximately, then for a symmetric interval choose a 1 α = b 1 α = z 1 α/2 v. Note: [ˆθ a 1 α, ˆθ + b 1 α ] is not immediately a confidence interval for θ if v depends on θ: in that case replace v(θ) by v(ˆθ), which is a further approximation. 7

Approximate confidence intervals Sometimes we do not have an exact distribution available, but normal approximation is known to hold. Example: asymptotic normality of m.l.e. We have seen that, under regularity, ˆθ N (θ, I 1 (θ)). If θ is scalar, then (under regularity) ˆθ ± z 1 α/2 / I(ˆθ) is an approximate 1 α confidence interval for θ. 8

Sometimes we can improve the accuracy by applying (monotone) transformation of the estimator, using the delta method, and inverting the transformation to get the final result. As a guide line for transformations, in general a normal approximation should be used on a scale where a quantity ranges over (, ). 9

Example: Bivariate normal distribution. Let (X i, Y i ), i = 1,..., n, be a random sample from a bivariate normal distribution, with unknown mean vector and covariance matrix. The parameter of interest is ρ, the bivariate normal correlation. The MLE for ρ is the sample correlation R = n i=1 (X i X)(Y i Y ) n i=1 (X i X) 2 n i=1 (Y i Y ) 2, whose range is [ 1, 1]. For large n, R N(ρ, (1 ρ 2 ) 2 /n), using the expected Fisher information matrix to obtain an approximate variance (see the section on asymptotic theory). But the distribution of R is very skewed, the approximation is poor unless n is very large. 10

For a variable whose range is (, ), we use the tranformation Z = 1 2 log[(1 + R)/(1 R)]; this transformation is called the Fisher z transformation. By the delta method, Z N(ζ, 1/n) where ζ = 1 2 log[(1 + ρ)/(1 ρ)]. So a 1 α confidence interval for ρ can be calculated as follows: for ζ compute the interval limits Z ± z 1 α/2 / n, then transform these to the ρ scale using the inverse transformation ρ = (e 2ζ 1)/(e 2ζ +1). 11

Confidence intervals derived from hypothesis tests Define C(X) to be the set of values of θ 0 for which H 0 would not be rejected in size-α tests of H 0 : θ = θ 0. Here the form of the 1 α confidence set obtained depends on the alternative hypotheses. Example: to produce an interval with finite upper and lower limits use H 1 : θ θ 0 ; to find an upper confidence limit use H 1 : θ < θ 0. 12

Example: Normal, known variance, unknown mean. Let X 1,..., X n N(µ, σ 2 ) be i.i.d., where σ 2 known. For H 0 : µ = µ 0 versus H 1 : µ µ 0 the usual test has an acceptance region of the form X µ 0 σ/ n z 1 α/2. So the values of µ 0 for which H 0 is accepted are those in the interval [X z 1 α/2 σ/ n, X + z 1 α/2 σ/ n]; this interval is a 100(1 α)% confidence interval for µ. 13

For H 0 : µ = µ 0 versus H 1 : µ < µ 0 the UMP test accepts H 0 if X µ 0 z 1 α/2 σ/ n i.e., if µ 0 X + z 1 α/2 σ/ n. So an upper 1 α confidence limit for µ is X + z 1 α/2 σ/ n. 14

Hypothesis test from confidence regions Conversely, we can also construct tests based on confidence interval: For H 0 : θ = θ 0 against H 1 : θ θ 0, if C(X) is 100(1 α)% two-sided confidence region for θ, then for a size α test reject H 0 if θ 0 C(X): The confidence region is the acceptance region for the test. If θ is a scalar: For H 0 : θ = θ 0 against H 1 : θ < θ 0, if C(X) is 100(1 α)% upper confidence region for θ, then for a size α test reject H 0 if θ 0 C(X). 15

Example: Normal, known variance. Let X 1,..., X n N(µ, σ 2 ) be i.i.d., where σ 2 is known. For H 0 : µ = µ 0 versus H 1 : µ µ 0 the usual 100(1 α)% confidence region is [X z 1 α/2 σ/ n, X + z 1 α/2 σ/ n], so reject H 0 if X µ 0 σ/ n > z 1 α/2. 16

To test H 0 : µ = µ 0 versus H 1 : µ < µ 0 : an upper 100(1 α)% confidence region is X + z 1 α/2 σ/ n, so reject H 0 if µ 0 > X + z 1 α/2 σ/ n i.e. if X < µ 0 z 1 α/2 σ/ n. 17

We can also construct approximate hypothesis test based on approximate confidence intervals. Example: Use asymptotic normality of m.l.e. to derive a Wald test. 18

Prediction Sets What is a set of plausible values for a future data value? A 1 α prediction set for an unobserved random variable X n+1 based on the observed data X = (X 1,..., X n ) is a random set P (X) for which P(X n+1 P (X)) = 1 α. Sometimes such a set can be derived by finding a prediction pivot t(x, X n+1 ) whose distribution does not depend on θ. If a set R is such that P(t(X, X n+1 ) R) = 1 α, then a 1 α prediction set is P (X) = {X n+1 : t(x, X n+1 ) R}. 19

Example: Normal, unknown mean and variance. Let X 1,..., X n N(µ, σ 2 ) be i.i.d., where both µ and σ 2 are unknown. A possible prediction pivot is t(x, X n+1 ) = X n+1 X S 1 + 1 n. As X N(µ, σ2 n ) and X n+1 N(µ, σ 2 ) is independent of X, it follows that X n+1 X N(0, σ 2 (1 + 1/n)), and so t(x, X n+1 ) has t n 1 distribution. Hence a 1 α prediction interval is {X n+1 : t(x, X n+1 ) t n 1,1 α/2 } = X n+1 : X S 1 + 1 n t n 1,1 α/2 X n+1 X + S 1 + 1 n t n 1,1 α/2. 20