Session 4 - Testing hypotheses Roland Sciences Po July 2011
Motivation After estimation, delivering information involves testing hypotheses Did this drug had any effect on the survival rate? Is this drug more efficient than that one? The approach is close to the one of confidence intervals
Framework After the estimation phase, we obtain an estimator ˆθ of the population parameter θ The test aims at answering the question: Do θ belong to subset Θ 0? Or in particular, is θ equal to t? Assume that the parameter θ takes value in Θ R
Null hypothesis Two hypotheses are separated The null hypothesis, H 0 : θ Θ 0 The alternative hypothesis, H a : θ Θ \ Θ 0 The hypotheses are asymmetric
Rejecting Hypothesis testing is about making decisions The decisions is either To reject H0 Not to reject H0 Asymmetry: you can never accept H 0 (or H a )
Decision making To be more formal, hypothesis testing can be viewed as to process to make decision, namely to reject or not H 0, given the data In this sense, you could also consider the decision as a rv
Test statistics Any kind of test involve a test statistic A test statistic is a rv, function of the sample S = s(y 1, Y 2,... Y n ) Usually, the test statistic is chosen so that its behavior is very different under H 0 and H a More precisely, the distribution of S should be known under H 0 : f S Given the statistic S, the critical region W is defined as the values of the statistic such that the decision is taken to reject
Tests and mistakes There exist two errors associated with hypotheses testing Type I error is rejecting H 0 while it is actually true Type II error is failing to reject H 0 while it is actually false
Truth table Decision Truth H 0 true H 0 false Don t reject ok Type II Reject Type I ok
Significance level The Type I error, for historical and practical reasons, is believed to be the most problematic Thus, the practioner, before performing the test, chooses the significance level of the test α, equal to the probability of a Type I error. α = P(RejectH 0 H 0 )
Power The test statistic and the rejection region should be chosen so that the probability of a Type I error is equal to α the probability of a Type II error is minimized The power, which is equal to one minus the probability of a Type II error, should be maximized
Summary 1. Choose the null hypothesis one wants to tests, and the alternative hypothesis one wants to test against 2. Fix the significance level 3. Choose the test statistic and the critical region so that the power of the test is maximized 4. Compute the test statistic based on your data 5. Compare with the critical region 6. Make the decision: rejection or not
The sample mean A simple example: the sample mean Ȳ based on {Y 1,... Y n } iid in a N(µ, σ 2 ) We want to test whether H 0 : µ = 1, vs H a : µ 1 The level of significance is, for instance, fixed at α = 1% Usually, one chooses the standardized version, under H 0 of Ȳ as the test statistic: T = n(ȳ 1)/σ
The t-statistic As σ is unknown, it is estimated by the sample standard deviation S and the statistic is: T = n Ȳ 1 S Under H 0, T has a t n 1 distribution Under H a, T will more often take larger positive or negative values than a typical t n 1 distribution The rejection rule will be: if the statistic is larger, in absolute terms, than a threshold, then we will reject the null
The t-statistic (2) The critical region will be W = (, c) (c, ) c is determined by α, the significance level, such that P(W ) = α This is given by the properties of the t n 1 distribution
p-values For a given H 0 and a given statistic, the choice of significance level determines the result of the test Instead of fixing the significance level ex ante, the practioner may wonder: What is the lowest level of significance at which you can still reject the null? This value, at which one the decision switches, is known as the p-value