Hypothesis testing: theory and methods

Statistical Methods Warsaw School of Economics November 3, 2017

Statistical hypothesis is the name of any conjecture about unknown parameters of a population distribution. The hypothesis should be verifiable (true or false) on the basis of sample. Permissible hypothesis set - ω is a set of possible distributions that might characterized the population. Parametric hypotheses - are those related to parameters of a distribution, e.g. θ Non-Parametric hypotheses - are those related to the family of distributions that characterize a population, e.g. Poisson vs Negative Binomial.

Statistical Inference theory is based on the assumption of having a simple random sample. Simple hypothesis are statistical hypothesis that unambiguously define the population distribution. Composite hypotheses are hypothesis that are not simple. Statistical hypotheses are of the form H : F (x) ω, where ω Ω. If the set ω consists of a single element, then the hypothesis is simple.

Example of hypothesis 1. X 1, X 2,..., X 3 are drawn from a normal distribution. 2. X 1, X 2,..., X 3 are drawn from a normal distribution with parameter µ = 3 3. X 1, X 2,..., X n are independent. 4. E(X ) = E(Y ) 5. Var(X ) > 1

Null Hypothesis denoted H 0 is the hypothesis that we want to verify. H 0 : F (x) ω 0, ω 0 Ω Alternative Hypothesis denoted H 1 is the hypothesis that we are ready to accept in the case H 0 is rejected by the data. H 1 : F (x) ω 1, ω 1 Ω, ω 0 ω 1 = In parametric hypothesis H 0 : θ Θ 0 and H 1 : θ Θ c 0, where Θ 0 is a subset of the parameter space and Θ c 0 is the complement

Statistical test is a rule that allows to decide for which sample values we decide to reject H 0 and for which we don t reject H 0 is called (non)parametric if verifies (non)parametric hypothesis. Rejection region, aka Critical region the subset of sample values (w) for which we reject H 0. Test statistic a function of the sample on which we specified the statistical text W (X 1,..., X n ) = W (X).

Example A researcher believes that the duration of phone talks follows an exponential distribution with a parameter λ. On the basis of a single call, she would like to test whether λ 2. She is ready to reject the null hypothesis, whenever the duration is above c. 1. Identify the statistical model 2. Write down H 0 and H 1 3. Identify the test statistic and the rejection region

Example A researcher believes that the duration of phone talks follows an exponential distribution with a parameter λ. On the basis of a single call, she would like to test whether λ 2. She is ready to reject the null hypothesis, whenever the duration is above c. 1. Identify the statistical model Sample space = (0, ) 20 ; Parameter space = λ(0, ); P λ = exp 2. Write down H 0 and H 1 3. Identify the test statistic and the rejection region

Example A researcher believes that the duration of phone talks follows an exponential distribution with a parameter λ. On the basis of a single call, she would like to test whether λ 2. She is ready to reject the null hypothesis, whenever the duration is above c. 1. Identify the statistical model Sample space = (0, ) 20 ; Parameter space = λ(0, ); P λ = exp 2. Write down H 0 and H 1 H 0 : λ 2 H 1 : λ > 2 3. Identify the test statistic and the rejection region

Example A researcher believes that the duration of phone talks follows an exponential distribution with a parameter λ. On the basis of a single call, she would like to test whether λ 2. She is ready to reject the null hypothesis, whenever the duration is above c. 1. Identify the statistical model Sample space = (0, ) 20 ; Parameter space = λ(0, ); P λ = exp 2. Write down H 0 and H 1 H 0 : λ 2 H 1 : λ > 2 3. Identify the test statistic and the rejection region W (X) = X and rejection region w = {(x) : X c}

Error types I & II Test H 0 H 1 Population H 0 H 1

Error types I & II Test H 0 H 1 Population H 0 H 1 Perfect Perfect

Error types I & II Test Population H 0 H 1 H 0 Perfect H 1 Error type I Perfect

Error types I & II Test Population H 0 H 1 H 0 Perfect Error type II H 1 Error type I Perfect

Error types I & II Test Population H 0 H 1 H 0 Perfect Error type II H 1 Error type I Perfect Since W (X) is a random variable, errors as probabilities Type I : P(W (X) w H 0 ) = α(w) Type II : P(W (X) / w H 1 ) = β(w) * In Casella and Berger, β(w) is the power of the test

Power of a test Definition is the probability that a test rejects H 0 when H 0 is false. Power(W ) = P(W w H 1 ) It follows that Power(W ) = 1 P(W / w H1) Power(W ) = 1 β(w)

Graphical interpretation 1. Identify the probability of error of type I, type II and the power of the test. 2. How do these values vary with H1 and n?

Graphical interpretation 1. Identify the probability of error of type I, type II and the power of the test. 2. How do these values vary with H1 and n?.4.3 ƒ(x).2.1 0 4 2 0 2 4 6 Values Distribution of T(X): H0:µ=0 H1:µ=2 Rejection area: T(X) > 1.96

Graphical interpretation 1. Identify the probability of error of type I, type II and the power of the test. 2. How do these values vary with H1 and n?.4.4.3.3 ƒ(x).2 ƒ(x).2.1.1 0 0 4 2 0 2 4 6 Values 5 0 5 10 Values Distribution of T(X): H0:µ=0 H1:µ=2 Distribution of T(X): H0:µ=0 H1:µ=5 Rejection area: T(X) > 1.96 Rejection area: T(X) > 1.96

Graphical interpretation 1. Identify the probability of error of type I, type II and the power of the test. 2. How do these values vary with H1 and n?.4 1.5.3 1 ƒ(x).2 ƒ(x).1.5 0 4 2 0 2 4 6 Values Distribution of T(X): H0:µ=0 H1:µ=2 Rejection area: T(X) > 1.96 0 1 0 1 2 3 Values of H0:µ=0 Distribution T(X): H1:µ=2 Rejection area: T(x) > 1.96

Significance level and p-value Significance level equals α if θ Θ 0 we have P(W (X) w) α

Significance level and p-value Significance level equals α if θ Θ 0 we have P(W (X) w) α Size level equals α if θ Θ 0 we have P(W (X) w) = α

Significance level and p-value Significance level equals α if θ Θ 0 we have P(W (X) w) α Size level equals α if θ Θ 0 we have P(W (X) w) = α P-value: probability of type I error when obtained statistic is used as the critical value P(W (X) > W H 0 )

Bias & consistency To test the a hypothesis H 0 : θ = θ 0 against the alternative H 1 : θ θ 0, we construct a critical region w. The test is unbiased if: P(W w θ = θ 0 ) < P(W w θ = θ 1 ) θ 1 θ 0 θ 1 H 1 The test is consistent if lim P(W w H 1) = 1 n

Example II A researcher would like to test whether average expenditure on food on a population equals one half of the minimum wage. This value is given by the constant m 0. The alternative hypothesis is that the value is different from m 0. Assuming that the expenditure follows a normal distribution with a standard deviation of 1. The researcher is willing to reject the null hypothesis whenever ( x m 0 ) (n) > 1.96.

Most powerful test Most powerful test : For given α(w), the most powerful test maximizes 1 β(w) Imagine 2 tests such that P(W 1 θ 0 ) = P(W 2 θ 0 ) = α W 1 is the most powerful test if for any W 2, P(W 1 θ 1 ) > P(W 2 θ 1 ) The most powerful test might not exist

The Lemma States that a likelihood ratio test is the most powerful test of H 0 : θ = θ 0 against H 1 : θ = θ 1 L(x; θ 0 ) k in critical region w and L(x; θ 1 ) L(x; θ 0 ) k outside of critical region w L(x; θ 1 ) If the test is the most powerful for all H 1 Uniformly most powerful test

Example III X is a single observation from a distribution with a density function f (x) = θx θ 1, for 0 < x < 1. We want to test H 0 : θ = 3 for a significance level of 0.05. What is the most powerful test against H 1 : θ = 2? What is the most powerful test against H 1 : θ = 1.5? and against H 1 : θ = 1.5

Example IV We extract a random sample consisting of n elements from a normal distribution N(µ, σ). We want to test H 0 : µ = m 0 against H 1 : µ = m 1, where m 1 < m 0. Assuming that σ is known, find the most poweful test with a probability of error type I = α. Hint: Normal distribution density function: f (x; µ, σ) = 1 σ 2 2π e (x µ) /2σ 2

Definition LR test is any test with a rejection region (w) of the form w{x 1,..., x n : λ(x) c}, for 0 c 1, and λ(x) = sup Θ 0 L(θ x) sup Θ L(θ x) where θ in the denominator can be replaced by ˆθ MLE and c is chosen based on the error of type I

Log-likelihood test Computing P(λ(x) < c H 0 ) = α might be hard when n. Alternative: Log-likelihood test ( D = 2 ln(λ(x)) = 2 ln ) L(θ 0 x) L(ˆθ MLE x) D χ 2 q where q is the number of restricted parameters

Sampling distributions Review Population Pop. SD. Sample size Statistic distribution Normal Known any x N(µ, σ/ n) Any Known large Binomial large ˆp N(p, p(1 p)/n) Normal Unknown small ((x µ) n/s) t n 1 Normal Unknown large x N(µ, S/ n) Notes: results from the Central limit theorem and the Laplace and DeMoivre theorem.

Thank you for your attention!