Assumptions of classical multiple regression model

Transcription

1 ESD: Recitation #7

2 Assumptions of classical multiple regression model Linearity Full rank Exogeneity of independent variables Homoscedasticity and non autocorrellation Exogenously generated data Normal distribution

3 Hypothesis Testing Method: 1. Formulate the null hypothesis H 0 and the alternative hypothesis H A. 2. Identify a test statistic to assess the truth of the null hypothesis. 3. Compute the P-value, which is the probability that a test statistic as least as significant as the one observed would be obtained, assuming that the null hypothesis were true. 4. Compare the P-value to an acceptable significance level, α. If p α, the null hypothesis is ruled out.

4 α levels and P-values Examples of confidence levels: Proportions: Means: Differences between two population proportions (large samples): Differences between two population means (large samples):

5 One tailed or two tailed tests One tailed (right) One tailed (left) Two tailed H 0: μ μ o H 0: μ μ o H 0: μ = μ o H A: μ > μ o H A: μ < μ o H A: μ = μ o μ o is a specified value of the population mean

6 Z-test One-tailed (right) One-tailed (left) Two-tailed H 0: H A: μ 1 μ 2 μ 1 > μ 2 μ 1 μ 2 μ 1 < μ 2 μ 1 = μ 2 μ 1 μ 2 Test statistic z z z Decision Rule Reject if z >z α Reject if z < -z α Reject if z >z α/2 or z < - z α/2

7 Z-test (2) Large sample proportion: Large sample mean: p = p o,z = μ = μ o,z = x μ o s/ n Large sample difference between two population proportion: p 1 = p 2,or, p 1 p 2 = 0,z = Large sample difference between two population means: x 1 x 2 μ 1 = μ 2, z = 2 2 s 1 n 1 + s 2 n 2 x np o np o (1 p o ) ˆ p 1 ˆ p 2 p ˆ (1 p ˆ )( ) n 1 n 2

8 T-test Same method Tests: Small sample mean: μ = μ o,t = x μ o s/ n Small sample difference: (S p is the pooled standard deviation) μ 1 = μ 2,t = x 1 x 2 s p 1 n n 2

9 Example: Medical treatment Is your treatment more successful than control? Test at the 1% significance level What are we comparing? Small or large sample? One or two tailed? Treatment Control Sample mean (%): µ Sample SD: s 3 2 N 75 60

10 Medical treatment (2) Comparing means, z test statistics, one tailed (right tailed) H 0: μ t μ c H A: μ t > μ c Test Statistic: z-test z = μ t μ c Decision Rule: For significance level α = 0.01, reject null hypothesis if computed test statistic value: z= > z α =2.33, p= (from z-table) s t 2 2 n t + s c n c Conclude: reject null hypothesis.

11 ANOVA ANOVA allows for comparing points estimates for more than 2 groups. ANOVA separates the total variability of outcome in two categories: variability within or between groups. H 0 : same average for each group H A : all averages are not the same

12 ANOVA: Method 1) Calculate variation between groups. - Compare mean of each group with mean of overall sample: between sum of squares (BSS): m i=1 BSS = n i (x i x) 2 - Divide BSS by number of degree of freedom (m-1) - Get estimate of variation between groups

13 ANOVA: Method (2) 2) Calculate variation within groups. - Find sum of squared deviation between individual results and the group average, calculating separate measures for each group - Sum the group values: obtain the within sum of squares (WSS): m i=1 WSS = (n i 1) σ i 2 m - Divide WSS by number of degrees of freedom ( ): i=1 get estimate of variation within groups. n i 1

14 ANOVA: Method (3) 3) Calculate F-statistics. F = BSS/WSS Compare value with standard table for the F distribution: calculate significance level of F value If null rejected: use z- and t-tests between each pair of groups.

15 Example: shift productivity H o: μ morning = μ afternoon = μ night The null indicates that all groups have the same average score and by assumption the same standard deviation H A: μ morning μ afternoon μ night The alternative is that all means are not the same Note: the alternative is not that all means are different. It is possible that some of the means could be the same, yet if they are not all the same, we would reject the null.

16 Shift productivity (2) H o: µ morning = µ afternoon = µ night score morning afternoon night H A: µ morning µ afternoon µ night score morning afternoon night

17 Shift productivity (3) Av. Prod. SD # of workers Morning Afternoon Night Average BSS = n 1 (x 1 -x) 2 + n 2 (x 2 -x) 2 + n 3 (x 3 -x) 2 = 313 ( ) ( ) ( ) 2 = Between Mean Squares = BSS/v = 23.36/2 = 11.69

18 Shift productivity (4) 2. WSS = (n 1-1) SD 12 + (n 2-1) SD 22 + (η 3-1) SD 2 3 = (313-1) (340-1) (297-1) = Within Mean Squares = WSS/v = /947 = 1.69

19 Shift productivity (5) 3) F = (Between Mean Squares/Within Mean Squares) = (11.68/1.69) = 6.91 Compare value to standard table for the F distribution In this case, significance level is less than.01 Reject the null, students performance varies significantly across the three classes

20 Testing for heteroscedasticity White s test Goldfeld-Quandt test Lagrange Multiplier test If E[εε Ω]=σ 2.Ω is known: Weighted least squares

21 Maximum Likelihood Estimation Definition: PDF of a random variable y, conditioned on a set of parameters : Joint density of n independent and identically ditributed observations from this process: f (y 1,...,y n θ) = f (y i θ) = L( θ y) The joint density is the likelihood function. i=1 Maximize with respect to θ: L(θ y) θ θ f (y θ) L(θ y) n = 0 and 2 L(θ y) θ 2 < 0

22 Maximum Likelihood Estimation (2) Conditions: Parameter vector identified θ θ* θ,l(θ* y) L(θ y) Properties: Asymptotically efficient