Some Review and Hypothesis Tes4ng. Friday, March 15, 13

Size: px

Start display at page:

Download "Some Review and Hypothesis Tes4ng. Friday, March 15, 13"

Amber Haynes
6 years ago
Views:

1 Some Review and Hypothesis Tes4ng

2 Outline Discussing the homework ques4ons from Joey and Phoebe Review of Sta4s4cal Inference Proper4es of OLS under the normality assump4on Confidence Intervals, T test, p value your thoughts on the course so far

3 The Homework Thinking about missing values cap drop vs. drop why aren t the coefficients and standard errors the same when I switch to a regression using z- scores? But shouldn t the t- stats be the same? What was the popula4on when you set to z- scores?

4 The Homework Does the DV and/or the IVs need to be normally distributed for OLS to produce an unbiased result? Short answer: No and No Long answer: It helps.

5 Linear Regression Regression is based on the concept of a simple propor4onal rela4onship We want to es4mate the PRF Y i =B 1 +B 2 X i +u i On the basis of the SRF

6 Method of OLS Ordinary Least Squares (OLS) is a method of finding the linear model which minimizes the sum of the squared errors.

7 More Review To find the minimum of a func4on, we can use calculus We minimize the sum of the squared errors for OLS This gives us (and gives so]ware) equa4ons to es4mate B 1 and B 2.

8 Today We know how to es4mate our parameters. What are the proper4es of the OLS Es4mators? How precise are the es4mates? How close are the es4mates to the true rela4onship in the popula4on? How can we use that to test hypotheses

9 Proper4es of OLS Es4mators Recall the Es4mators Property 1: They are Linear Es4mators

10 More Proper4es Property 2: They are Unbiased

11 Property 3: Efficiency OLS produces efficient es4mates An efficient es4mator is one with the least variance compared to any other es4mator of the parameter

12 Gauss- Markov Theorem Given the assump4ons of the classical linear regression model, OLS, in the class of unbiased linear es4mators is BLUE (Best Linear Unbiased Es4mator), where best is minimum variance Property 4: OLS es4mates are consistent: As sample size, N, grows the OLS es4mators converge to the true popula4on value

13 Next Topic We can es4mate our unknown parameters We know that the OLS es4mators are the Best Linear Unbiased Es4mators But, how precise are the es4mators in a given sample?

14 How precise are our es4mates? Standard error is a measure of the precision of our es4mator Variance of our es4mate of beta_hat is directly propor4onal to σ 2 (i.e. the variance of the error term) Variance of both es4mates is inversely propor4onal to Σ(X i - Xbar) (i.e. the variance of x) As N increases, the variability of our es4mates will go down

15 Sta4s4cal Inference A methodology for learning about popula4ons and their probability distribu4ons by sampling from them

16 Review: The Normal Distribu4on For many random variables, the probability distribu4on is a bell curve, or a normal curve Use normal distribu4on to calculate probability beyond a given value of z

17 Review: The Sample Mean How good of an es4mate of the popula4on mean is the sample mean? The central limit theorem gives an answer

18 Central Limit Theorem Let X 1, X 2,, X n denote n independent random variables, all of which have the same pdf with mean, u, and variance, σ 2. The sample mean is denoted as: As n increases indefinitely, English- The sample mean approaches the normal distribu4on with a mean of u, and a variance σ 2 /n.

19 Proper4es of the Sample Mean A linear func4on of the Random Variable X i It is a normal Random Variable The expected value is the popula4on mean The standard devia4on decreases as sample size increases

20 Standard Normal Distribu4on A special type of normal distribu4on, with mean 0 and standard devia4on 1 Z Formula: Can use to convert normal distribu4on to standard normal:

21 Review of Hypotheses A hypothesis is a statement about some characteris4c of a variable or a collec4on of variables. The null hypothesis H 0 is the hypothesis that is directly tested. It states that the parameter tested has no effect. The alterna4ve hypothesis H A is a hypothesis that contradicts the null hypothesis. This is usually a statement that the researcher believes to be true. Proof by contradic4on: reject the null hypothesis and support the alterna4ve hypothesis.

22 Significance Tests A significance test is a way of sta4s4cally tes4ng a hypothesis by comparing the data to values predicted by the hypothesis.

23 Review of Type I and Type II Error

24 Hypothesis Tes4ng Recall that there are two types of hypothesis tests, significance tests and confidence intervals, but both will yield the same results. To do hypothesis tests, we must first transform a normal RV to a standard normal RV:

25 Example: Is America Moderate? Responses (n=627) Count 1 Extremely liberal 12 2 Liberal 66 3 Slightly liberal Moderate, middle of road Slightly conserva4ve Conserva4ve 74 7 Extremely conserva4ve 11

26 Using the Test Sta4s4c H 0 : µ = 4.0 (middle of road) H A : µ 4.0 Test Sta4s4c P- value Pr[(Z>0.64) or (Z< )]= =0.52 Do not reject the null hypothesis

27 Using the Confidence Interval Or we can use the confidence interval ± 1.96 SE ± 1.96(.05) ± (3.93, 4.13) 4.0 is contained within the 95% confidence interval, so we can not reject the null hypothesis that America is moderate

28 Types of Tests Two- tailed tests Use both the right and le] tail H 0 : µ = 4.0 (middle of road) H A : µ 4.0 One- tailed tests Only use probability in one tail H 0 : µ <= 4.0 (middle of road) H A : µ > 4.0

29 Review If the sampling distribu4on of a RV is normal, then we can make inferences about how close our sample is to the true popula4on. To be able to use test sta4s4cs with OLS, we need the sampling distribu4on of B 2 to be normal.

30 Classical Linear Regression Model Since B 1, B 2 and σ 2, are random variables, we need to know if their probability distribu4ons are normal

31 Classical Normal Linear Regression Model The CLRM does not make assump4ons about the probability distribu4on of u To see how close our es4mated es4mated B 2 is to the true B 2, we need to know the probability distribu4on of our es4mate which depends on the probability distribu4on of u. The Classical Normal Linear Regression Model solves this problem

32 Normality Assump4on of U i under CNLRM The CNLRM assumes that each u i is distributed normally with Mean E(u i )=0 Variance E[u i - E(u i )] 2 = E(u i2 )=σ 2 Covariance (u i, u j ) E{[u i - E(u i )][u j - E(u j )]} =E(u i, u j ) =0, i j Or, u i ~N(0, σ 2 ) or u i ~NID(0, σ 2 ) u i is normally and independently distributed with mean zero and variance sigma squared

33 Why do we make the normality assump4on? Recall that the error term is a random variable From central limit theorem, know distribu4on of the sampling mean of a random variable tends to be normal in the limit. Since B 1 and B 2 are linear func4ons of u i, they are normally distributed and we can use sta4s4cal tests such as t test.

34 Review of Proper4es of OLS Es4mators Recall our OLS es4mators Earlier, we showed that they are: Linear Unbiased Efficient They are BLUE

35 Now we added another property B 1 and B 2 are normally distributed This will allow us to do hypothesis tests

36 So what are t- sta4s4cs? B 1 _hat and B 2 _hat are normally distributed So is But the ra4o of these two is NOT (quite) normal it follows a t distribu4on, which is a lot like a normal distribu4on, but not quite. where k is the # of parameters to be es4mated

37 t- sta4s4cs (cont) Note the addition of a degrees of freedom constraint Thus the more data points we have relative to the number of parameters we are trying to estimate, the more the t distribution looks like the z distribution. When n>100 the difference is negligible

38 Sta4s4cal vs. substan4ve significance If n is large, our es4mated may be sta4s4cally significant bust substan4vely trivial If n is small (or the standard errors are large for other reasons), we may know the effect is not zero but have NO IDEA how big it is.

39 How to Present Results? This could be half a class. But lets talk a bit.

Linear Regression and Correla/on. Correla/on and Regression Analysis. Three Ques/ons 9/14/14. Chapter 13. Dr. Richard Jerz

Linear Regression and Correla/on Chapter 13 Dr. Richard Jerz 1 Correla/on and Regression Analysis Correla/on Analysis is the study of the rela/onship between variables. It is also defined as group of techniques