Lecture 2 Multiple Regression and Tests

Transcription

1 Lecture 2 and s Dr.ssa Rossella Iraci Capuccinello

2 Simple Regression Model The random variable of interest, y, depends on a single factor, x 1i, and this is an exogenous variable. The true but unknown relationship is defined as being y i = β 0 + β 1 x 1i + u i The values of y are expected to lie on a straight line, depending on the corresponding values of x Their values will differ from those predicted by that line by the amount of the error term u i

3 Simple Regression Model Fig1 Source: Chap. 2 Woolwridge

4 Simple Regression Model Fig2 Source: Chap. 2 Woolwridge

5 Simple Regression Model Fig3 Source: Chap. 2 Woolwridge

6 Simple Regression Model Fig4 - Homoskedasticity Source: Chap. 2 Woolwridge The errors are considered drawn from a fixed distribution, with a mean of zero and a constant variance of σ 2

7 Simple Regression Model Fig5 - Heteroskedasticity Source: Chap. 2 Woolwridge

8 CLRM assumptions Running The random variable of interest, y, depends upon a number of different factors, x 1i, x 2i,..., x ki, and these are exogenous variables. The true but unknown relationship is defined as being y i = β 0 + β 1 x 1i + β 2 x 2i... x ki + u i i = 1,..., n

9 CLRM assumptions Running CLRM assumptions the Classical Linear Regression Model (CLRM) assumptions are: 1 x ij j = 1,... k are non stochastic 2 E(u i x 1, x 2,..., x k ) = 0 (Exogeneity regressors are uncorrelated with the errors) 3 Var(u i x 1, x 2,..., x k ) = σ 2 (error variance constant (homoscedasticity), points distributed around true regression line with a constant spread) 4 cov(u i, u j x 1, x 2,..., x k ) = 0 (errors serially uncorrelated over observations) 5 (u i x 1, x 2,..., x k ) N(0, σ 2 ) Normality

10 CLRM assumptions Running Running Simple regressions are easy: Type reg followed by 1 Dependent variable : y 2 Independent variables : x 1, x 2,..., x k

11 Simplest specification Scaled dependent variable Standardized regressors Log forms Simplest specification y =β 0 + β 1 x 1 + β 2 x 2 + u y x 1 = β 1 change in y for a unit increase in x 1

12 Simplest specification Scaled dependent variable Standardized regressors Log forms Scaled dependent variable y/α =β 0 + β 1 x 1 + β 2 x 2 + u y = β 1 x 1 α y = β 2 x 2 α Interpretation coefficient: each new coefficient and s.e. will be the corresponding old coefficient and s.e. multiplied by the scalar α t statistics are identical.

13 Simplest specification Scaled dependent variable Standardized regressors Log forms Standardized regressors where zy =δ 0 + δ 1 zx 1 + δ 2 zx 2 + ɛ zy = δ 1 = σ 1 β 1 zx 1 σ y zy = y y σ y zx = x x j σ j if x 1 increases by 1 s.d. then y changes by δ 1 standard deviations. to generate a sdtzed variable: egen zvarname=std(varname) Interpretation: 1 s.d. increase in x 1 decreases y by δ 1 s.d.

14 Simplest specification Scaled dependent variable Standardized regressors Log forms Log forms ln(y) = α + β 1 ln(x 1 ) + β 2 x 2 + β 3 (1/x 3 ) + β 4 x 3 + β 5 x 4 + β 6 x u ln(y) ln(x 1 ) = β 1 % change in y for a 1% increase in x 1, elasticity of y w.r.t. x 1 ln(y) x 2 = β 2 change in ln(y) for a unit increase in x 2 ; when β 2 multiplied by 100, this is the percentage change in y (also called semi elasticity of y w.r.t. x 2 )

15 Simplest specification Scaled dependent variable Standardized regressors Log forms Examples reg logearn age ages yearsed deg all 1 ln(y) = logearn and x 2 = s years of education. Suppose β 2 = says that each year of education increases wages by a constant percentage, 5.4%. % wage (100 β 2 ) x 2 2 The coefficient of deg all (0.5613) says that having a degree or a higher qualification increases wages by 56.13% relative to those individuals with lower or no qualifications, holding other factors fixed.

16 Simplest specification Scaled dependent variable Standardized regressors Log forms Log and quadratics forms ln(y) =α + β 1 ln(x 1 ) + β 2 x 2 + β 3 x 3 + β 4 (1/x 3 ) + β 5 x 4 + β 6 x u ln(y) x 3 = β 3 β 4 x 2 3 proportionate change in y for a unit increase in x 3 ln(y) x 4 = β 5 + 2β 6 x 4 proportionate change in y for a unit increase in x 4 if β 5 > 0 and β 6 < 0 = quadratic relationship between x and y, diminishing effects of x on y. e.g. tot effect at age 35 = = tot effect at age 60 = =

17 t-test t-test in Stata p-value t-test - other commands t-test H o :β 1 = a 1 β age = 0 H 1 :β 1 a 1 β age 0 t = β 1 a 1 s.e.( β 1 ) t α/2,dof =n k 1 = t 0.025, Recalling that the degrees of freedom (dof) are the difference: number of observations minus number of estimated parameters Rejection rule: if t > t c = > 1.96 = reject H o

18 t-test t-test in Stata p-value t-test - other commands t-test in Stata The t-stat appears in the regression output. You can perform the test manually test age=0 but it shows an knowing that t 2 n k 1 = F 1,n k 1 the results are identical F (1, 13701) = di sqrt( ) = di invttail(13702, 0.025) = 1.96

19 t-test t-test in Stata p-value t-test - other commands p-value Stata shows also the p-value: the largest significance level at which the null hypothesis would not be rejected, given the observed t. Generally, one rejects the null hypothesis if the p-value is smaller than or equal to the significance level. P(T > t observed H o ) = p P( T > t H o ) = 2 P(T > t ) = p in Stata ttail(n, t) di 2 ttail(13724, 41.97) = 0.000

20 t-test t-test in Stata p-value t-test - other commands t-test - other commands testparm dresid2 dresid3, equal test whether the coeff are equal test age=5 testnl To test non-linear constraints

21 Confidence Interval Goodness of fit Confidence Interval From β i β i se( β i ) t α/2,n k 1 simple manipulations leads to (1 α)% CI for unknown β i : β i ± t c α/2,n k 1 se( β i ) where t c α/2,n k 1 is (1 α/2)th percentile in t α/2,n k 1 distribution. In our example, 95% CI for deg all: β i = = β i = = It is a good practice, when running models to check the CI for the same parameter estimated.

22 Confidence Interval Goodness of fit R 2 Defining SST = n i=1 (y i y) 2 total sum of squares SSE = n i=1 (ŷ i y) 2 explained sum of squares RSS = n i=1 ûi 2 residual sum of squares Knowing that SST = SSE + RSS The R 2 is defined to be R 2 = SSE SST = 1 RSS SST

23 Confidence Interval Goodness of fit R 2 interpretation It is the proportion of the sample variation in y i explained by the OLS line. It never decreases and increases when an additional regressor is added to a regression. In our example, R 2 = means that all the independent variables together explain about 21.71% of the variation of log wages for our sample of workers.

24 of multiple restrictions Unrestricted and Restricted models Perform the Rejection Rule Example in Stata of multiple restrictions H o :β 1 = β 0 1, β 2 = β β q = β 0 q H 1 :β j β 0 j, j = 1,..., q The null constitutes q restrictions = multiple (or joint) hypothesis test.

25 of multiple restrictions Unrestricted and Restricted models Perform the Rejection Rule Example in Stata Unrestricted and Restricted models The unrestricted model has k independent variables + the intercept y = β 0 + β 1 x β k x k + u (1) Suppose that the restriction is that q of the k variables (for simplicity the last q) have zero coefficients, then H o : β k q+1 = 0,..., β k = 0 and imposing these restriction in (1) we obtain the restricted model y = β 0 + β 1 x β k q x k q + u (2)

26 of multiple restrictions Unrestricted and Restricted models Perform the Rejection Rule Example in Stata Perform the 1 Run regression 1 and get RSS ur 2 Run regression 2 and get RSS r 3 Compute the F statistic F = (RSS r RSS ur )/q RSS ur /dof F α q,dof q = numerator degrees of freedom = dof r dof ur = (n (k q + 1)) (n (k + 1)) = number of restrictions under H o, i.e. the number of equality signs in H o dof= denominator degrees of freedom = dof ur = n k 1

27 of multiple restrictions Unrestricted and Restricted models Perform the Rejection Rule Example in Stata Rejection Rule If F > F c = reject H 0 then x k q+1,..., x k are jointly statistical significant. If H o is not rejected, then the variables are jointly insignificant. In this context the p-value is defined as p = P(F > F ) where F is an F random variable with (q, n k 1) degrees of freedom, and F is the actual value of the test statistic. A small p-value is evidence against H o.

28 of multiple restrictions Unrestricted and Restricted models Perform the Rejection Rule Example in Stata in Stata H o : nonwhite = female = married = numdep = 0 = q = 4 use wage1.dta Unrestricted model reg lwage educ exper tenure nonwhite female married numdep = k = 7 test nonwhite female married numdep H o : female =-0.3, married =0.15, numdep = 0 = q = 3 testnl ( b[female] = 0.3) ( b[married] = 0.15) ( b[numdep] = 0)

29 of multiple restrictions Unrestricted and Restricted models Perform the Rejection Rule Example in Stata Example - manually H o : nonwhite = female = married = numdep = 0 = q = 4 Unrestricted model reg lwage educ exper tenure nonwhite female married numdep = k = 7 take note of RSS UR Restricted model reg lwage educ exper tenure = k q = 3 take note of RSS R compute F = (RSSr RSSur )/q RSS ur /(n k 1) F α q,(n k 1) find in the Tables F critical value or use Stata command di invf (q, n k 1, α)

30 of overall significance Saving typing Example overall of overall significance H o :β 1 = β 2 =... = β q = 0 UR : y = β 0 + β 1 x β k x k + u R : y = β 0 + u F = (RSS r RSS ur )/k RSS ur /(n k 1) F α k,(n k 1) Rur 2 /k F = (1 Rur 2 )/(n k 1) H 1 : Any β j 0 j = 1,..., q The F statistic with the R 2 is valid only for testing joint exclusion of all regressors. or

31 of overall significance Saving typing Example overall Saving typing - macro For defining lists of vars ( globally). global 1 type global followed by the groupname followed by the vars that you want to group together 2 in a regression type reg depvar followed by $groupname Pay careful attention to the sign $ in the global.

32 of overall significance Saving typing Example overall Example - overall significance The F test of overall significance is reported automatically in Stata output. You can also perform it either computing the F statistic or by using that Stata command test. For example, using a macro global indvars educ exper tenure nonwhite female married numdep reg lwage $indvars test $indvars