Regression with Qualitative Information. Part VI. Regression with Qualitative Information

Transcription

1 Part VI Regression with Qualitative Information As of Oct 17, 2017

2 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving Dummy Variables Interaction among Dummy Variables Different Slopes Chow Test Binary Dependent Variable The Linear Probability Model The Logit and Probit Model

3 Qualitative information: family owns a car or not, a person smokes or not, firm is in bankruptcy or not, industry of a firm, gender of a person, etc Some of the above examples can be both in a role of background (independent) variable or dependent variable. Technically these kinds of variables are coded by a binary values (1 = yes ) (0 = no ). In Econometrics these variables are called generally called dummy variables. Remark 6.1 Usual practice is to denote the dummy variable by the name of one of the categories. For example, instead of using gender one can define the variable e.g. as female, which equals 1 if the gender is female and 0 if male.

4 Exploring Further (W 5ed 7.1, p. 216) Suppose, in a study comparing election outcomes between Democratic and Republican candidates, the researcher wishes to indicate the party of each candidate. Is a name such as party a wise choice for a binary variable in this case. What would be a better name?

5 Single Dummy Independent Variable 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving Dummy Variables Interaction among Dummy Variables Different Slopes Chow Test Binary Dependent Variable The Linear Probability Model The Logit and Probit Model

6 Single Dummy Independent Variable Dummy variables can be incorporated into a regression model as any other variables. Consider the simple regression y = β 0 + δ 0 D + β 1 x + u, (1) where D = 1 if individual has the property and D = 0 otherwise, and E[u D, x] = 0. Parameter δ 0 indicated the difference with respect to the reference group (D = 0), for which the parameter is β 0. Then δ 0 = E[y D = 1, x] E[y D = 0, x]. (2) The value of x is same in both expectations, thus the difference is only due to the property D.

7 Single Dummy Independent Variable y β 0 + δ 0 β 0 E[y D = 1, x] = β 0 + δ 0 + β 1 x E[y D = 0, x] = β 0 + β 1 x x Figure 6.1: E[y D, x] = β 0 + δ 0 D + x + u, δ 0 > 0. The category with D = 0 makes the reference category, and δ 0 indicates the change in the intercept with respect to the reference group.

8 Single Dummy Independent Variable From interpretation point of view it may also be beneficial to associate the categories directly to the regression coefficients. Consider the wage example, where log(w) = β 0 + β 1 educ + β 2 exper + β 3 tenure + u. (3) Suppose we are interested about the difference in wage levels between men an women. Then we can model β 0 as a function of gender as β 0 = β m + δ f female, (4) where subscripts m and f refer to male and female, respectively. Model (3) can be written as log(w) = β m + δ f female + β 1 educ +β 2 exper + β 3 tenure + u. (5)

9 Single Dummy Independent Variable In the model the female dummy is zero for men. All other factors remain the same. Thus the expected difference between wages in terms of the model is according to (2) equal to δ f Because the left hand side is in logs, 100 δ f indicates the difference approximately in percentages.

10 Single Dummy Independent Variable The log approximation may be inaccurate if the percentage difference is big. A more accurate approximation is obtained by using the fact that log(w f ) log(w m ) = log(w f /w m ), where w f and w m refer to a woman s and a man s wage, respectively, thus ( ) wf w m 100 % = 100 (exp(δ f ) 1) %. (6) w m

11 Single Dummy Independent Variable Example 1 Augment the wage example with squared exper and squared tenure to account for possible reducing incremental effect of experience and tenure and account for the possible wage difference with the female dummy. Thus the model is log(w) = β m + δ f female + β 1 educ +β 2 exper + β 3 tenure (7) +β 4 (exper) 2 + β 5 (tenure) 2 + u.

12 Single Dummy Independent Variable Example 1 continues Dependent Variable: LOG(WAGE) Included observations: 526 ======================================================= Variable Coefficient Std. Error t-statistic Prob C FEMALE EDUC EXPER TENURE EXPER^ TENURE^ ====================================================== R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid Schwarz criterion F-statistic Prob(F-statistic) ===========================================================

13 Single Dummy Independent Variable Example 1 continues Using (6), with ˆδ f = ŵf ŵ m ŵ m = 100[exp(ˆδ f ) 1] 25.7%, (8) which suggests that, given the other factors, women s wages (w f ) are on average 25.7 percent lower than men s wages (w m ). It is notable that exper and tenure squared have statistically significant negative coefficient estimates, which supports the idea of diminishing marginal increase due to these factors.

14 Multiple Categories 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving Dummy Variables Interaction among Dummy Variables Different Slopes Chow Test Binary Dependent Variable The Linear Probability Model The Logit and Probit Model

15 Multiple Categories Additional dummy variables can be included to the regression model as well. In the wage example if married (married = 1, if married, and 0, otherwise) is included we have the following possibilities female married characteization 1 0 single woman 1 1 married woman 0 1 married man 0 0 single man and the intercept parameter refines to β 0 = β sm + δ f female + δ ma marr. (9) Coefficient δ ma is the wage marriage premium.

16 Multiple Categories Example 2 Including married dummy into the wage model yields Dependent Variable: LOG(WAGE) Method: Least Squares Sample: Included observations: 526 ====================================================== Variable Coefficient Std. Error t-statistic Prob C FEMALE MARRIED EDUC EXPER TENURE EXPER^ TENURE^ ====================================================== R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info crit Sum squared resid Schwarz criterion Log likelihood F-statistic Durbin-Watson stat Prob(F-statistic) ======================================================

17 Multiple Categories Example 2 continues The estimate of the marriage premium is about 5.3%, but it is not statistically significant. A major limitation of this model is that it assumes that the marriage premium is the same for men and women. We relax this next.

18 Multiple Categories Generating dummies: singfem, marrfem, and marrmale we can investigate the marriage premiums for men and women. The intercept term becomes β 0 = β sm + δ mm marrmale +δ mf marrfem + δ sf singfem. (10) The needed dummy-variables can be generated as cross-products form the female and married dummies. For example, the singfem dummy is singfem = (1 married) female. So, how do you generate marrmale (married man)?

19 Multiple Categories Example 3 Estimating the model with the intercept modeled as (10) gives Dependent Variable: LOG(WAGE) Method: Least Squares Sample: Included observations: 526 ====================================================== Variable Coefficient Std. Error t-statistic Prob C MARRMALE MARRFEM SINGFEM EDUC EXPER TENURE EXPER^ TENURE^ ====================================================== R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info crit Sum squared resid Schwarz criterion Log likelihood F-statistic Durbin-Watson stat Prob(F-statistic) ======================================================

20 Multiple Categories Example 3 continues The reference group is single men. Single women and men estimated wage difference: -11.0%, just borderline statistically significant in the two sided test. Marriage premium for men: 21.3% (more accurately, using (7), 23.7%). Married women are estimated to earn 19.8% less than single men. Marriage premium for women: or 8.8%. ˆδ mf ˆδ sf ( 0.110) = The statistical significance of this can be tested either by redefining the dummies such that the single women become the base.

21 Multiple Categories Example 3 continues Another option is to use advanced econometric software. EViews Wald test produces for H 0 : δ mf = δ sf (11) F = (df 1 and 517) with p-value , which is not statistically significant and there is not empirical evidence for wage difference between married and single women. In the same manner testing for H 0 : δ mm = δ mf produces F = with p-value , i.e., highly statistically significant. Thus, there is strong empirical evidence of marriage premium for men.

22 Multiple Categories Remark 6.1: If there are q then q 1 dummy variables are needed. The category which does not have a dummy variable becomes the base category or benchmark. Remark 6.2: Dummy variable trap. If the model includes the intercept term, defining q dummies for q categories leads to an exact linear dependence, because 1 = D D q. Note also that D 2 = D, which again leads to an exact linear dependency if a dummy squared is added to the model. All these cases which lead to the exact linear dependency with dummy-variables are called the dummy variable trap.

23 Multiple Categories Exploring Further (W 5ed 7.2, p. 227) In the baseball salary data MLB1 (available at players are given one of six positions: frstbase, scndbase, thrdbase, shrtsop, outfield, or catcher. To allow for salary differentials accross position, with outfielders as the base group, which dummy variables would you include as independent variables?

24 Multiple Categories 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving Dummy Variables Interaction among Dummy Variables Different Slopes Chow Test Binary Dependent Variable The Linear Probability Model The Logit and Probit Model

25 Multiple Categories If the categories include ordinal information (e.g. 1 = good, 2 = better, 3 = best ), sometimes people these variables as such in regressions. However, interpretation may be a problem, because one unit change implies a constant partial effect. That is the difference between better and good is as big as best and better.

26 Multiple Categories The usual alternative to use dummy-variables. In the above example two dummies are needed. D 1 = 1 is better, and 0 otherwise, D 2 = 1 for best and 0 otherwise. As a consequence, the reference group is good.

27 Multiple Categories Exploring Further (Adapted from W 5ed 7.3, p228) Consider the effect of credit rating (CR) on the municipal bond rate (MBR). Credit ratings are indicated by ranking from zero to four, with zero the worst and four being the best. Using using model MBR = β 0 + δ 1 CR 1 + δ 2 CR 2 + δ 3 CR 3 + δ 4 CR 4 + other factors, where CR 1 = 1 if CR = 1, and CR 1 = 0 otherwise, CR 2 = 1 if CR = 2, and CR 2 = 0 otherwise, and so on. What can you expect about the signs of the coefficients δ i? How would you test that credit rating has no effect on MBR?

28 Multiple Categories The constant partial effect can be tested by testing the restricted model y = β 0 + δ(d 1 + 2D 2 ) + x + u (12) against the unrestricted alternative y i = β 0 + δ 1 D 1 + δ 2 D 2 + x + u. (13) Exploring Further How would you test using Eviews whether the credit rating affects linearly on the municipal bond rate?

29 Multiple Categories Example 4 Effects of law school ranking on starting salaries. Dummy variables top10, r11 25, r26 40, r41 60, and r The reference group is the schools ranked below 100.

30 Multiple Categories Example 4 continues Below are estimation results with some additional covariates (Wooldridge, Example 7.8). Dependent Variable: LOG(SALARY) Sample (adjusted): Included observations: 136 after adjustments ====================================================== Variable Coefficient Std. Error t-statistic Prob C TOP R11_ R26_ R41_ R61_ LSAT GPA LOG(LIBVOL) LOG(COST) ====================================================== R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info crit Sum squared resid Schwarz criterion F-statistic Prob(F-statistic) ======================================================

31 Multiple Categories Example 4 continues The estimation results indicate that the ranking has a big influence on the staring salary. The estimated median salary at a law school ranked between 61 and 100 is about 13% higher than in those ranked below 100. The coefficient estimate for the top 10 is , using (7) we get 100 [exp(0.6996) 1] 101.4%, that is median starting salaries in top 10 schools tend to be double to those ranked below 100.

32 Multiple Categories Example 5 Although not fully relevant, let us for just illustration purposes test constant partial effect hypothesis. I.e., whether δ top10 = 5δ , δ H 0 : = 4δ , δ = 3δ , δ = 2δ (14) Using Wald test for coefficient restrictions in EViews gives F = with df 1 = 4 and df 2 = 126 and p-value This indicates that the there is not much empirical evidence against the constant partial effect for the starting salary increment. The estimated constant partial coefficient is , i.e., at each ranking class starting median salary is estimated to increase approximately by 14%.

33 Multiple Categories Example 5 continues LOG(SALARY) RANK Figure 6.1: Starting median salary and law school ranking.

34 Interaction Involving Dummy Variables 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving Dummy Variables Interaction among Dummy Variables Different Slopes Chow Test Binary Dependent Variable The Linear Probability Model The Logit and Probit Model

35 Interaction Involving Dummy Variables 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving Dummy Variables Interaction among Dummy Variables Different Slopes Chow Test Binary Dependent Variable The Linear Probability Model The Logit and Probit Model

36 Interaction Involving Dummy Variables In Example 3 additional dummy variables were generated by multiplying various other dummy variables. Factually these define interaction terms among dummy variables. Example 6 The same model as in Example 3 can be specified as log(wage) = β 0 +δ f female+δ mar married+δ m f female married+ (15)

37 Interaction Involving Dummy Variables Example 6 continues lm(formula = log(wage) ~ female + married + I(female * married) + educ + educ + exper + tenure + I(exper^2) + I(tenure^2), data = wdfr) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) ** female * married *** I(female * married) e-05 *** educ < 2e-16 *** exper e-07 *** tenure e-05 *** I(exper^2) e-06 *** I(tenure^2) * --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 517 degrees of freedom Multiple R-squared: ,Adjusted R-squared: F-statistic: on 8 and 517 DF, p-value: < 2.2e-16

38 Interaction Involving Dummy Variables Example 6 continues This example allows to estimate wage differential for all the four groups (single men, single women, married men, married women), but we need to be careful in deriving the differentials. It is better to compute the differential as differences of the intercepts of the groups. For example marriage premium for women (compared to single women): The intercept for married women is β 0 + δ f + δ mar + δ m f and the intercept for single women is β 0 + δ f, such that the difference is δ mar + δ m f with estimate as in Example 3. We observe that the marriage premium for men to single men in this specification is directly δ mar (= β 0 + δ mar β 0 ).

39 Interaction Involving Dummy Variables 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving Dummy Variables Interaction among Dummy Variables Different Slopes Chow Test Binary Dependent Variable The Linear Probability Model The Logit and Probit Model

40 Interaction Involving Dummy Variables Consider y = β 0 + β 1 x 1 + u. (16) If the slope depends also on the group, we get in addition to β 0 = β 00 + δ 0 D, (17) for the slope coefficient similarly β 1 = β 11 + δ 1 D. (18) The regression equation is then y = β 00 + β 1 D + β 11 x 1 + δ 1 x 2 + u, (19) where x 2 = D x 1.

41 Interaction Involving Dummy Variables Example 7 Wage example. Test whether return to eduction differs between women and men. This can be tested by defining The null hypothesis is H 0 : δ feduc = 0. β educ = β meduc + δ feduc female. (20)

42 Interaction Involving Dummy Variables Example 7 continues Dependent Variable: LOG(WAGE) Method: Least Squares Sample: Included observations: 526 ====================================================== Variable Coefficient Std. Error t-statistic Prob C MARRMALE MARRFEM SINGFEM FEMALE*EDUC EDUC EXPER TENURE EXPER^ TENURE^ ====================================================== R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info crit Sum squared resid Schwarz criterion Log likelihood F-statistic Durbin-Watson stat Prob(F-statistic) ======================================================

43 Interaction Involving Dummy Variables Example 7 continues ˆδ feduc = with p-value Thus there is no empirical evidence that the return to education would differ between men and women.

44 Interaction Involving Dummy Variables 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving Dummy Variables Interaction among Dummy Variables Different Slopes Chow Test Binary Dependent Variable The Linear Probability Model The Logit and Probit Model

45 Interaction Involving Dummy Variables Suppose there are two populations (e.g. men and women) and we want to test whether the same regression function applies to both groups. All this can be handled by introducing a dummy variable, D with D = 1 for group 1 and zero for group 2.

46 Interaction Involving Dummy Variables If the regression in group g (g = 1, 2) is y g,i = β g,0 + β g,1 x g,i,1 + + β g,k x g,i,k + u g,i, (21) i = 1,..., n g, where n g is the number of observations from group g. Using the group dummy, we can write β g,j = β j + δ j D, (22) j = 0, 1,..., k. An important assumption is that in both groups var[u g,i ] = σ 2 u. The null hypothesis is H 0 : δ 0 = δ 1 = = δ k = 0. (23)

47 Interaction Involving Dummy Variables The null hypothesis (23) can be tested with the F -test, given in (4.20). In the first step the unrestricted model is estimated over the pooled sample with coefficients of the form in equation (21) (thus 2(k + 1)-coefficients). Next the restricted model, with all δ-coefficients set to zero, is estimated again over the pooled sample. Using the SSRs from restricted and unrestricted models, test statistic (4.20) becomes F = (SSR r SSR ur )/(k + 1), (24) SSR ur /[n 2(k + 1)] which has the F -distribution under the null hypothesis with k + 1 and n 2(k + 1) degrees of freedom.

48 Interaction Involving Dummy Variables Exactly the same result is obtained if one estimates the regression equations separately from each group and sums up the SSRs. That is SSR ur = SSR 1 + SSR 2, (25) Where SSR g is from the regression estimated from group g, g = 1, 2. Thus, statistic (24) can be written alternatively as F = [SSR r (SSR 1 + SSR 2 )]/(k + 1), (26) (SSR 1 + SSR 2 )/[n 2(k + 1)] which is known as Chow statistic (or Chow test).

49 Binary Dependent Variable 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving Dummy Variables Interaction among Dummy Variables Different Slopes Chow Test Binary Dependent Variable The Linear Probability Model The Logit and Probit Model

50 Binary Dependent Variable 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving Dummy Variables Interaction among Dummy Variables Different Slopes Chow Test Binary Dependent Variable The Linear Probability Model The Logit and Probit Model

51 Binary Dependent Variable Up until now in regression y = x β + u, (27) where x β = β 0 + β 1 x β k x k, y has had quantitative meaning (e.g. wage). What if y indicates a qualitative event (e.g., firm has gone to bankruptcy), such that y = 1 indicates the occurrence of the event ( success ) and y = 0 non-occurrence ( fail ), and we want to explain it by some explanatory variables?

52 Binary Dependent Variable The meaning of the regression y = x β + u, when y is a binary variable. Then, because E[u x] = 0, E[y x] = x β. (28) Because y is a random variable that can have only values 0 or 1, we can define probabilities for y as P(y = 1 x) and P(y = 0 x) = 1 P(y = 1 x), such that E[y x] = 0 P(y = 0 x) + 1 P(y = 1 x) = P(y = 1 x).

53 Binary Dependent Variable Thus, E[y x] = P(y = 1 x) indicates the success probability and regression in equation 28 models P(y = 1 x) = β 0 + β 1 x β k x k, (29) the probability of success. This is called the linear probability model (LPM). The slope coefficients indicate the marginal effect of corresponding x-variable on the success probability, i.e., change in the probability as x changes, or P(y = 1 x) = β j x j. (30)

54 Binary Dependent Variable In the OLS estimated model ŷ = β 0 + ˆβ 1 x ˆβ k x k (31) ŷ is the estimated or predicted probability of success. In order to correctly specify the binary variable, it may be useful to name the variable according to the success category (e.g., in a bankruptcy study, bankrupt = 1 for bankrupt firms and bankrupt = 0 for non-bankrupt firm [thus success is just a generic term]).

55 Binary Dependent Variable Example 8 Married women participation in labor force (year 1975). (See R-snippet for the R-commands) Call: lm(formula = inlf ~ huswage + educ + exper + I(exper^2) + age + kidslt6 + kidsge6, data = mdfr) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-05 *** huswage * educ e-07 *** exper e-12 *** I(exper^2) ** age e-11 *** kidslt e-14 *** kidsge Signif. codes: 0 *** ** 0.01 * Residual standard error: on 745 degrees of freedom Multiple R-squared: ,Adjusted R-squared: F-statistic: on 7 and 745 DF, p-value: < 2.2e-16

56 Binary Dependent Variable Example 8 continues All but kidsge6 are statistically significant with signs as might be expected. The coefficients indicate the marginal effects of the variables on the probability that inlf = 1. Thus e.g., an additional year of educ increases the probability by (other variables held fixed). Marginal effect of experince on marri women labor force participation Marginal effect of eduction on marrie women labor force participation Probability Probability Experience (years) Education (years)

57 Binary Dependent Variable Some issues with associated to the LPM. Dependent left hand side restricted to (0, 1), while right hand side (, ), which may result to probability predictions less than zero or larger than one. Heteroskedasticity of u, since by denoting p(x) = P(y = 1 x) = x β var[u x] = (1 p(x))p(x) (32) which is not a constant but depends on x, and hence violating Assumption 2.

58 Binary Dependent Variable 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving Dummy Variables Interaction among Dummy Variables Different Slopes Chow Test Binary Dependent Variable The Linear Probability Model The Logit and Probit Model

59 Binary Dependent Variable The first of the above problems can be technically easily solved by mapping the linear function on the right hand side of equation 29 by a non-linear function to the range (0, 1). Such a function is generally called a link function. That is, instead we write equation (19) as P(y = 1 x) = G(x β). (33) Although any function G : R [0, 1] applies in principle, so called logit and probit transformations are in practice most popular (the former is based on logistic distribution and the latter normal distribution). Economists favor often the probit transformation such that G is the distribution function of the standard normal density, i.e., G(z) = Φ(z) = z 1 2π e 1 2 v 2 dv, (34)

60 Binary Dependent Variable In the logit tranformation G(z) = Both as S-shaped ez 1 + e z = e z = z e v dv. (35) (1 + e v 2 ) Probit transformation Logit transformation G(z) G(z) z z

61 Binary Dependent Variable The price, however, is that the interpretation of the marginal effects is not any more as straightforward as with the LPM. However, negative sign indicates decreasing effect on the probability and positive increasing.

62 Binary Dependent Variable Example 9 Probit and logit estimatation married women labor force participation probability. Probit: (family = binomial(link = probit ) in glm) glm(formula = inlf ~ huswage + educ + exper + I(exper^2) + age + kidslt6 + kidsge6, family = binomial(link = "probit"), data = wkng) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) huswage * educ e-07 *** exper e-11 *** I(exper^2) ** age e-11 *** kidslt e-13 *** kidsge Signif. codes: 0 *** ** 0.01 * Null deviance: on 752 degrees of freedom Residual deviance: on 745 degrees of freedom AIC:

63 Binary Dependent Variable Example 9 continues Logit: (family = binomial(link = logit ) in glm) glm(formula = inlf ~ huswage + educ + exper + I(exper^2) + age + kidslt6 + kidsge6, family = binomial(link = "logit"), data = wkng) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) huswage * educ e-07 *** exper e-11 *** I(exper^2) ** age e-10 *** kidslt e-12 *** kidsge Signif. codes: 0 *** ** 0.01 * Null deviance: on 752 degrees of freedom Residual deviance: on 745 degrees of freedom AIC: Qualitatively the results are similar to those of the LPM. (R exercise: create similar graphs to those of the linear case for the marginal effects.)