Discrete Dependent Variable Models James J. Heckman

Transcription

1 Dscrete Dependent Varable Models James J. Heckman Here s the general approach of ths lecture: Economc model (e.g. utlty maxmzaton) Decson rule (e.g. FOC) }{{} Sec. 1 Motvaton: Index functon and random utlty models Econometrc Underlyng model regresson (e.g. dependng on (e.g. solve the FOC for a dependent observed data, dscrete or lmted varable) dependent varable model) }{{} Sec. 2 Setup [Estmaton] }{{} Sec. 4 Estmaton [Interpretaton] }{{} Sec. 3 Margnal Effects We assume that we have an economc model and have derved mplcatons of the model, e.g. FOCs, whch we can test. Convertng these condtons nto an underlyng regresson usually nvolves lttle more than rearrangng terms to solate a dependent varable. Often ths dependent varable s not drectly 1

2 observed, n a way that we ll make clear later. In such cases, we cannot smply estmate the underlyng regresson. Instead, we need to formulate an econometrc model that allows us to estmate the parameters of nterest n the decson rule/underlyng regresson usng what lttle nformaton we have on the dependent varable. We wll present two models n part A whch wll help us brdge the gap between nestmable underlyng regressons and an estmable econometrc model. In part B, we wll further develop the econometrc model ntroduced n part A so that t s ready for estmaton. In part C, we jump ahead to nterpretng our results. In partcular we wll explan why, unlke n the lnear regresson models, the estmated β does not gve us the margnal effect of a change n the ndependent varables on the dependent varable. We jump ahead to ths topc because t wll gve us some nformaton we need when we estmate the model. Fnally, part D wll descrbe how to estmate the model. 1 Motvaton Dscrete dependent varable models are often cast n the form of ndex functon models or random utlty models. Both models vew the outcome of a dscrete choce as a reflecton of an underlyng regresson. The desre to nform econometrc models wth economc models suggests that the underlyng regresson be a margnal cost-beneft analyss calculaton. The dfference between the two models s that the structure of the cost-beneft calculaton n ndex functon models s smpler than that n random utlty models. 2

3 1.1 Index functon models Snce margnal beneft calculatons are not observable, we model the dfference between beneft and cost as an unobserved varable y such that: y = β x + ε where ε f(0, 1), wth f symmetrc. Whle we do not observe y, we do observe y, whch s related to y n the sense that: y = 0 f y 0 and y = 1 f y > 0. In ths formulaton β x s called the ndex functon. Note two thngs. Frst, our assumpton that var(ε) = 1 could be changed to var(ε) = σ 2 nstead, by multplyng our coeffcents by σ 2. Our observed data wll be unchanged; y = 0 or 1, dependng only on the sgn of y, not ts scale. Second, settng the threshold for y gven y at 0 s lkewse nnocent f the model contans a constant term. (In general, unless there s some compellng reason, bnomal probablty models should not be estmated wthout constant terms.) Now the probablty that y = 1 s observed s: Pr{y = 1} = Pr{Y > 0} = Pr{β x + ε > 0} = Pr{ε > β x}. Then under the assumpton that the dstrbuton f of ε s symmetrc, we can wrte: Pr{y = 1} = Pr{ε < β x} = F (β x). where F s the cdf of ε. Ths provdes the underlyng structural model for estmaton by MLE or NLLS estmaton. 3

4 1.2 Random utlty models Suppose the margnal cost beneft calculaton was slghtly more complex. Let y 0 and y 1 be the net beneft or utlty derved from takng actons 0 and 1, respectvely. We can model ths utlty calculus as the unobserved varables y 0 and y 1 such that: y 0 = β x 0 + ε 0, y 1 = γ x 1 + ε 1. Now assume that (ε 1 ε 0 ) f(0, 1), where f s symmetrc. Agan, although we don t observe y 0 and y 1, we do observe y where: y = 0 f y 0 > y 1, y = 1 f y 0 y 1. In other words, f the utlty from acton 0 s greater than acton 1,.e. y 0 > y 1, then y = 0. y = 1 when the converse s true. Here the probablty of observng acton 1 s: Pr{y = 1} = Pr{y 0 y 1 } = Pr{β x 0 + ε 0 γ x 1 + ε 1 } = Pr{ε 1 ε 0 β x 0 γ x 1 } = F (γ x 1 β x 0 ). 2 Setup The ndex functon and random utlty models provde the lnk between an underlyng regresson and an econometrc model. Now we ll begn the process of flushng out the econometrc model. Frst we ll consder dfferent specfcatons for the dstrbuton of ε and later, n part C, examne how margnal effects are derved from our probablty model. Ths wll pave the way for 4

5 our dscusson of how to estmate the model. 2.1 Why Pr{y = 1}? In both ndex functon and random utlty models, the probablty of observng y = 1 has the structure: Pr{y = 1} = F (β x). Why are we so nterested n the probablty that y = 1? Because the expected value of y gven x s just that probablty: E[y] = 0 (1 F ) + 1 F = F (β x). 2.2 Common specfcatons for F(β x) How do we specfy F (β x)? There are four basc specfcatons that domnate the lterature. (a) Lnear probablty model (LPM): F (β x) = β x (b) Probt: F (x) = Φ(β x) = β x φ(t)dt = β x (c) Logt: F (β x) = Λ(β x) = (d) Extreme Value Type I: eβ x 1 + e β x F (β x) = W (β x) = 1 e eβ x 1 2π e t2 2 dt 2.3 Decdng whch specfcaton to use Each specfcaton has ts advantages and dsadvantages. 5

6 (1) LPM. The lnear probablty model s popular because t s extremely smple to estmate. Ths smplcty, however, comes at a cost. To see what we mean, set up the NLLS regresson model. y = E[y x] + (y E[y x]) = F (β x) + ε = β x + ε. Because F s lnear, ths just collapses down to the CR model. Notce that the error term: ε = 1 β x wth probablty F = β x and β x wth probablty 1 F = 1 β x Ths mples that: var[ε x] = E[ε 2 x] E 2 [ε x] = E[ε 2 ] = F (1 β x) 2 + (1 F ) ( β x) 2 = F 2F β x + F [β x] 2 + [β x] 2 F [β x] 2 = F 2F β x + [β x] 2 = β x 2[β x] 2 + [β x] 2 = β x(1 β x). So our frst problem s that ε s heteroscedastc n a way that depends on β. Of course, absent any other problems, we could manage ths wth an FGLS estmator. A second more serous problem, however, s that snce β x s not confned to the [0, 1] nterval, the LPM leaves open the possblty of predcted probabltes that le outsde the [0, 1] nterval, whch s nonsenscal, and of negatve varances: β x > 1 E[y] = F = β x > 1, var[ε] = β x(1 β x) < 0 β x < 0 E[y] < 0, var[ε] < 0. Ths s a problem that s harder to correct. We could defne F = 1 f F (β x) = β x > 1 and F = 0 f F (β x) = β x < 0, but ths procedure creates unrealstc knks at the truncaton ponts for (y, x β x = 0 or 6

7 1). (2) Probt vs. Logt. The probt model, whch uses the normal dstrbuton, may be justfed by appealng to a central lmt theorem,whle the logt model can be justfed by the fact that t s smlar to a normal dstrbuton but has a much smpler form. The dfference between the logt and normal dstrbuton s that the logt has slghtly heaver tals. The standard normal has mean zero and varance 1 whle the logt has mean zero and varance equal to π 2 /3. (3) Extreme Value Type I. The extreme value type I dstrbuton s the least common of the four models. It s mportant to note that ths s an asymmetrc pdf. 3 Margnal effects Unlke n lnear models such as the CR or Neo-CR models, the margnal effect of a change n x on E[y] s not smply β. To see why, dfferentate E[y] by x: E[y] x = F (β x) (β x) = f(β x)β. (β x) x These margnal effects look dfferent n each of the four basc probablty models. 1. LPM. Note that f(β x) = 1, so f(β x)β = β, whch s the same as n the CR-type models, as expected. 2. Probt. Now, f(β x) = φ(β x) = 1 2π e (β x) 2 2, so f(β x)β = φβ. 7

8 3. Logt. In ths case: f(β x) = Λ(β x) (β x) = eβ x 1 + e e β x x β x (1 + e β x ) ) 2eβ e β x x = (1 eβ 1 + e β x 1 + e β x = Λ(β x) (1 Λ(β x)) Gvng us the margnal effect f(β x)β = Λ(1 Λ)β. 3.1 Convertng probt margnal effects to logt margnal effects To convert a probt coeffcent estmate to a logt coeffcent estmate, from the dscusson above comparng the varances of probt and logt random varable, t would make sense to multply the probt coeffcent estmate by π 3 = 1.8 (snce varance of logt s π 2 /3 whereas varance of the normal s 1). But Amemya suggests a dfferent converson factor. Through tral and error he found that 1.6 works better at the center of the dstrbuton, whch demarcates the mean value of the regressors. At the center of the dstrbuton, F = 0.5 and β x = 0. Well Φ(0) = whle Λ(0) = So we want to solve the equaton, β probt = 0.25β logt ths gves us β logt = 1.6β proft. 4 Estmaton and hypothess testng There are two basc methods of estmaton, MLE and NLLS estmaton. Snce the former s far more popular, we ll spend most of our tme on t. 4.1 MLE Gven our assumpton that the ε are d, by the defnton of ndependence, we can wrte the jont probablty of observng {y } =1,...,n as Pr{y 1, y 2,..., y n } = Π y =0[1 F (β x )] Π y =1[F (β x )]. 8

9 Usng the notatonal smplfcaton F (β x ) = F, f(β x ) = f, f (β x ) = f we can wrte the lkelhood functon as: L = Π (1 F ) 1 y (F ) y. Snce we are searchng for a value of β that maxmzes the probablty of observng what we have, monotoncally ncreasng transformatons wll not affect our maxmzaton result. Hence we can take logs of the lkelhood functon; and snce maxmzng a sum s easer than maxmzng a product, we take the log of the lkelhood functon: Now estmate β by: ln L = {(1 y ) ln[1 F ] + y ln F }. β = arg max ln L. β Wthn the MLE framework, we shall now examne the followng sx (estmaton and testng) procedures: A. Estmatng β; B. Estmatng asymptotc varance of β; C. Estmatng asymptotc varance of the predcted probabltes; D. Estmatng asymptotc varance of the margnal effects; E. Hypothess testng; and F. Measurng goodness of ft A. Estmatng β To solve max ln L we need to examne the frst and second order condtons. β 9

10 Frst Order Condtons (FOCs): A necessary condton for maxmzaton s that the frst dervatve equal zero: If we wrte: and we plug n: ln L β then we just need to solve: [ (1 y ) f ] f + y x 1 F F = ln L (β x) (β x) β F (β x) (β x) = f(β x), = ln L (β x) x = 0. ln L = {(1 y ) ln[1 F ] + y ln F }, [ (y 1)f F + y f (1 F ) = (1 F )F (y F )f x (1 F )F = 0 {FOCs} ] x = 0 Now we look at the specfc FOCs n three man models: (1) LPM. Snce F = β x and f = 1, our FOC becomes: (y F )f x = (y β x )x = 0. (1 F )F (1 β x )β x Ths s just a set of lnear equatons n x and y whch we can solve explctly for β n two ways. () Least squares. The frst soluton gves us a result that s remnscent of famlar least squares predctors. (a) GLS. Solvng for the β n the numerator, we get somethng resemblng the generalzed least squares estmator, where each x s 10

11 weghted by the varance of ε. β x 2 y x = (1 β x )β x (1 β x )β x y x (1 β β = x )β x = x 2 (1 β x )β x (b) OLS. If we assume homoscedastcty,.e: y x var(ε ) x 2 var(ε ). (1 β x )β x = var(ε ) = var(ε) = σ 2 Then the equaton above collapses nto the standard OLS estmator of β : β = 1 var(ε) 1 var(ε) () GMM. If we rewrte y β x = ε y x y x =. x 2 x 2 then the FOC condtons resemble the generalzed method of moments condton for solvng the heteroscedastc lnear LS model: ε x = 0 (1 β x )β x ε x var(ε ) = 0. Agan, f we assume homoskedastcty, we get the moment condton for solvng the CR model: 1 ε x = var(ε) ε x = 0. Note that each of these estmators s dentcal. Some may be more effcent than others n the presence of heteroscedastcty, but, n general, they are just dfferent ways of motvatng the LS estmator. 11

12 (2) Probt. Notng that F = Φ, f = φ, the FOC s just: (y F )f x = (y Φ )φ x (1 F )F (1 Φ )Φ = y φ x φ x (1 Φ )Φ (1 Φ ) If we defne (refer the results n the Roy Model handout): λ 0 = E(z z > β x ) = φ (1 Φ ) λ 1 = E(z z < β x ) = φ Φ Then we can rewrte the FOC as: λ x = 0 where: λ = λ 0 f y = 0, and λ 1 f y = 1. Note that, unlke n the LPM, these FOC are a set of nonlnear equatons n β. They cannot be easly solved explctly for β. So β has to be estmated usng the numercal methods outlned n the Asymptotc Theory Notes. (3) Logt. Here F = Λ and f = Λ (1 Λ ), so the FOC becomes: (y F )f x = (y Λ )Λ (1 Λ )x = 0 (1 F )F (1 Λ )Λ (y Λ )x = 0. Interestngly, note that we can wrte y Λ = ε so that the FOC can be wrtten (y Λ )x = ε x = 0, whch s smlar to the moment 12

13 condtons for the LPM. Lke the probt model, however, the FOC for the logt model are nonlnear n β and must therefore be solved usng numercal methods. Second Order Condton (SOC): Together, the FOCs and the SOC that the second dervatve or Hessan be negatve defnte are necessary and suffcent condtons for maxmzaton. To verfy the second order condton, let: So that we need to check: f(β x) (β x) = f (β x), 2 [ ] ln L ln L (β = β β (β x) (β x) x x) β 2 ln L = (β x) (β x) xx = [ ] (y F )f x x (β < 0. x ) (1 F )F (1) LPM. We can prove that the LPM satsfes the SOC β B: [ (y β ] x )x x (β x ) (1 β x )β x = [ x (y β ] x )x (1 β x )β x (1 β x ) 2 (β x ) 2(1 2β x ) x = [ ββ x 3 y x + 2y β x 2 ] x (1 β x ) 2 (β x ) 2 = [ (y β x ) 2 ] x (1 β x ) 2 (β x ) 2 x < 0 (Usng fact y {0, 1} y 2 = y ) (2) Probt. The same can be sad about the probt model, and the proof follows from the results n the Roy model. Frst, note that φ (β x) = 13

14 β xφ(β x). Takng the dervatve of the frst dervatve we need to show: (β x ) [λ x ]x = (β x ) [λ ]x x < 0. We can smplfy ths expresson usng results for the truncated normal (see results on truncated normal n Roy Model handout): [ ] λ 0 φ = (β x ) (β x) 1 Φ = β x φ φ 2 1 Φ (1 Φ ) = 2 β x λ 0 λ 2 0 λ 1 (β x ) So that we can wrte the SOC as: = λ 0 (β x + λ 0 ) < 0 [ ] φ = = β x φ φ2 (β x ) Φ Φ Φ 2 = β x λ 1 λ 2 1 = λ 1 (β x + λ 1 ) < 0 λ (β x + λ )x x < 0, Where: λ = λ 0 = φ (1 Φ ), f y = 0, and λ 1 = φ Φ, f y = 1 (3) Logt. Takng the dervatve of the FOC for logt, we get the SOC : [y Λ )x ] x (β x ) = Λ (1 Λ )x x < 0 whch clearly holds β B. Note that snce the Hessan does not nclude y, the Newton-Raphson method of numercal optmzaton, whch uses H n ts teratve algorthm, and the method of scorng, whch uses E[H], are dentcal n the case of the logt model. Why? Because E[H] s taken wth respect to the dstrbuton of y. 14

15 We ve shown that the LPM, probt and logt models are globally concave. So the Newton-Raphson method of optmzaton wll converge n just a few teratons for these three models unless the data s very badly condtoned. B. Estmatng the Asy Cov matrx for β Recall the followng two results from the MLE notes: (a) T ( β β 0 ) N(0, I(β 0 ) 1 ) ( 1 2 ln L where I(β 0 ) = plm T β β (b) lm 1 ln L ln L = E T T β β β [ 2 ] ( ln L 1 = E = plm β β T β0 ( 1 T 2 ln L β β ) ln L β β0 ) ln L ) β 1 = lm T T 2 ln L β β β. We have three possble estmators for Asy.Var[ β] based on these two facts. (1) Asy.Var[ β] = Ĥ 1 where Ĥ = [ ] (y F )f x (β x x ) (1 F )F. β [ (2) Asy. Var[ β] = E[H] 1 2 ] ln L where E[H] = E. β β In any model where H does not depend on y, E[H] = Ĥ snce the expectaton has taken over the dstrbuton of y. So n models such as logt the frst and second estmators are dentcal. In the probt model, Ĥ depends on y so Ĥ E[H]. Amemya ( Qualtatve Response Models: A Survey, Journal of Economc Lterature, 19, 4, 1981, pp ) showed that: E[H] probt = λ 0 λ 1 x x = φ 2 (1 Φ ) x x. 15

16 (3) The Berndt, Hall, Hall and Hausman took the followng estmator from T.W. Anderson (1959) whch we call the TWA estmator: Asy.Var[ β] = Ĥ 1 where Ĥ = ( ) ( ) (y F )f x x (y F )f (1 F )F (1 F )F Notce there s no negatve sgn before the Ĥ 1, as the two negatve sgns multply out. Note that the three estmators lsted here are the basc three varants on the gradent method of teratve numercal optmzaton explaned n the numercal optmzaton notes. C. Estmatng the Asy Cov matrx for predcted probabltes, F ( β x). For smplcty, let F ( β x) = F. Recall the delta method: f g s twce contnuously dfferentable and T (θ T θ 0 ) d N(0, σ 2 ), then: T (g(θt ) g(θ 0 )) d N(0, [g (θ 0 )] 2 σ 2 ). Applyng ths to F we get ( d T F ( β) F (β 0 )) N(0, [ F (β 0 )] 2 V ar[ β]), where β 0 s the true parameter value. So a natural estmator for the asymptotc covarance matrx for the predcted probabltes s: Asy.Var[ F ( ) ] = F ( ) V F β β where V =Asy.Var[ β]. Snce: F β = F ( β x) ( β x) β = ( f)x, we can wrte the estmator as: Asy.Var[ F ] = ( f) 2 x V x. D. Estmatng the Asy Cov matrx for margnal effects, f( β x)β. 16

17 To recap, the margnal effects are gven by: E[y] x = F x = F (β x) (β x) x = fβ. To smplfy notaton, let f( β x) β = f β = γ. Agan, usng the delta method as motvaton, a sensble estmator for the asymptotc varance of γ( β) would be: ( ) ( ) γ γ Asy. Var[ γ] = β V β, where V s as above. We can be more explct n defnng our estmator by notng that: γ f β β = = f β β β + = fi + f β x, ( β x) f ( β x) (β x) β β Ths gves us: ( Asy.Var[ f β] = fi + f ) ( β x V fi + f β x). ( β x) ( β x) Ths equaton stll does not tell us much. It may be more nterestng to look at what the estmator looks lke under dfferent specfcatons of F. (1) LPM. Recall F = β x, f = 1, and f = 0, so: Asy.Var[ f β] LP M = V =Asy.Var[ β] (2) Probt. Here F = Φ, f = φ and f = β xφ, leavng us wth: Asy.Var[ f β] probt = φ (I ( β ) ( 2 x) ( β β x V I x) β x (1) Logt. Now F = Λ, f = Λ(1 Λ), and f = Λ(1 Λ)[1 2Λ], so: Asy. Var[ f β] ] 2 ( ) ( logt = [ Λ(1 Λ) I + (1 2 Λ) β x V I + (1 2 Λ) β x ) ) 17

18 E. Hypothess testng Suppose we want to test the followng set of restrctons, H 0 : Rβ = q. If we let p be the number of restrctons n R,.e. rank (R), then MLE provdes us wth three test statstcs (refer also the Asymptotc Theory notes). (1) Wald test W = ( R β q) [R Est.Asy.Var( β)r ](R β q) χ 2 (p). Example. Suppose H 0 : the last L coeffcents or elements of β are 0. Defne R = [0, I L ] and q = 0; and let β L be the last L elements of β. Then we get W = β L V 1 L β L. (2) Lkelhood rato test LR = 2[ln L R ( β) ln L( β)] χ 2 (p) where ln L R ( β) and ln L( β) are the log lkelhood functon evaluated wth and wthout the restrctons on β, respectvely. Example. To test H 0 : all slope coeffcents except that on the constant term are 0, let ln L R ( β) = {y ln F + (1 y ) ln(1 F )} = n {y /n) ln F + ([1 y ]/n) ln(1 F )} = n{p ln P + (1 P ) ln(1 P )} where P s the proporton of observatons wth y = 1. (3) Score or Lagrange multpler test Wrte out the Lagrangan for the MLE problem gven the restrcton β = β R : L = ln L λ(β β R ). The frst order condton s ln L β = λ. 18

19 So the test statstc s LM = λ R V λ R, where λ R s just λ evaluated at β R. Example. In the logt model, suppose we want to test H 0 : all slopes are 0. Then LM = nr 2, where R 2 s the uncentered coeffcent of determnaton n the regresson of (y P ) on x, where P s the proporton of y = 1 observatons n the sample. (Don t worry about how ths s derved.) F. Measurng goodness of ft There are three basc ways to descrbe how well a lmted dependent varable model fts the data. (1) Log lkelhood functon, ln L. The most basc way to descrbe how successful the model s at fttng the data s to report the value of ln L at β. Snce the hypothess that all other slopes n the model are zero s also nterestng, ln L computed wth only a constant term (ln L 0 ), whch should also be reported. Comparng ln L 0 to ln L gves us an dea of how much the lkelhood mproves on addng the explanatory varables. (2) Lkelhood rato ndex, LRI. An analog to the R 2 n the CR model s the lkelhood rato ndex, LRI = 1 (ln L/ ln L 0 ). Ths measure has an ntutve appeal n that t s bounded by 0 and 1 snce ln L s a small negatve number whle ln L 0 s a large negatve number, makng ln L/ ln L 0 < 1. If LRI = 1, F = 1 whenever y = 1 and F = 0 whenever y = 0, gvng us a perfect ft. LRI = 0 when the ft s mserable,.e. ln L = ln L 0. Unfortunately, values between 0 and 1 have no natural nterpretaton lke they do n the R 2 measure. 19

20 (3) Ht and mss table. A useful summary of the predctve ablty of the model s a 2 2 table of the hts and msses of a predcton rule: ŷ = 1 f F ( β) x) > F, and 0 otherwse. y = 0 y = 1 Hts # of obs. where ŷ = 0 # of obs. where ŷ = 1 Msses # of obs. where ŷ = 1 # of obs. where ŷ = 0 The usual value for F = 0.5. Note, however, that 0.5 may seem reasonable but s arbtrary. For more on ths measure, see Greene, p