1 Binary Response Models

Transcription

1 Bnary and Ordered Multnomal Response Models Dscrete qualtatve response models deal wth dscrete dependent varables. bnary: yes/no, partcpaton/non-partcpaton lnear probablty model LPM, probt or logt models ordered: self-rated heath status excellent/good/bad dscrete responses latent ndex models multnomal categorcal mutually exclusve: transportaton mode random utlty models Bnary Response Models An Example: Outcome of nterest y: an ndvdual s { decson on gong to college or not 0 49 went to college 396 left school before or after gettng a hgh school degree Data: NLS, 2 85 observatons Covarates x: IQ father s educaton mother s educaton Let s consder the followng three econometrc models.. Lnear Probablty Model LPM y = x β + ɛ assumng E ɛ x = 0. Notce that ths model must have heteroskedastcty f y x Bernoull dstrbuton, whch s specfed as Pry = x = x β: Ey x = Pry = x = x β, V ary x = Pry = x Pry = 0 x = x β x β constant. If an OLS estmator s appled to LPM, then standard errors must be corrected for heteroskedastcty. A problem wth LPM s that for some values of the covarates x t may lead to predcted values beng outsde the nterval [0, ]. Such predctons would be less eff cent and awkward, because the nformaton that the dependent varable s bounded between 0 and s gnored. Alternatve models that take nto account the bounds of y nclude probt and logt models. Table gves results for ths LPM usng the NLS data. Table : LPM Estmates, NLS data N = 85 varables βols standard error average margnal effect ntercept IQ father s educaton mother s educaton Ths handout s supplemental to MMA Chapters 4 and 5. Part of ths handout s based on my prevous notes taken from Gudo Imbens s courses gven at UC-Berkeley. A must-vst webste s 2 See for detal.

2 .2 Probt and Logt Models As alternatves to LPM, the probt and logt model can be motvated by a latent ndex model, whch can also be appled to ordered multnomal responses. Latent ndex models could provde a ratonale for a partcular functonal form for Ey x. Latent varables can be ntroduced nto bnary outcome or response models through two dfferent ways:. The latent varable s contnuous and unobservable wth a threshold whch makes t partally observable; 2. The latent varable s the dfference n the utlty that s assocated wth each alternatve, whch assumes that the observed bnary outcome s a result of ndvdual choce based on maxmum utlty. 3 Latent ndex models have the advantage of provdng extensons to multnomal and ordered outcomes and censored outcomes. Now consder the followng latent ndex model: y = x β + ɛ, where y s unobservable, but x s are observable, and E ɛ x = 0; y = {y > 0}, where y s observable and bnary. The probt and logt model lnk the unobservable y and the observable y through a dstrbuton F ɛ x beng symmetrc about zero: Also note that Pry = x = Ey x = Pry > 0 x = Prx β + ɛ > 0 = Prɛ > x β = F ɛ x x β }{{} a nonlnear model for Ey x Pry = 0 x = Ey x = F ɛ x x β. and the margnal effect of x s V ary x = Pry = x Pry = 0 x = F ɛ x x β F ɛ x x β, Ey x x = f ɛ x x β β, where f ɛ x s the p.d.f. of F ɛ x. Ths mples that, for bnary choce models, Ey x sgn = sgn β. x The log lkelhood functon s therefore: ln Lβ = ln N Pry = x {y = } Pry = 0 x {y = 0} }{{} mutually exclusve events = ln N Pry = x y Pry = 0 x y = ln N [ Fɛ x x β ] y [ F ɛ x x β ] y = N y lnf ɛ x x β + y ln F ɛ x x β, 3 Ths method emphaszes the need to dstngush between covarates that vary across alternatves such as costs for a gven ndvdual and covarates that are nvarant across alternatves such as gender for a gven ndvdual. 2

3 wth F.O.C. s: 0 = ln L β β = N y x f ɛ x x β F ɛ x x β y x f ɛ x x β F ɛ x x β = N f ɛ x x y x β F ɛ x x β x f ɛ x x β F ɛ x x β + y f ɛ x x x β F ɛ x x β = N f ɛ x x y x β F ɛ x x β F ɛ x x β x F ɛ x x βf ɛ xx β F ɛ x x β F ɛ x x β = N y F ɛ x x β F ɛ x x β F ɛ x x β f ɛ xx β x, where f ɛ x s the p.d.f. of F ɛ x. Solvng for the maxmum lkelhood estmator requres numercal optmzaton. There s no explct expresson for β ML except for y F ɛ x x β ML f ɛ x x β ML x = 0. * F ɛ x x β ML F ɛ x x β ML There are a few ponts to stress here:. Equaton * can be vewed as FGLS {}}{ y Êy x = ɛ { }} { f ɛ x x β ML x y F F ɛ x x β ML F ɛ x x ɛ x x β ML β ML }{{}}{{} OLS V arɛ x }{{} WLS = For bnary response models, β ML s consstent f Pry = x s correctly specfed as F ɛ x x β. Ths also means that Ey x = F ɛ x x β The asymptotc varance of β ML s the nverse of the nformaton matrx evaluated at the true parameter value β 0 [Iβ 0 ] : a β ML N β 0, [Iβ 0 ] 4 Read MMA p ln L Iβ 0 = E fɛ x β β = E fɛ x ln L β=β0 = E fɛ x β fɛ x x β 0 2 x x F ɛ x x β 0 F ɛ x x β 0 ln L β. β=β0 3

4 4. The estmated asymptotc varance of β ML s AsyV ar β ML = = Î βml 2 f ɛ x x β ML x x F ɛ x x β ML F ɛ x x β ML..3 Probt Model A probt model assumes: ɛ x N0,. The reason for restrctng the varance to be s that the probt model cannot dstngush between one data generatng process wth parameters β and 2 and the other wth β/ and. The probt model lnks the unobservable y and the observable y n the followng way: Pry = x = Pry > 0 x = Prx β + ɛ > 0 = Prɛ > x β = Prɛ < x β = Φx β Pry = 0 x = Φx β, where Φ s the c.d.f. of a standard normal dstrbuton. For a probt model, equaton * becomes φx β probt x y Φx β probt Φx β probt Φx β probt where φ s the p.d.f. of a standard normal dstrbuton. The estmated asymptotc varance of β probt s = 0, AsyV ar β probt = = Î βprobt 2 φx β probt x x Φx β probt Φx β probt. Table 2 gves the estmates for ths probt model. Table 2: Probt Estmates, NLS data N = 85 varables βprobt standard error average margnal effect ntercept IQ father s educaton mother s educaton The last column n the table calculates the average margnal effect based on β probt. Ths s the key results to report because β themselves are not very meanngful. For a probt model, the margnal effect for observaton s: Ey x = Pry = x = Φx β = φx x x x ββ. 4

5 Ths s then averaged across all observatons to get the average margnal effect: N φx ββ, whch s preferred to and not equal to φx ββ where x = N x. Ths average margnal effect s vewed as a more nterpretable summary measure of the effect of the covarates on the dstrbuton of the outcome..4 Logt Model A logt model assumes that the dstrbuton of ɛ x s logstc, wth p.d.f.: f ɛ ɛ = Notce how smlar t s to ts c.d.f.: expɛ + expɛ 2 = exp ɛ, whch s symmetrc about zero. + exp ɛ 2 F ɛ ɛ Λɛ = expɛ + expɛ = + exp ɛ f ɛ ɛ = Λɛ Λɛ. Gven f ɛ ɛ, a logstc random varable ɛ has mean 0 and varance π 2 / See wolfram.com/logstcdstrbuton.html for detals about ths dstrbuton. The logt model lnks the unobservable y and the observable y n the followng way: Pry = x = Pry > 0 x = Prx β + ɛ > 0 = Prɛ > x β = expx β + expx β Λx β Pry = 0 x = Λx β = For a logt model, equaton * becomes + expx β. x y Λx β logt = N x y expx β logt + expx β logt where Λ s the c.d.f. of the logstc dstrbuton. For the ntercept term, we have Ths mples that y Λx β logt = N y expx β logt + expx β logt = 0, y = Λx N N β logt. y = Pry = x p. Ths means that, for logt models, the average n-sample predcted probablty p wll always be equal to the sample proporton y. = 0, 5

6 The estmated asymptotc varance of β logt s AsyV ar β logt = = Î βlogt Λx β logt Λx β logt x x. The results are n Table 3. The average margnal effect s: N Ey x x = N = N Pry = x = x N expx β + expx β Λx β Λx β β + expx β Table 3: Logt Estmates, NLS data N = 85 β. varables βlogt standard error average margnal effect ntercept IQ father s educaton mother s educaton Note that the parameter estmates are very dfferent for the logt and the probt model. However, the estmates of average margnal effects are very close, suggestng that n practce t does not make much dfference whch model probt or logt to choose for bnary outcomes. Nonetheless, the logt model s much easer to work wth computatonally than the probt model..5 Notes. For both probt and logt models for bnary outcomes, the rato of the coeff cents for two dfferent regressors equals the rato of the two margnal effects. 2. For both probt and logt models for bnary outcomes, the sgn of the coeff cent gves the sgn of the margnal effect. 3. For the logt model: odds rato or relatve rsk Pry = x Pry = 0 x = Λx β Λx β = expx β log-odds rato Pry = x ln Pry = 0 x = x β. 4. Read MMA p.470 about the nterpretaton of β logt. 5. Identfcaton of a latent ndex model requres a restrcton on the varance of the error term ɛ because the latent ndex model can only dentfy β up to a scale. All that s observed s whether or not y > 0, or equvalently whether or not x β + ɛ > 0. However, ths s equvalent to whether or not x β/λ + ɛ/λ > 0 for any λ > 0. Placng a restrcton on the varance of the error term ɛ secures the unqueness of β. The error varance s set to n the probt model and π 2 /3 n the logt model. 6. The threshold for the latent ndex model doesn t need to be 0. If more generally Pry = x, z = Pry > z δ x, z = Prx β + ɛ > z δ = Prɛ > z δ x β = F ɛ x,z x β z δ, 6

7 then β and δ can be separately dentfed f and only f all components of x and z dffer. In partcular, f both x and z nclude ntercept terms, then these two ntercepts cannot be separately dentfed. Therefore, we must normalze one of the two ntercepts to be 0. Also note that the mean of the error ɛ needs to be normalzed to 0 by ncludng an ntercept term. 7. Consder a bnary choce wth the followng addtve random utlty model: U 0 = x 0β 0 + z γ 0 + ɛ 0 U = x β + z γ + ɛ Pry = x, x 0, z = PrU > U 0 x, x 0, z = Prɛ ɛ 0 > x β + x 0β 0 z γ γ 0 = F ɛ ɛ 0 x,x 0,zx β x 0β 0 + z γ γ 0 assumng that F ɛ ɛ 0 x,x 0,z s symmetrc about zero. For alternatve-nvarant regressors z, such as ndvdual s attrbutes race and sex, for example, only the dfference n the assocated parameters γ γ 0 can be dentfed. For alternatve-varyng regressors x and x 0 that dffer by alternatve such as the cost assocated wth each alternatve, the coeff cents β and β 0 can dffer by alternatve, but t s customary to set β 0 = β = β. Thus, we often adopt the followng model: 2 Ordered Response Models Pry = x, x 0, z = F ɛ ɛ 0 x,x 0,z x x 0 β + z γ γ 0. Here we consder dscrete response models wth more than two possble responses, and mportantly, these dscrete responses are ordered. There are two mportant cases:. There s an underlyng contnuous varable but we only observe an ndcator for a partcular range. An example s earnngs data that may come coded n ntervals. 2. Examples such as educatonal choces where outcomes may be coded as less than hgh school, hgh school, some college and more than college. Such data are referred to as nterval-coded data. For case, we typcally model the underlyng contnuous varable y as lnear and normal or logstc wth a set of covarates x: y = x β + ɛ, where y s contnuous and unobservable wth the observed outcome an ndcator for j = 0,, 2,, J: y = j f α j y < α j+ wth α 0 =, α J+ =, and α j < α j. In ths case the key assumpton s that the boundary values α j s are known. In ths case we are typcally nterested n the condtonal expectaton of the latent outcome Ey x, rather than n the dstrbuton of the observed outcome, Pry = j x. For case 2, the responses are ordered but there s no clear mappng from the underlyng contnuous response whch may tself be somewhat vague to the dscrete response. For example, respondents n a survey may be asked about ther nterest n a partcular servce and asked to respond n one of three categores: not nterested, somewhat nterested and very nterested. In that case we may stll wsh to model 7

8 the response through the same latent varable approach: y = x β + ɛ, where y s contnuous and unobservable, y = j f α j y < α j+ wth j = 0,, 2,, J; α 0 =, α J+ =, and α j < α j. However, n ths case we may not wsh to assume that the boundary values α j s are known, and we may prefer to estmate them jontly wth the parameters β. In ths case we may be also nterested n the dstrbuton of the observed outcome. For both cases and 2, we may assume that the ɛ s normal wth mean zero and varance 2. The condtonal probabltes are: Pry = j x = Pry < α x = Prx β + ɛ < α = Pr ɛ < α x β = Φ α x β Prα j y < α j+ x = Prα j x β ɛ < α j+ x β = Φ α j+ x β Pry α J x = Prɛ α J x β = Pr ɛ α J x β = Φ α J x β Φ α j x β j = 0 j =,, J j = J Ths way we can buld up the log lkelhood functon as a functon of β, 2 and α usng the above results. ln Lα, β, 2 = ln N J Pry = j x {y =j} j=0 = N {y = 0} ln Pry = 0 x + N J j= There are several caveats n maxmzng ths log lkelhood functon. {y = j} ln Pry = j x + N {y = J} ln Pry = J x.. Consder the case where α s are known. If J =, then there are two alternatves. We cannot dentfy 2. We are back n the bnary response case, where the error varance s not dentfed, and we typcally set t equal to n a probt model. Wth J >, we can dentfy the varance separately and we do not need to normalze the parameters. Wth large J, we get approxmately back to the case where we observe the actual value of the outcome. In fact, n all cases we only observe y dscretely, so we could argue that the ordered dscrete response model s always approprate. In practce, f J s bg, people rarely bother usng a dscrete response model and just go ahead wth the standard lnear model, treatng y as a contnuous varable. 2. Consder the case wth α unknown. Ths s a much more dff cult case, both n terms of computaton and n terms of nterpretaton. Now there are a couple of normalzatons. Frst, we cannot dentfy the ntercept n β separately from the locaton of the boundary values α, so typcally we just normalze the ntercept to zero. Second, we cannot dentfy the scale of the boundary values from the error varance, so typcally we normalze 2 to. 3. Another ssue n the case of unknown α s what we should report. The parameters β are not very useful n ther own rght, except that they tell us whether the correspondng covarates are postvely or negatvely assocated wth the latent outcome y, f that latent outcome s of any nterest. Also keep n mnd that a postve value for β tells us that an ncrease n x s assocated wth a lower probablty of choosng the lowest ranked category y = 0, and a hgher probablty of choosng the hghest ranked category y = J; t does not tell us what the sgn s on the probablty that y = j for nteror j unless we can calculate that margnal effects ourselves. 8