Chapter 14: Logit and Probit Models for Categorical Response Variables

Transcription

1 Chapter 4: Logt and Probt Models for Categorcal Response Varables Sect 4. Models for Dchotomous Data We wll dscuss only ths secton of Chap 4, whch s manly about Logstc Regresson, a specal case of the famly of Generalzed Lnear Models presented n Chap 5. Note: Do not confuse Generalzed least squares wth Generalzed lnear models Example. Votng Intentons n the 988 Chlean Plebscte Varables: regon: C, Central; M, Metropoltan Santago area; N, North; S, South; SA, cty of Santago. populaton: Populaton sze of respondent's communty. sex: F, female; M, male. age: n years. educaton: P, Prmary; S, Secondary; PS, Post secondary. ncome: Monthly ncome, n Pesos. statusquo: Scale of support for the status quo. vote: Y, wll vote yes (for Pnochet); N, wll vote no (aganst Pnochet); U, undecded; A, wll abstan.

2 Q: What type of statstcal analyss s called for here? What s the queston?! [R code] 2

3 Lnear least squares regresson s certanly problematc for bnary (dchotomous) outcome: errors cannot be normally dstrbuted or have constant varance and the lnear predctor does not constran predctons to the nterval [0,]. Fundamental dea: specfy a model for π = PY ( = ). The usual approach s to specfy a model n whch a functon g( π ) s lnear n the explanatory varables: g( π ) = α + βx. In ths case we want to consder functons that satsfy g ( α + βx ) ( 0, ). One famly of functons that satsfy ths are CDF s. For a normal (Gaussan) cdf, Φ(), ths s called the lnear probt model. π =Φ ( α + βx ) z The more commonly used CDF s the logstc dstrbuton functon, Λ ( z) = (+ e ) whch we call the lnear logstc regresson or lnear logt model π =Λ ( α + βx ) = + exp ( α + βx ) [ ] There sn t much dfference between these two. 3

4 Fgure 4.3 When the varances of the cumulatve logstc and cumulatve normal dstrbutons are equalzed, they are nearly ndstngushable. The logt model has a more convenent nterpretaton when we re express t π = exp( α + βx ) π π loge = α + βx π 4

5 Wth ths model the log odds (of a yes, or of Y = rather than 0) s a lnear functon of the explanatory varable(s). π The functon loge s called the logt of the probablty. π But how do we ft ths model to data? Not by least squares! We have specfed two of the three components of a generalzed lnear model: () The systematc component, the lnear predctor α + β X, or more generally, α + β X + β X + + β X 2 2 k k (2) The lnk functon g( π ) that relates the mean of the response varable to the lnear predctor (note that the mean of a bnary response varable s EY ( ) = PY ( = ) = π ). And fnally we need the (3) The random component, probablty model for the response Y. Y Bern( π = (+ exp( α + βx )) py PY y π π y y y ( ) = ( = ) = ( ), where s 0 or. y py (,, y) = py ( ) py ( ) py ( ) = py ( ) = π ( π ) n 2 n = = n n y 5

6 We optmze the log lkelhood whch can be wrtten n [ y π π π ] log L= log + ( )log( ) e = n π = ylog + log( π ) = π Techncal note : It turns out that the maxmum lkelhood estmates of the regresson parameters β satsfy a T set of normal equatons ( ) ˆ T XWXβ= Xzwhere W s a dagonal weght matrx and z represents a vector of observatons on an adjusted dependent varable that can be derved from a Taylor seres lnear approxmaton of the lnk functon. Ths s solved by an teratve procedure called ether Iteratvely Reweghted Least Squares (IRLS), or Iteratve Weghted Least Squares (IWLS). The approxmate (asymptotc, or large sample) covarance matrx of the estmator of the vector of regresson T parameters s cov( ˆ β) = ( XWX ) Techncal note 2: If y represents a Bnomal random varable, the number of successes n m trals, the log lkelhood s n π logel= ylog + mlog( π ) = π 6

7 Each ftted model produces a maxmzed lkelhood and the generalzed lkelhood rato test tells us that we compare a hypotheszed (null) model wth maxmzed lkelhood L 0 to a ftted full model wth maxmzed lkelhood L usng the test statstc G = 2(log L log L ) χ e e 0 q under the assumpton of the null (smaller) model where q s the dfference n the number of parameters between the full and null model. Typcally q s the number of parameters set to zero. 2 The quantty G = 2loge L s called the resdual devance for the model wth maxmzed lkelhood L. It s a generalzaton of the resdual sum of squares for a lnear model, whch leads to the generalzaton of R 2 : R G log L 2 2 = = 2 G log 0 L2 7

8 Example Wave of the Survey of Labour and Income Dynamcs The data are for marred women, years old. Data Fle: SLID women.txt Source: Statstcs Canada (publc use sample), made avalable by the York Unversty Insttute for Socal Research. Varables: workng: Labor force partcpaton (Yes, No) regon: Regon of the country (Atlantc, Quebec, Ontaro, Prares, BC). kds0004: Chldren 0 to 4 years old (Yes, No). kds0509: Chldren 5 to 9 years old (Yes, No). kds04: Chldren 0 to 4 years old (Yes, No). famlyincome: Famly after tax ncludng n $000s, excludng the woman s own ncome, f any. educaton: Number of years of schoolng. [R code] Issues wth logstc regresson: - Collnearty: same ssue as for lnear models. Here the lkelhood surface s flat so that the maxmum lkelhood estmates are not well defned. - Separablty : can t estmate coeffcents f the predctor perfectly separates the s from the 0 s n the response varable. 8

9 - - Perfect predctablty n the absence of separablty: at some level of a predctor all the responses are so that probablty of success s. - Dagnostcs (see Chap 5): o Lnearty assumpton for the lnear predctor o Over dsperson for Bnomal models: s the count of successes n m trals lke m ndependent trals all wth the same probablty of success? Or could there be varaton n probablty of success or lack of ndependence? 9

10 Other most common generalzed lnear models: - Count data modeled wth a Posson dstrbuton and log lnk o Current example from consultng: The clent s nterested n nectar avalablty for nestng hummngbrds on Robnson Crusoe Island, Chle. Her dataset conssts of counts of flowers and canopy volume of ndvdual trees from two natve and two ntroduced speces. She also has counts of hummngbrd nests and chcks from a sngle ste as the hummngbrds are qute rare and very moble. Flower counts were collected at multple stes durng the 2006 and 2007 nestng seasons, and all tree speces were not represented at all stes. The clent wants to compare flowerng patterns over tme between natve and ntroduced speces. She has hypotheszed that nestng tmes wll follow natve speces flowerng trends. Natve speces have hgher flower qualty (contan more nectar). - Polytomous or multnomal data o Multnomal logt model. Ex. Party voted for n Brtsh electon (Labour, Conservatve, Lberal Democrats) o Ordered polytomous data. Ex. Government dong too lttle, about rght, too much 0

11 - More generally, any exponental famly model : Dstrbuton Normal g( μ) Canoncal lnk = μ Posson g( μ) = log μ Bnomal g( π ) = log( π / ( π)) Gamma g( μ) = μ 2 Inverse Gaussan g( μ) = μ