Logistic regression (continued)

Transcription

1 STAT562 page 138 Logstc regresso (cotued) Suppose we ow cosder more complex models to descrbe the relatoshp betwee a categorcal respose varable (Y) that takes o two (2) possble outcomes ad a set of p explaatory varables where the explaatory varables are a mxture of cotuous ad dscrete varables. For the th observato where 1,...,, let x be a vector that represets the values the explaatory varables assume...e. Let Y x 1,x,x 2,...,x p 1 f the observato has the respose of terest otherwse Lke before we wrte that PY 1 x ad the vector of parameters s, 1,..., p We model the probablty of success usg a multple logstc regresso model volvg the p explaatory varables, where x e 1 1 x 2 x 2... px p 1e 1 1 x 2 x 2... px p e x 1e x Equvaletly we ca wrte the logt or log odds as x log logtx 1 x 2 x 2... p x p 1x x To derve maxmum lkelhood estmates(m.l.e. s) for ths multple logstc regresso model, we frst cosder the lkelhood of the data whch s

2 STAT562 page 139 y 1 1y ad use the fact that x e 1 1 x 2 x 2... px p 1e 1 1 x 2 x 2... px p e x 1e x We the wrte the log lkelhood of the data as L y log 1y log1 the maxmum lkelhood equatos are as follows: L y L j x j y x j, for j 1,...,p We eed to get a estmate for ; we employ the Newto-Raphso algorthm to solve these equatos. We ca wrte the vector of frst partal dervatves as: q L, L 1,..., L p y, x y x,..., x p y x p p ad the matrx of secod partal dervatves as:

3 STAT562 page 14 H p 1 p p 1 p p 2 so that we have 1 x 1 x p 1 x 1 x 2 1 x x p 1 H x p 1 x x p 1 x 2 p 1 (RECALL: The matrx H s called the formato matrx. ) To estmate we beg wth a tal guess, say ad the perform a teratve process. At the t th step of ths process we obta estmate t1 usg t1 t H t 1 q t where q t ad H t are equvalet to vector q ad matrx H evaluated at t, the t th guess for. If the algorthm coverges at terato t T, the T s the M.L.E. for ; we ca the use t to determe estmates for x usg x e 1 x 2 x 2... px p 1e 1 x 2 x 2... px p e x 1e x NOTE:The egatve of the verse of the estmated formato matrx,.e. H 1, whe evaluated at, s the estmated asymptotc covarace matrx. for the estmated parameters.

4 STAT562 page 141 Example. The data o 189 brths were collected at Baystate Medcal Ceter, Sprgfeld, Mass. durg The dataset cotas a dcator of low fat brth weght as a respose ad several rsk factors assocated wth low brth weght. The actual brth weght s also cluded the dataset. Data descrpto: The dataset cossts of the followg 1 varables: low: dcator of brth weght less tha 2.5kg age: mother s age years lwt: mother s weght pouds at last mestrual perod race: mother s race ( whte, black, other ) smoke: smokg status durg pregacy ht: hstory of hyperteso u: presece of utere rrtablty ftv: umber of physca vsts durg the frst trmester ptl: umber of prevous premature labours bwt: brth weght grams A tal study was coducted order to detfy some of the rsk factors assocated wth gvg brth to a low weght baby (a baby weghg less tha 25 grams). Four varables whch were thought to be of mportace were age ad weght of the subject at her last mestrual perod, race, ad the umber of physca vsts durg the frst trmester of pregacy. I the sample of 189 brths, t was dscovered that 59 of these resulted a low brth weght baby. A detaled explaato of the varables ad a porto of the data are as follows: Varable Abbrevato LOW AGE LWT RACE FTV Let Descrpto Low Brth Weght (1 f brth weght less tha 25 grams; otherwse Age of the Mother Years Weght Pouds at the Last Mestrual Perod Whte1;Black2;Other3 Number of Physca Vsts Durg the Frst Trmester LOW AGE LWT RACE FTV

5 STAT562 page 142 Y 1 f the th mother gves brth to a low brth weght baby otherwse where X Age of the th mother X 2 Weght of the th mother at the last mestrual perod X 3 1 f the th mother s black; otherwse X 4 1 f the th mother s of a race other tha whte or black; otherwse X 5 Number of physca vsts by the th mother durg the frst trmerster SAS code: data brthwt; fle :/215-16/STAT562/data/wleybrth.txt ; put d low age lwt race smoke ptl ht u ftv bwt; f race2 the raced11; else raced1; f race3 the raced21; else raced2; proc logstc descedg; model lowage lwt raced1 raced2 ftv / covb; ru; From the SAS output, the maxmum lkelhood estmates are: 1.295, 1.24, 2.14, 3 1.4, 4.433,ad Wth these values we ca wrte the estmated model for : e x.14x 2 1.4x 3.433x 4.49x 5 1e x.14x 2 1.4x 3.433x 4.49x 5 To determe the overall ft of the model, we eed to test the hypothess H o : versus H a : At least oe H s ot zero We ca test ths hypothess usg a lkelhood rato Ch-square statstc. The LRT requres us to maxmze the lkelhood uder H o (.e. wthout all the varables the model ) ad also uder H o H a (.e. wth all the varables cluded ) ad to form, the rato of the frst of these to the secod of these. The we compute

6 STAT562 page 143 G 2 2 log whch s the test statstc that s asymptotcally dstrbuted as Ch-squared. The lkelhood wth all explaatory varables the model (.e.uder H o H a ) s y 1 1y ad s maxmzed for e x.14x 2 1.4x 3.433x 4.49x 5 1e x.14x 2 1.4x 3.433x 4.49x 5 Computg ths for our data, the maxmzed lkelhood wth all explaatory varables the model (.e.uder H o H a ) s y 1 1y 4.67x1 49 Now we eed to cosder the maxmzed lkelhood wthout all the explaatory varables the model.(.e. uder H o ) The sample lkelhood ca aga be wrtte as L y 1 1y but ow x e 1e Thus the log lkelhood ths case becomes but ow recall y log 1y log1 e 1e To maxmze L L, we ow oly requre a maxmum lkelhood estmator for. obta ths we compute the frst partal dervatve of L w.r.t. ad equate t to zero: Usg L y To

7 STAT562 page 144 e 1e we fd ad hece log y y e 1e y For our example, Usg ths value, the maxmzed lkelhood wth all the 189 explaatory varables removed from our model yelds y 1 1y x Formg 1.1x x1 49 we the obta or G 2 2 log 2 log 1.1x x1 49 G 2 2log1.1x1 51 log4.67x Now G 2 follows a Ch-square dstrbuto wth df 5 (.e. the umber of parameters dfferet betwee the model uder H ad uder H a ). The p-value assocated wth ths value of the test statstc (.e. PG s approxmately.335. Ths s farly strog evdece that the ull hypothess ca be rejected (.e. we caot elmate all 5 explaatory varables) ad therefore suggests that there s evdece that the model wth the 5 explaatory varables fts the data.