STK3100 and STK4100 Autumn 2017

Transcription

1 SK3 ad SK4 Autum 7 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Sectos 4..5, 4.3.5, 4.3.6, 4.4., 4.4., ad Sectos 5.., 5.., ad 5.5. Ørulf Borga Deartmet of Mathematcs Uversty of Oslo Delta method Assume that we have a estmator ξ for a scalar arameterξ Let h be dfferetable fucto wth hξ ( By a aylor easo we have whe ξ s close to It follows that h( ξ h( ξ + h ( ξ( ξ ξ var[ h( ξ] [ h ( ξ] var( ξ Assume more geerally we have a estmator for a vector-valued arameter ξ ( ξ,..., ξ ξ ξ (,..., Assume that h : R R for,..., q ad cosder hξ ( [ h ( ξ,..., h ( ξ ] ξ ξ By aylor easos we have whe ξ s close to h ( h ( h ( ξ ( ξ ξ ξ ξ + k k k ξk hs gves hξ ( hξ ( hξ ( + ( ξ ξ ξ hξ ( where s the q wth (, k elemet ξ It follows that ( ( var[ ( ] hξ var( hξ hξ ξ ξ ξ ξ h ( ξ ξ k 3 Aromate (covaraces for GLMs We cosder a GLM where Y, Y,..., Y are deedet wth µ for a lear redctor η β ad a lk E( Y fucto gµ ( η he we have { } ~ N[,( β β X WX ], aromately where X s the model matr, ad W s a dagoal matr wth elemets µ w var( Y η hus we have var( ( X WX For the vector of estmated lear redctors η Xβ we have var( η X var( X X( X WX X 4

2 he vector of ftted values μ ( µ,..., µ s gve by µ η g (,..., Let D be the dagoal matr wth dagoal elemets he µ η η µ ( g ( µ var( μ D var( η D g [ g ( η ] DX( X WX X D Combed wth aromately ormalty, ths may be used to fd cofdece tervals for the µ 's But t s better to trasform cofdece tervals for the η 's 5 Cofdece tervals by vertg tests Cosder a model wth a sgle arameter β We may obta a ( α% cofdece terval for β by vertg a two-sded α-level test of H : β β he cofdece terval s the set of all that are ot reected he Wald test of H : β β reects the ull hyothess whe β SE( > z α/ he corresodg cofdece terval for β s ± z SE( α/ β 6 he score test of H : β β wll ofte reect whe β SE ( > z α/ where SE ( s the stadard error of β whe β β he ( α% cofdece terval s gve as the set of all that satsfy β β SE ( z α/ Here SE( wll deed o β ad that comlcates the equaltes 7 he lkelhood rato test of H : β β reects f [ L( β L( β] > χ ( α where Lβ ( s the log-lkelhood A cofdece terval s gve as the set of all β that satsfy [ L( β L( β] χ ( α Assume the that we have a model that also cotas other arameters ψ he Wald tye cofdece tervals are as above A alteratve s to base the cofdece o the rofle lkelhood he terval s gve as the set of all β that satsfy [ L( β, ψ ( β L( β, ψ ] χ ( α where ( β s the ML estmator of ψ whe ψ β β 8

3 Iterretato of the β's logstc regresso Let Y be a Beroull varable wth P( Y π( for a (row vector of covarates (,..., We assume a logstc regresso model e( β e π( + e( β + e he the odds takes the form ( β ( β For the values * ( *,..., * ad (,..., of the covarates, the odds rato becomes odds( * e( β * odds( e( β e{( β} If * + ad *k k for k the odds( * e β odds( * π( odds( π( e( β + e( β + e( β e( β 9 hus e β s the effect of oe uts crease the -th covarate whe the other covarates rema the same If π( s small (as the study of rare dseases, the π( odds( π( π( ad odds( * π( * odds( π( so the odds rato s aromately equal to the relatve rsk I geeral we have π( e( β + e( β e( β β [ ] + e( β e( β β + e( β + e( β βπ( [ π( ] Cofdece tervals for logstc regresso A cofdece terval [ L, U ] for β may be obtaed by vertg the Wald test or the lkelhood rato test (usg the rofle lkelhood A cofdece terval for the odds rato e β s the gve by [e( L, e( U ] We may also be terested a cofdece terval of for a gve value of the covarates π( We the frst cosder the estmated lear redctor η ( β We have η( ~ N[ η(, ( X WX ], aromately

4 A Wald ( α% cofdece terval for η( s gve by η( ± ( X WX z α / Deote ths terval by [ L(, U ( ] he a ( α% cofdece terval for s gve by e{ η( } π( + e{ η( } e{ L( } e{ U ( }, + e{ L( } + e{ U ( } 3 Devace ad sum of squares For the ormal lear model we have that Y, Y,..., Y are deedet wth commo varace σ ad meas µ β β Deote the observed values by y, y,..., y ad the ftted values by µ For the ormal lear regresso model, the sum of squares SSE ( y µ s key quatty coecto wth hyothess testg ad for assessg model ft 4 We wat to defe a quatty for GLMs that corresods to the sum of squares for ormal lear models o ths ed we frst cosder the relato betwee the log-lkelhood ad the sum of squares for the ormal model (whe we cosder σ as a fed quatty For the ormal lear regresso model, the log-lkelhood takes the form L( μ; y log( π log σ ( y µ σ he log-lkelhood obtas ts largest value for the saturated model,.e. the model where there are o restrctos o the ɶ he mamum s obtaed for µ y ad the mamum value s L( y; y log( π logσ µ 5 For a gve secfcato of the lear regresso model, the µ are estmated by µ ad the log-lkelhood takes the value L( μ ; y log( π log σ ( y µ σ he we obta ma lkelhood for actual model log ma lkelhood for saturated model [ L( μ ; y L( y; y] ( y µ σ 6

5 Now cosder a geeral GLM ad remember the coecto betwee the arameters: b θ g µ g η β For a gve secfcato of the GLM, the are estmated by µ wth corresodg estmates θ of the atural arameters θ For a the saturated GLM, the are estmated by µ ɶ y wth corresodg estmates θɶ of the atural arameters θ µ µ 7 We wll assume that a( φ φ / w, where the w 's are kow, ad cosder φ as fed he log-lkelhood s gve by yθ b( θ L( μ; y + c( y, φ φ / w We the obta ma lkelhood for actual model log ma lkelhood for saturated model [ L( μ ; y L( y; y] w ( / ɶ ( ɶ yθ bθ φ w yθ bθ / φ w ( ɶ ( ɶ ( yθ θ bθ bθ + / φ 8 We troduce ad have D( yμ ; w ( ɶ ( ɶ ( yθ θ bθ + bθ [ L( μ ; y L( y; y] D( yμ ; / φ Here D( yμ ; s called the devace, ad s called the scaled devace For the lear ormal model we have D( yμ ; ( y µ D( yμ ; / φ so the devace s a geeralzato of sum of squares, ad the smaller the devace the better the model ft 9 Eamle: Bomal GLM Assume that Y, Y,..., YN are deedet, ad that Y V /, where V ~ b(, π Here we have µ π ad θ log[ π / ( π ] θ θ π e / ( + e b( θ log( π log( + e θ a( φ / herefore ɶ θ log[ y / ( y ] b( θ log( y θ log[ π / ( π ] bθ ( log( π ɶ

6 he devace ( scaled devace s gve by D( yμ ; w ( ɶ ( ɶ ( yθ θ bθ + bθ y π y log log log( y log( π + y π y y y y log log log + π π π y y log ( log + y y π π Alteratvely oe may fd the devace by drectly cosderg (cf. age 8 the book: ma lkelhood for actual model log ma lkelhood for saturated model For bomal data we may cosder two tyes of large samle results For groued data we have N fed, ad all the 's are large he the devace D( Yμ ; s aromately ch-squared dstrbuted wth df N, where dm( β hs s called small dserso asymtotcs the book For ugroued data we have all ad the total umber of observatos N s large he we do ot have a aromate ch-squared dstrbuto of the devace A data fle wth groued data may be coverted to ugroued form. he two formats wll gve the same estmates ad stadard errors, but ot the same devace Devace ad the lkelhood rato test We cosder a GLM wth a( φ / w (e.g. Posso or bomal, ad assume that model holds M Let model be ested model We wat to test the ull hyothess that model holds M M he lkelhood rato statstc s gve by ma M l( μ; y l( ; Λ μ y ma Ml( μ; y l( μ, y herefore G ( M M logλ [ L( μ ; y L( μ ; y] M D( yμ ; D( yμ ; whch s aromately ch-squared dstrbuted wth df degrees of freedom, where ad are the umber of arameters for models ad M M