STK3100 and STK4100 Autumn 2018

Transcription

1 SK3 ad SK4 Autum 8 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Cofdece tervals by vertg tests Cosder a model wth a sgle arameter β We may obta a ( α% cofdece terval for β by vertg a two-sded α-level test of H : β = β he cofdece terval s the set of all that are ot reected β Sectos 4..5, 4.3.5, 4.3.6, 4.4., 4.4., 4.4.3, ad Sectos 5.., 5.5., ad 5.5. Ørulf Borga Deartmet of Mathematcs Uversty of Oslo he Wald test of H : β = β reects the ull hyothess whe β SE( > z he corresodg cofdece terval for β s ± z SE( he score test of H : β= β wll ofte reect whe β SE ( > z where SE ( s the stadard error of β whe β= β he ( α% cofdece terval s gve as the set of all that satsfy β β SE ( z Here SE( wll deed o β ad that comlcates the equaltes 3 he lkelhood rato test of H : β= β reects f [ L( β L( β] > χ ( α where Lβ ( s the log-lkelhood A cofdece terval s gve as the set of all β that satsfy [ L( β L( β] χ ( α Assume the that we have a model that also cotas other arameters ψ he Wald tye cofdece tervals are as above A alteratve s to base the cofdece o the rofle lkelhood he terval s gve as the set of all β that satsfy [ L( β, ψ ( β L( β, ψ ] χ ( α where ( β s the ML estmator of ψ whe ψ β= β 4

2 Cofdece tervals for logstc regresso A cofdece terval [ L, U ] for may be obtaed by vertg the Wald test or the lkelhood rato test (usg the rofle lkelhood A cofdece terval for the odds rato e β s the gve by [e( L, e( U ] We may also be terested a cofdece terval of π e( ( = β + e( β β for a gve value of the covarates We the frst derve a cofdece terval for the lear redctor η ( = β hs may be estmated by η ( = β 5 For logstc regresso we have that ~ N[,( β β X WX ], aromately where X= s the model matr ad W s the dagoal matr wth dagoal elemets It follows that { } w = π ( π η( ~ N[ η(, ( X WX ], aromately A Wald ( α% cofdece terval for η( s gve by η( ± ( X WX z α / Deote ths terval by [ L(, U ( ] he a ( α% cofdece terval for π( s gve by e{ L( } e{ U ( }, + e{ L( } + e{ U ( } 6 Delta method Assume that we have a estmator for a scalar arameter Let h be dfferetable fucto wth h ( By a aylor easo we have whe s close to By aylor easos we have whe s close to h ( h ( h ( ( + k k k= k It follows that h( h( + h ( ( var[ h( ] [ h ( ] var( Assume more geerally we have a estmator for a vector-valued arameter = (,..., = (,..., hs gves h ( h ( h ( + ( h ( where s the q wth (, k elemet It follows that h ( k Assume that h : R R for =,...,q ad cosder h ( = [ (,..., ( ] h h q 7 ( ( var[ ( ] h var( h h 8

3 Aromate (covaraces for GLMs We cosder a GLM where Y, Y,..., Y are deedet wth µ for a lear redctor η = β ad a lk = = E( Y fucto gµ ( = η he we have { } ~ N[,( β β X WX ], aromately where X= s the model matr, ad W s a dagoal matr wth elemets µ w = var( Y η hus we have var( ( X WX For the vector of estmated lear redctors η = Xβ we have var( η = X var( X X( X WX X 9 he vector of ftted values μ= ( µ,..., µ s gve by µ = η = g (,..., Let D be the dagoal matr wth dagoal elemets he µ η = η µ ( = g ( µ = var( μ = D var( η D g [ g ( η ] DX( X WX X D Combed wth aromately ormalty, ths may be used to fd cofdece tervals for the µ 's But t s better to trasform cofdece tervals for the η 's Devace ad sum of squares For the ormal lear model we have that Y, Y,..., Y are deedet wth commo varace σ ad meas µ = β = β = Deote the observed values by y, y,..., y ad the ftted values by µ = = = For the ormal lear regresso model, the sum of squares SSE = ( y µ = s key quatty coecto wth hyothess testg ad for assessg model ft We wat to defe a quatty for GLMs that corresods to the sum of squares for ormal lear models o ths ed we frst cosder the relato betwee the log-lkelhood ad the sum of squares for the ormal model (whe we cosder σ as a fed quatty For the ormal lear regresso model, the log-lkelhood takes the form L( μ; y = log( π log σ ( y µ σ = he log-lkelhood obtas ts largest value for the saturated model,.e. the model where there are o restrctos o the ɶ he mamum s obtaed for µ = y ad the mamum value s L( y; y = log( π logσ µ

4 For a gve secfcato of the lear regresso model, the µ are estmated by µ ad the log-lkelhood takes the value L( μ ; y = log( π log σ ( y µ σ = he we obta ma lkelhood for actual model log ma lkelhood for saturated model =[ L( μ ; y L( y; y] = ( y µ σ = Now cosder a geeral GLM ad remember the coecto betwee the arameters: b θ g µ g = η β For a gve secfcato of the GLM, the are estmated by µ wth corresodg estmates θ of the atural arameters θ For a the saturated GLM, the are estmated by µ ɶ = y wth corresodg estmates θɶ of the atural arameters θ ( b µ µ 3 4 We wll assume that a( φ = φ / w, where the w 's are kow, ad cosder φ as fed he the log-lkelhood s gve by yθ b( θ L( μ; y = + c( y, φ = φ / w ad we obta ma lkelhood for actual model log ma lkelhood for saturated model =[ L( μ ; y L( y; y] = w ( / ɶ ( ɶ yθ bθ φ w yθ bθ / φ = = = w ( ɶ ( ɶ ( yθ θ bθ bθ + / φ = We troduce ad have D( yμ ; = w ( ɶ ( ɶ ( yθ θ bθ + bθ = [ L( μ ; y L( y; y] = D( yμ ; / φ Here D( yμ ; s called the devace, ad s called the scaled devace For the lear ormal model we have D( yμ ; = ( y µ D( yμ ; / φ 5 6 = so the devace s a geeralzato of sum of squares, ad the smaller the devace the better the model ft

5 Eamle: Bomal GLM Assume that Y, Y,..., YN are deedet, ad that Y = V /, where V ~ b(, π Here the devace (=scaled devace s gve by ma lkelhood for actual model D( yμ ; =log ma lkelhood for saturated model N y y π ( π y = =log N y y y ( y y = y y = log ( log + y y π = = π 7 For bomal data we may cosder two tyes of large samle results For groued data we have N fed, ad all the 's are large he the devace D( Yμ ; s aromately ch-squared dstrbuted wth df = N, where = dm( β hs s called small dserso asymtotcs the book For ugroued data we have all = ad the total umber of observatos N s large he we do ot have a aromate ch-squared dstrbuto of the devace A data fle wth groued data may be coverted to ugroued form. he two formats wll gve the same estmates ad stadard errors, but ot the same devace 8 Devace ad the lkelhood rato test We cosder a GLM wth a( φ = / w (e.g. Posso or bomal, ad assume that model holds M Let model be ested model We wat to test the ull hyothess that model holds M M he lkelhood rato statstc s gve by ma M l( μ; y l( ; Λ= = μ y ma Ml( μ; y l( μ, y herefore G ( M M =logλ =[ L( μ ; y L( μ ; y] M = D( yμ ; D( yμ ; whch s aromately ch-squared dstrbuted wth df = degrees of freedom, where ad are the umber of arameters for models ad M M Score test ad Pearso ch-squared statstc Cosder a GLM wth aφ ( = ad var( Y = vµ ( Oe may show that the score test statstc for comarg a gve model wth the saturated model s gve by the Pearso statstc ( Y µ X = v( µ For stuatos where the umber of terms s fed ad the Y 's are aromately ormal, we have that X s aromately ch-squared dstrbuted For comarg to ested models ad M oe may use the Pearso statstc (whch s ot the score statstc ( µ µ ( = v( µ X M M = M whch s aromately ch-squared wth df =

6 Eamle: Bomal GLM Assume that Y, Y,..., YN are deedet, ad that Y = V /, where V ~ b(, π For groued data wth N fed, ad all the 's large, we have N N ( Y µ ( Y π X = = v( µ π ( π / = = N ( Y π π ( π = = N ( Y π [( Y ( π ] = + π ( π = N = = (observed ftted ftted whch s aromately ch-squared dstrbuted wth df = N