Generalized Linear Models. Statistical Models. Classical Linear Regression Why easy formulation if complicated formulation exists?

Size: px

Start display at page:

Download "Generalized Linear Models. Statistical Models. Classical Linear Regression Why easy formulation if complicated formulation exists?"

Bertha Waters
5 years ago
Views:

Statstcal Models Geeralzed Lear Models Classcal lear regresso complcated formlato of smple model, strctral ad radom compoet of the model Lectre 5 Geeralzed Lear Models Geeralzed lear models geeral

1 Statstcal Models Geeralzed Lear Models Classcal lear regresso complcated formlato of smple model, strctral ad radom compoet of the model Lectre 5 Geeralzed Lear Models Geeralzed lear models geeral descrpto ad examples Parameter estmato maxmm lelhood method, comptatoal sses Statstcal ferece goodess of ft, aalss of devace Wpeda I statstcs, the geeralzed lear model GLM s a sefl geeralzato of ordar least sqares regresso. It relates the radom dstrbto of the measred varable of the expermet the dstrbto fcto to the sstematc o-radom porto of the expermet the lear predctor throgh a fcto called the l fcto. he sbject of geeralzed lear models was formlated b Joh Nelder ad Robert Wedderbr as a wa of fg varos other statstcal models der oe framewor, allowg for oe geeral method of effcetl performg maxmm lelhood estmato for these models. Classcal Lear Regresso Wh eas formlato f complcated formlato exsts? Respose varable Y has a ormal dstrbto Expected vale EY of Y depeds o explaator varables Y ~ N, σ η = x = radom compoet ag more geeral dstrbto ad a more geeral l fcto g stcg to ths lear form sstematc compoet, lear predctor l fcto, lg ad Geeralzed Lear Model Geeralzed Lear Models Kphoss, medcal cotext Kphoss s a deformato that ca occr wth chldre that derwet correctve spal srger. Qesto: gve the age of the chld at the tme of srger, what s the probablt of occrrece of Kphoss? > lbrarrpart > dataphoss > phoss Kphoss Age Nmber Start > phoss Kphoss Age Nmber Start M M M M M > parmfrow=c, > plotkphoss~age, + data=phoss > plotkphoss~nmber, + data=phoss > plotkphoss~start, + data=phoss Kphoss Data vsalsato Kphoss Kphoss Kphoss Age Nmber Start

2 Notato: Y x Ital ave model: Y Y, j Kphoss 0 Y ~ Beroll,,.e., Y = depedso x some wa j towards a model dcator of presece of Kphoss meas, 0 ot age of chld at tme of srger w.p. w.p. Notato: η = x = log Kphoss Geeralzed lear model Y dcator of presece of Kphoss meas, 0 ot x age of chld at tme of srger does ot deped o x learl, bt a olear fcto g l fcto of depeds o x learl dep 0 Y ~ Beroll,,.e., Y = radom compoet dep exp x Y ~ Ber, + exp x w.p. - w.p. sstematc compoet, lear predctor l fcto logt Logstc Regresso model = log Logstc Regresso dep 0 Y ~ Beroll,,.e., Y = radom compoet w.p. - w.p. η = x sstematc compoet, lear predctor Beroll l fcto x Estmate parameter vector based o the avalable data estmated model specfed dep exp x Y ~ Ber x, + exp Geeralzed Lear Models > d.ad treatmet otcome cots > parmfrow=c, > plotcots~treatmet,d.ad > plotcots~otcome,d.ad cots cots ext example treatmet otcome Notato: Y dep ~ = log Cots η = x sstematc compoet, lear predctor Geeralzed lear model Y measremet o cots, assmed to be Posso x vector of explaator varables for expermet does ot deped o learl, bt a olear fcto x g l fcto of depeds o learl Posso, radom compoet l fcto x Y η = x dep ~ = log Posso mea Log-lear Regresso Posso, 0 4 Estmate parameter vector based o the avalable data estmated model specfed x exp x, How to estmate the logstc ad log-lear regresso model? Y dep ~ Posso Y exp x, dep ~ Posso Log-lear Regresso model Frst: geeral geeralzed lear model ML estmato GLM s

3 Y ~ f Geeralzed lear model Geeral strctre ad examples, a probablt dest fcto wth η = x, lear predctor EY = ; see below, wth g a geeral, mootoc l fcto Here f s the probablt dest of a oe-dmesoal expoetal faml dstrbto f = exp +, c φ A φ A φ dsperso parameter atral parameter Scale parameter, statstcal problems ow Loos complcated, bt.. Geeralzed lear model f = exp +, c φ A φ A σ exp π σ = examples of expoetal faml exp log σ σ πσ exp = exp log log!! = exp log + log + log Note: expoetal faml the caocal l fcto s the fcto mappg the mea to the atral parameter. I other words: a GLM wth expoetal faml dest f ad caocal l, the atral parameter s modeled as lear fcto of the parameter vector!! Expoetal famles momets f = exp +, c φ A φ A Here otato for cotos radom varable; dscrete case follows completel aalogosl f d = 0 reglart codtos b' f d = 0 E Y = b' φ A Smlarl: 0 reglart codtos f d = b' b'' φ f d = 0 Var Y = b'' φ A φ A A Expoetal famles momets f = exp +, c φ A φ A φ E Y = b', Var Y = b'' A Normal dstrbto: b =, φ A = σ Posso dstrbto: Beroll dstrbto: E Y = b' = =, Var Y = σ b = e, φ A = E Y = b' = e =, Var Y = e = b = log + e, φ A = e E Y = b' = =, Var Y = + e e + e = Maxmm lelhood estmato geeral geeralzed lear model η η Model for Y f ; = exp + c, φ A η = x φ A, where Data: x,, x,, K, x, Log lelhood: η η l = log f ; = + c, φ A η = x φ A where = = Maxmze l as fcto of maxmm lelhood estmate ˆ Isaac Newto Joseph Raphso Maxmze f = e

4 Maxmze f = e Maxmze f = e Maxmze f = e Maxmze f = e Maxmze f = e Maxmze the followg fcto over the real le: f = e Iteratve procedre: Step: = 0, choose Repeat step ad tl stoppg crtero s satsfed Step : q = f + f ' + + f ' Step : = argmax q = f '' f ''

5 0 4 5 f = e f ' = + e f '' = 6 + e f f ' f '' Maxmzg the log lelhood η η l = log f ; = + c, φ A η = x φ A where = = Maxmze l as fcto of maxmm lelhood estmate ˆ Newto Raphso choose tal gess approxmate l b local secod-order correct qadratc approxmato l l + l + l [ l ] l l 0 + [ l ] l maxmze ths qadratc approxmato 0 = Newto-Raphso, caocal l 0 [ l 0 ] l 0 Comptg the MLE R > phoss Kphoss Kphoss Age Nmber Start M M M M M > Kph.glm<-glmKphoss~.,faml=bomal,data=phoss > smmarkph.glm Coeffcets: Estmate part of otpt Std. Error z vale Pr> z Itercept Age Nmber Start ** --- Sgf. codes: 0 `***' 0.00 `**' 0.0 `*' 0.05 `.' 0. ` ' Nmber of Fsher Scorg teratos: 5 Remember prevosl show pctre Secod vsalzato Kphoss Kphoss Kphoss Kphoss Age Nmber Start Smaller model ad predcto Kphoss > KphStart.glm<- + glmkphoss~start, faml=bomal,data=phoss > smmarkphstart.glm part of otpt Coeffcets: Estmate Std. Error z vale Pr> z Itercept Start *** > x<-phoss$start > <-as.tegerphoss$k=="" > mx<-maxx; x<-:mx; relfreq<-0*x > for j :mx{ relfreq[j]<-mea[x==j]} > plotx,relfreq > xas<-seq,mx,legth=00 > etaas<-coefkphstart.glm[]+coefkphstart.glm[]*xas > predas<-expetaas/+expetaas > lesxas,predas,col="red" relfreq Predctos based o the MLE Kphoss exp ˆ ˆ 0 + x xa + exp ˆ + x ˆ Relatve freqeces of occrrece of phoss satrated model ad predcto of freqec based o MLE GLM x 5

6 Geeralzed Lear Models smmar Classcal lear regresso Vew as led radom- ad determstc part Examples of geeralzed lear regresso Logstc regresso model Kphoss Log-lear model Cots Geeral geeralzed lear models Expoetal famles Maxmm lelhood estmato GLM Newto-Raphso Fttg a GLM R 6

Generalized Linear Models. Statistical Models. Classical Linear Regression Why easy formulation if complicated formulation exists?

Generalized Linear Models. Statistical Models. Classical Linear Regression Why easy formulation if complicated formulation exists? Statstcal Models Geeralzed Lear Models Classcal lear regresso complcated formlato of smple model, strctral ad radom compoet of the model Lectre 5 Geeralzed Lear Models Geeralzed lear models geeral descrpto