Logistic regression models 1/12

Transcription

1 Logstc regresson models 1/12

2 2/12 Example 1: dogs look lke ther owners? Some people beleve that dogs look lke ther owners. Is ths true? To test the above hypothess, The New York Tmes conducted a quz onlne. A group of dogs and owners are photographed by Fred Conrad. For each dog, four possble owners are gven n the quz. Please choose the owner for each dog. http: // sports/westmnster-dog-show-quz.html?_r=0

3 3/12 Example 2: breast cancer data set Consder the data set collected by Rchardson et al. (2006) as an example. The study ams to fnd genes that are assocated wth the sporadc basal-lke cancers (BLC), a dstnct class of human breast cancers. In ths example, the response varable Y s the types of the breast cancer. For nstance, we use Y = 0 to represent the non-blc type and Y = 1 to represent the BLC type. The predctors n ths example are the gene expresson data or the SNPs data. For example, we could consder the gene CSF2RA as one of the canddate gene.

4 4/12 Logstc regresson models Consder Y to be Bernoull dstrbuted response. For example, Y could be falure or success, or could be dfferent treatment groups. Assume Y Bernoull(p ) and Y s assocated wth the covarates X. Model the condtonal expectaton of Y. Recall that, n lnear models, we assume that E(Y X ) = X T β and n the non-lnear models, E(Y X ) = f (X ; β). In logstc regresson model, assume that E(Y X ) = p depends on X. Namely, E(Y X ) = p(x ) for 0 p(x ) 1.

5 5/12 Lnk functons In general, we assume that E(Y X ) = h(x T β). Here h 1 ( ) s the lnk functon, whch lnks E(Y X ) wth a lnear functon of X. Three commonly used lnk functons: logt lnk, probt lnk and complementary log-log lnk. (Logt lnk) If p = h(z) = exp(z) 1+exp(z), then h 1 (p) = log( p 1 p ). (Probt lnk)if p = Φ(z) where Φ(z) s the CDF functon of a standard normal, then h 1 (p) = Φ 1 (p). (Complementary log-log) If p = h(z) = 1 exp{ exp(η)}, then h 1 (p) = log{ log(1 p)}.

6 6/12 A logstc regresson model Response: Bernoull dstrbuted random varable Y Bernoull(p ) = 1,, n. Systematc component: η = p j=1 X jβ j. Lnk functon: h(η ) = p.

7 7/12 Estmaton of β The estmaton of β can be obtaned by the maxmum lkelhood method. The lkelhood functon for β s L(β) = n The log-lkelhood functon for β s l(β) = log L(β) = = p Y (1 p ) 1 Y. p Y log( ) + 1 p Y X T β log(1 p ) log{1 + exp(x T β)}.

8 8/12 MLE of β The MLE of β s β = arg max l(β) β where l(β) s log-lkelhood functon of β. We do not have closed form soluton of β. But l(β) s a concave functon of β, whch s relatvely easy to optmze.

9 9/12 Score functon and Hessan matrx The score functon of β s l(β) β = X Y The hessan matrx of β s l(β) β β T = = X T VX X X T X exp(x T β) 1 + exp(x T β) = exp(x T β) 1 + exp(x T X (Y p ). β) {1 exp(x T β) 1 + exp(x T β) } where V = dag{p 1 (1 p 1 ),, p n (1 p n )} and X = (X 1,, X n ) T. Here p = exp(x T β)/{1 + exp(x T β)}.

10 10/12 Extenson to Bnomal dstrbuted data Suppose we observe Bnomal dstrbuted response S Bnomal(n, p ), where n s known. We would lke to study the assocaton between the response S and some covarates X. A correspondng logstc regresson model s for = 1,, m. S Bnomal(n, p ) ( p ) log = X T β 1 + p

11 11/12 Estmaton of β We can stll apply the maxmum lkelhood method to estmate β. The lkelhood functon for β s L(β) = m ( n S The log-lkelhood functon for β s l(β) = log L(β) = C + = C + ) p S (1 p ) n S. m p S log( ) + 1 p S X T β m m n log(1 p ) n log{1 + exp(x T β)}. where C s constant that has nothng to do wth β.

12 12/12 Score functon and Hessan matrx The score functon of β s l(β) β m = X S n X exp(x T β) 1 + exp(x T β) = m X (S n p ). The hessan matrx of β s l(β) m β β T = = X T VX n X X T exp(x T β) 1 + exp(x T β) {1 exp(x T β) 1 + exp(x T β) } where V = dag{n 1 p 1 (1 p 1 ),, n m p m (1 p m )} and X = (X 1,, X m ) T. Here p = exp(x T β)/{1 + exp(x T β)}.