Introduction to logistic regression

Size: px

Start display at page:

Download "Introduction to logistic regression"

Augusta Shelton
5 years ago
Views:

1 Itroducto to logstc rgrsso Gv: datast D { } whr s a k-dmsoal vctor of ral-valud faturs or attrbuts ad s a bar class labl or targt. hus w ca sa that R k ad {0 }. For ampl f k 4 a datast of 3 data pots or ampls would b: I ths datast thr ar o postv data pot 2 ad two gatv data pots 3. All vctors ar 4-dmsoal. Each of th four dmsos s calld a fatur sa blood prssur sugar lvl bo dst tc.. Each class labl dots o of th two groups.g. ma ma that patt wth faturs has a partcular dsas or trat whl 0 would th ma that patt dscrbd wth faturs dos ot hav that dsas. I summar th datast D ths ampl cossts of th 3 vctors of faturs ach vctor bg assocatd wth a corrspodg class labl. h task of our data mg sstm s to costruct a prdctor that would fr class labl for a st of faturs. For ampl f a w patt coms ad w prform four chap tsts w mght b abl to fr a dsas wthout rug vr psv tsts whch would ffctvl labl that data pot. Evr frc s assocatd wth a qualt of frc. W wll masur qualt of frc b th umbr of mstaks th prdctor maks prct ol for prvousl us data pots. Formall th qualt of frc wll b prssd through prdctor accurac. You ca s a prdctor as a smpl mathmatcal fucto. hus t vrtuall maps a k- dmsoal vctor to a zro or o. hs tp of a prdctor s calld a bar classfr; classfr bcaus ca tak dscrt valus ad bar bcaus thr ar ol two such valus that ca tak. Of cours ou ca also mag gralzatos to ths cas A cor of a statstcal or mach larg approach s to assum that vctors ad thr labls obsrvd D wr gratd b a sourc that outputs vctors ad labls accordg to som probablt dstrbuto p. I such a cas w ca prss class mmbrshp probablstcall. h basc da for logstc rgrsso approach s to tr to stablsh a smpl possbl lar closd-form dpdc mag thr s a formula btw th probablt of a class mmbrshp ad th st of faturs. O such form could b /5

2 whr R k s a vctor of k ral-valud umbrs gral ad s a traspos of th vctor. hrfor a vctor product rsults a sgl umbr. For ampl f ad th h problm wth quato s that R whl th probablt ds to b lmtd to th trval [0 ]. hus w caot us closd-form from quato. Aothr approach to modl th probablt of a class mmbrshp s to tr to prss th odds fucto as a lar combato of paramtr vctor ad fatur vctor. hat s odds fucto 2 Closr look at ths fucto do vrf ths! rvals that ths fucto s co-doma s trval [0 whl R. hrfor ths s ot a approprat paramtrc dpdc thr. Our thrd tr wll b to tak a logarthm of th odds fucto ad rprst t as a lar combato of ad. hat s log 3 I ths cas both ad logodds blog to th trval -. h logarthm th prsso abov s to th bas whr A rorgazato of th prsso 3 gvs th followg 2/5

3 3/5 4 hrfor a probablt that th class of th vctor s ca ths cas b modld accordg to th prsso 4. h fucto ft t s calld a sgmod fucto or a logstc fucto s th plot all th wa at th bottom. Dtrmg optmal coffcts For a gv datast D ad assumd dpdc from prsso 4 th optmal st of coffcts s dtrmd b mamzg th followg prsso calld lklhood fucto 5 or a formal mathmatcal otato ma arg whch sas that vctor * s th o for whch prsso 5 s mamal. Now lt s us th followg rprstato ou ca asl vrf t s tru b substtutg 0 for for

4 Now w ca mamz th followg prsso ot that f th frst part dsappars whl f th scod part dsappars whch follows from prsso 4 or aftr takg a logarthm to th bas log log I assumd hr ou ar famlar wth som basc logarthm mapulatos. hs quato s mamzd usg tratv optmzato procss. Dtrmg optmal coffcts * s calld trag. Formall ths whol optmzato procss s calld mamum lklhood optmzato ad thr ar varous toolbos o th Itrt that ca do ths for ou ma b prtt compl. h trag procss s ow ovr. W just calculatd *. How do w prdct or fr class for a w data pot? For a us data pot ad optmal or somtms arl optmal st of coffcts * dtrmd from a trag datast D w smpl calculat th followg prsso If 0. 5 w smpl coclud that th data pot should b labld as postv. O th othr had f < 0. 5 w labl th us data pot as gatv. h prdcto ca b mad v wthout calculatg th logstc prsso: f 0 w prdct postv ad othrws w prdct gatv ou ca asl vrf ths b substtutg th prsso abov 4/5

5 What s th shap of th sgmod fucto? Fal pot Although th assumpto that ma ot b tutv logstc rgrsso has b show to work surprsgl wll practc!!! 5/5

Introduction to logistic regression

Introduction to logistic regression Itroducto to logstc rgrsso Gv: datast D {... } whr s a k-dmsoal vctor of ral-valud faturs or attrbuts ad s a bar class labl or targt. hus w ca sa that R k ad {0 }. For ampl f k 4 a datast of 3 data pots