Model Fitting and Robust Regression Methods

Transcription

1 Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Model Fttng nd Robust Regresson Methods CMPE 64: Imge Anlss nd Comuter Vson H o

2 Fttng lnes nd ellses to mge dt Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Ft the best lne or ellse model to gven grou mge dt onts Ellse ttng roblem Let... be set o mge onts [ ]. Assume the mlct equton o the generc ellse dened b ts rmeter vector [ b c d e ] s where [ ] Fnd the otml rmeter whch stses nd where D s sutble dstnce resdul + b + c + d + e + mn rg [ D ]

3 Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Algebrc dstnce t - ellse he lgebrc dstnce o ont rom curve s he ellse ttng roblem becomes subject to Remrks: he constrnt mkes sure the resultnt curve s n ellse C s clled the constrnt mtr hs roblem s oten clled lest squres ttng rg mn 4 c b C

4 Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Algebrc dstnce t - ellse Wrte the dstnce n mtr orm where s the desgn mtr s the sctter mtr he ellse ttng roblem s rewrtten s subject to S X X X... X X X S S mn rg C +

5 Algebrc dstnce t - ellse Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Usng Lgrnge multlers mn λ L λ S λ C + o obtn soluton we need to solve L S λc L C + λ From the rst equton generlzed egenvlue roblem should sts S λc Substtute n mn rg S λc λ It cn be roved tht the soluton s the egenvector whch corresonds to the onl negtve egenvlue

6 Algebrc dstnce t - ellse Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Algorthm ALG_ELLIPSE_FI Buld the desgn mtr X rom the dt onts Buld the sctter mtr S s S X X Buld the constrnt mtr C Solve the generlzed egenvlue roblem S λc he outut s the egenvector corresondng to the onl negtve vlue Emle

7 Algebrc dstnce t - lne Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz he generc lne model s Fnd the soluton subject to where X + b + c mn rg... X he soluton s the egenvector corresondng to the the lest egenvlue o X X Homework: Prove ths! X

8 Algebrc dstnce t vs. Eucldn t Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz In generl the lgebrc dstnce s derent rom the Eucldn dstnce. In ddton two onts wth the sme lgebrc dstnce to the curve m hve derent Eucldn dstnces to the curve. In most comuter vson lgorthms Eucldn dstnce s more desrble but usng lgebrc dstnce oten mkes lgorthms smler Eucldn dstnce. j Eucldn dstnce.5

9 Algebrc dstnce t wth Eucldn t Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Snce weghts round the lt rts o the ellse re lrger the Eucldn errors t lt re re smller thn the errors n the onted rt. As consequence the t tends to be tter.

10 Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Eucldn t - ellse he Eucldn dstnce ttng roblem becomes subject to Soluton Usng Lgrnge multlers he soluton should sts or rg mn mn L λ λ λ L λ λ

11 Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Eucldn t - ellse I we romte b the revous equton cn be rewrtten s I we romte the curve s we obtn thereore Substtutng n or + λ + λ λ 4

12 Eucldn t - ellse Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz he orgnl objectve uncton becomes mn rg ow the objectve uncton onl deends on. hs cn be solved usng grdent descent methods Remrks: he Eucldn dstnce s romted s the lgebrc dstnce dvded b the mgntude o the grdent o mke ths lgorthm work good ntl s needed. hs cn be obtned usng ttng lgorthm tht uses the lgebrc dstnce Both lgorthms re lest squre methods even smll number o outlers cn degrde the result bdl

13 Robust lgorthms Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz

14 Robust lgorthms Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz hree crter re oten used or evlutng robust lgorthms Meer Mntz Roseneld nd Km 99 Reltve ecenc the rto between the lowest chevble vrnce or the estmted rmeters nd the ctul vrnce rovded b the method Brekdown ont the smllest mount o outler contmnton tht m orce the vlue o the estmte outsde n rbtrr rnge. Emle: the brekdown ont o the men s snce sngle lrge outler cn corrut the result. me comlet comuttonl comlet. Fesble lgorthms should be t most O

15 Robust lgorthms Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Soluton chnge the dstnce mesure so tht t s less senstve to lrge errors: usng bsolute vlues nsted o squres Algorthm ROB_ELLIPSE_FI Run ALG_ELLIPSE_FI to obtn the ntl soluton Usng s the ntl soluton usng grdent descent method to nd the otml rmeters usng the ollowng objectve uncton mn rg

16 Robust lgorthms -results Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz

17 Robust lgorthms LMedS method Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz he lest-medn-o-squres method Rousseeuw 984 Brekdown ont s 5% Problem sttement- we wnt to estmte coecent or Soluton mn rg med [ D z ] Solve β j j... Fnd the mode o z β s β j β j j j j j β j

18 Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz Robust lgorthms RASAC RAdom SAmle Consensus Fschler nd Bolles 98 Algorthm Comute model b solvng sstem o equtons dened or rndoml chosen subset o dt onts All the dt s then clssed reltve to ths model. he onts wthn some error tolernce re clled the consensus set o the model I crdnlt o the consensus set eceeds threshold the model s cceted nd ts rmeters recomuted bsed on the whole consensus set I the model s not cceted new set o onts s chosen nd resultng model s tested or vldt he error tolernce nd the consensus set ccetnce threshold must be set ror O the lrgest consensus set wthn ed number o trls re used Brekdown ont s 5%

19 Homework Dertment o Comuter Engneerng Unverst o Clorn t Snt Cruz he generc lne model s Prove tht the soluton subject to where X + b + c mn rg... X s the egenvector corresondng to the the smllest egenvlue o X X X