Fitting a transformation: Feature based alignment May 1 st, 2018

Transcription

1 5//8 Fng a ransforaon: Feaure based algnen Ma s, 8 Yong Jae Lee UC Davs Las e: Deforable conours a.k.a. acve conours, snakes Gven: nal conour (odel) near desred objec Goal: evolve he conour o f eac objec boundar Man dea: elasc band s eravel adjused so as o be near age posons wh hgh gradens, and sasf shape preferences or conour prors [Snakes: Acve conour odels, Kass, Wkn, & Terzopoulos, ICCV987] Fgure cred: Yur Bokov Las e: Deforable conours 3 Iage fro hp://

2 5//8 Las e: Deforable conours Pros: Useful o rack and f non-rgd shapes Conour reans conneced Possble o fll n subjecve conours Flebl n how energ funcon s defned, weghed. Cons: Mus have decen nalzaon near rue boundar, a ge suck n local nu Paraeers of energ funcon us be se well based on pror nforaon 4 Krsen Grauan Toda Ineracve segenaon Feaure-based algnen D ransforaons Affne f RANSAC 5 Ineracve forces How can we pleen such an neracve force wh deforable conours? 6 Krsen Grauan

3 5//8 Ineracve forces An energ funcon can be alered onlne based on user npu use he cursor o push or pull he nal snake awa fro a pon. Modf eernal energ er o nclude a er such ha n r E push p Nearb pons ge pushed hardes Adaped b Dev Parkh fro Krsen Grauan 7 Beond boundar snappng Anoher for of neracve gudance: specf regons Usuall aken o sugges foreground/background color dsrbuons User Inpu Resul Bokov and Joll () How o use hs nforaon? 8 Recall: Iages as graphs q p w pq w Full-conneced graph node for ever pel lnk beween ever par of pels, p,q slar w pq for each lnk» slar s nversel proporonal o dfference n color and poson 9 Seve Sez 3

4 5//8 Recall: Segenaon b Graph Cus w A B C Break graph no segens Delee lnks ha cross beween segens Eases o break lnks ha have low slar slar pels should be n he sae segens dsslar pels should be n dfferen segens Seve Sez Recall: Segenaon b Graph Cus A Lnk Cu se of lnks whose reoval akes a graph dsconneced cos of a cu: cu( A, B) Fnd nu cu gves ou a segenaon fas algorhs es for dong hs B w p, q pa, qb Source: Seve Sez Graph cus for neracve segenaon Addng hard consrans: Add wo addonal nodes, objec and background ernals Lnk each pel To boh ernals To s neghborng pels Yur Bokov 4

5 5//8 Graph cus for neracve segenaon Addng hard consrans: Le he edge wegh o objec or background ernal reflec dsance o he respecve seed pels. 3 Yur Bokov Graph cus for neracve segenaon Bokov and Joll () 4 Graph cus for neracve segenaon Anoher neracon odal: specf boundng bo Slde cred: Krsen Grauan Graph Cus Bokov and Joll () GrabCu Roher e al. (4) 5 5

6 5//8 Grab Cu Loosel specf foreground regon Ieraed graph cu User nalzaon Inalsaon? K-eans for learnng colour dsrbuons Roher e al (4) Graph cus o nfer he segenaon 6 Grab Cu Loosel specf foreground regon Ieraed graph cu R Foreground & Background Ieraed graph cu R Foreground Background G Background G Gaussan Mure Model (pcall 5-8 coponens) Roher e al (4) 7 Grab Cu Roher e al (4) 8 6

7 5//8 Toda Ineracve segenaon Feaure-based algnen D ransforaons Affne f RANSAC 9 Movaon: Recognon Fgures fro Davd Lowe Movaon: edcal age regsraon Slde cred: Krsen Grauan 7

8 5//8 Movaon: osacs (In deal ne week) Iage fro hp://graphcs.cs.cu.edu/courses/5-463/_fall/ Algnen proble We have prevousl consdered how o f a odel o age evdence e.g., a lne o edge pons, or a snake o a deforng conour In algnen, we wll f he paraeers of soe ransforaon accordng o a se of achng feaure pars ( correspondences ). T Slde cred: Krsen Grauan 3 Paraerc (global) warpng Eaples of paraerc warps: ranslaon roaon aspec affne perspecve 4 Source: Alosha Efros 8

9 5//8 Paraerc (global) warpng p = (,) T p = (, ) Transforaon T s a coordnae-changng achne: p = T(p) Wha does ean ha T s global? Is he sae for an pon p can be descrbed b jus a few nubers (paraeers) Le s represen T as a ar: p = Mp M 5 Source: Alosha Efros Scalng Scalng a coordnae eans ulplng each of s coponens b a scalar Unfor scalng eans hs scalar s he sae for all coponens: 6 Source: Alosha Efros Scalng Non-unfor scalng: dfferen scalars per coponen:, Y.5 7 Source: Alosha Efros 9

10 5//8 Scalng Scalng operaon: Or, n ar for: b a b a scalng ar S Source: Alosha Efros 8 Wha ransforaons can be represened wh a ar? D Roae around (,)? * cos * sn * sn * cos cos sn sn cos D Shear? sh sh * * sh sh Source: Alosha Efros D Scalng? s s * * s s 9 Wha ransforaons can be represened wh a ar? Source: Alosha Efros D Mrror abou Y as? D Mrror over (,)? D Translaon? NO! 3

11 5//8 D Lnear Transforaons a c b d Onl lnear D ransforaons can be represened wh a ar. Lnear ransforaons are cobnaons of Scale, Roaon, Shear, and Mrror 3 Source: Alosha Efros Hoogeneous coordnaes Convenen coordnae sse o represen an useful ransforaons To conver o hoogeneous coordnaes: hoogeneous age coordnaes Converng fro hoogeneous coordnaes: Slde cred: Krsen Grauan 3 Hoogeneous Coordnaes Q: How can we represen d ranslaon as a 33 ar usng hoogeneous coordnaes? A: Usng he rghos colun: Translaon 33 Source: Alosha Efros

12 5//8 Translaon = = Hoogeneous Coordnaes Source: Alosha Efros 34 Basc D Transforaons Basc D ransforaons as 33 arces cos sn sn cos sh sh Translae Roae Shear s s Scale Source: Alosha Efros 35 D Affne Transforaons Affne ransforaons are cobnaons of Lnear ransforaons, and Translaons Parallel lnes rean parallel w f e d c b a w 36 Slde cred: Krsen Grauan

13 5//8 Toda Ineracve segenaon Feaure-based algnen D ransforaons Affne f RANSAC 37 Algnen proble We have prevousl consdered how o f a odel o age evdence e.g., a lne o edge pons, or a snake o a deforng conour In algnen, we wll f he paraeers of soe ransforaon accordng o a se of achng feaure pars ( correspondences ). T 38 Krsen Grauan Iage algnen Two broad approaches: Drec (pel-based) algnen Search for algnen where os pels agree Feaure-based algnen Search for algnen where eraced feaures agree Can be verfed usng pel-based algnen 39 Slde cred: Krsen Grauan 3

14 5//8 4 Fng an affne ransforaon Assung we know he correspondences, how do we ge he ransforaon? ), ( ), ( Slde cred: Krsen Grauan An asde: Leas Squares Eaple Sa we have a se of daa pons (, ), (, ), (3,3 ), ec. (e.g. person s hegh vs. wegh) We wan a nce copac forula (a lne) o predc s fro s: a + b = We wan o fnd a and b How an (, ) pars do we need? Wha f he daa s nos? b a b a b a A=B b a overconsraned n B A Source: Alosha Efros 4 Fng an affne ransforaon Assung we know he correspondences, how do we ge he ransforaon? ), ( ), ( Slde cred: Krsen Grauan

15 5//8 5 Fng an affne ransforaon How an aches (correspondence pars) do we need o solve for he ransforaon paraeers? 4 3 Krsen Grauan 43 Affne: # correspondences? How an correspondences needed for affne? T(,)? Alosha Efros Fng an affne ransforaon How an aches (correspondence pars) do we need o solve for he ransforaon paraeers? Once we have solved for he paraeers, how do we copue he coordnaes of he correspondng pon for? Where do he aches coe fro? 4 3 ), ( new new Krsen Grauan 45

16 5//8 Wha are he correspondences?? Copare conen n local paches, fnd bes aches. e.g., sples approach: scan wh eplae, and copue SSD or correlaon beween ls of pel nenses n he pach Laer n he course: how o selec regons usng ore robus descrpors. 46 Krsen Grauan Fng an affne ransforaon Fgures fro Davd Lowe, ICCV Fng an affne ransforaon 48 6

17 5//8 Fng an affne ransforaon 49 Fng an affne ransforaon 5 Fgures fro Davd Lowe, ICCV 999 Toda Ineracve segenaon Feaure-based algnen D ransforaons Affne f RANSAC 5 7

18 5//8 Oulers Oulers can hur he qual of our paraeer esaes, e.g., an erroneous par of achng pons fro wo ages an edge pon ha s nose, or doesn belong o he lne we are fng. 5 Krsen Grauan Oulers affec leas squares f Slde cred: Krsen Grauan 53 Oulers affec leas squares f Slde cred: Krsen Grauan 54 8

19 5//8 RANSAC RANdo Saple Consensus Approach: we wan o avod he pac of oulers, so le s look for nlers, and use hose onl. Inuon: f an ouler s chosen o copue he curren f, hen he resulng lne won have uch suppor fro res of he pons. Slde cred: Krsen Grauan 55 RANSAC: General for RANSAC loop:. Randol selec a seed group of pons on whch o base ransforaon esae. Copue ransforaon fro seed group 3. Fnd nlers o hs ransforaon 4. If he nuber of nlers s suffcenl large, re-copue esae of ransforaon on all of he nlers Keep he ransforaon wh he larges nuber of nlers Slde cred: Krsen Grauan 56 RANSAC for lne fng eaple Source: R. Ragura 57 Lana Lazebnk 9

20 5//8 RANSAC for lne fng eaple Leas squares f Source: R. Ragura 58 Lana Lazebnk RANSAC for lne fng eaple. Randol selec nal subse of pons Source: R. Ragura 59 Lana Lazebnk RANSAC for lne fng eaple. Randol selec nal subse of pons. Hpohesze a odel Source: R. Ragura 6 Lana Lazebnk

21 5//8 RANSAC for lne fng eaple. Randol selec nal subse of pons. Hpohesze a odel 3. Copue error funcon Source: R. Ragura 6 Lana Lazebnk RANSAC for lne fng eaple. Randol selec nal subse of pons. Hpohesze a odel 3. Copue error funcon 4. Selec pons conssen wh odel Source: R. Ragura 6 Lana Lazebnk RANSAC for lne fng eaple Source: R. Ragura. Randol selec nal subse of pons. Hpohesze a odel 3. Copue error funcon 4. Selec pons conssen wh odel 5. Repea hpohesze andverf loop 63 Lana Lazebnk

22 5//8 RANSAC for lne fng eaple Source: R. Ragura. Randol selec nal subse of pons. Hpohesze a odel 3. Copue error funcon 4. Selec pons conssen wh odel 5. Repea hpohesze andverf loop 64 Lana Lazebnk RANSAC for lne fng eaple Source: R. Ragura. Randol selec nal subse of pons. Hpohesze a odel 3. Copue error funcon 4. Selec pons conssen wh odel 5. Repea hpohesze andverf loop 65 Lana Lazebnk RANSAC for lne fng eaple Source: R. Ragura. Randol selec nal subse of pons. Hpohesze a odel 3. Copue error funcon 4. Selec pons conssen wh odel 5. Repea hpohesze andverf loop 66 Lana Lazebnk

23 5//8 RANSAC for lne fng Repea N es: Draw s pons unforl a rando F lne o hese s pons Fnd nlers o hs lne aong he reanng pons (.e., pons whose dsance fro he lne s less han ) If here are d or ore nlers, accep he lne and ref usng all nlers 67 Lana Lazebnk RANSAC pros and cons Pros Sple and general Applcable o an dfferen probles Ofen works well n pracce Cons Los of paraeers o une Doesn work well for low nler raos (oo an eraons, or can fal copleel) Can alwas ge a good nalzaon of he odel based on he nu nuber of saples 68 Lana Lazebnk Toda Ineracve segenaon Feaure-based algnen D ransforaons Affne f RANSAC 69 3

24 5//8 Cong up: algnen and age schng 7 Quesons? See ou Thursda! 7 4