Main questions Motivation: Recognition

Transcription

1 Tod Algnen & Wrpng Thursd, Oc 9 Algnen & wrpng d rnsforons Forwrd nd nverse ge wrpng Consrucng oscs Hoogrphes Rous fng wh RANSAC Krsen Grun UT-Ausn Mn quesons Movon: Recognon T Wrpng: Gven source ge nd rnsforon, wh does he rnsfored oupu look lke? T Algnen: Gven wo ges wh correspondng feures, wh s he rnsforon eween he? Fgures fro Dvd Lowe Movon: edcl ge regsron Movon: Moscs Geng he whole pcure Consuer cer: 5 35 Slde fro Brown & Lowe 3

2 Movon: Moscs Geng he whole pcure Consuer cer: 5 35 Hun Vson: Movon: Moscs Geng he whole pcure Consuer cer: 5 35 Hun Vson: Pnorc Mosc up o 36 8 Slde fro Brown & Lowe 3 Slde fro Brown & Lowe 3 Wrpng prole Gven se of pons nd rnsforon, genere he wrped ge Prerc (glol) wrpng Eples of prerc wrps: rnslon roon spec T(,) f(,) g(, ) ffne perspecve Fgure Alosh Efros Source: Alosh Efros Prerc (glol) wrpng T Sclng Sclng coordne ens ulplng ech of s coponens sclr Unfor sclng ens hs sclr s he se for ll coponens: p (,) p (, ) Trnsforon T s coordne-chngng chne: p T(p) Wh does en h T s glol? Is he se for n pon p cn e descred jus few nuers (preers) Le s represen T s r: p Mp M Source: Alosh Efros Source: Alosh Efros

3 Non-unfor sclng: dfferen sclrs per coponen: Sclng, Y.5 Source: Alosh Efros Sclng Sclng operon: Or, n r for: sclng r S Source: Alosh Efros Wh rnsforons cn e represened wh r? D Roe round (,)? cos sn sn cos cos sn sn cos D Sher? sh sh sh sh Source: Alosh Efros D Sclng? s s s s Wh rnsforons cn e represened wh r? Source: Alosh Efros D Mrror ou Y s? D Mrror over (,)? D Trnslon? NO! D Lner Trnsforons Onl lner D rnsforons cn e represened wh r. Lner rnsforons re conons of Scle, Roon, Sher, nd Mrror d c Source: Alosh Efros Hoogeneous Coordnes Q: How cn we represen rnslon s 33 r usng hoogeneous coordnes? Source: Alosh Efros

4 Hoogeneous Coordnes Q: How cn we represen rnslon s 33 r usng hoogeneous coordnes? A: Usng he rghos colun: Trnslon Source: Alosh Efros Trnslon Hoogeneous Coordnes Source: Alosh Efros Bsc D Trnsforons Bsc D rnsforons s 33 rces cos sn sn cos sh sh Trnsle Roe Sher s s Scle Source: Alosh Efros D Affne Trnsforons Affne rnsforons re conons of Lner rnsforons, nd Trnslons Prllel lnes ren prllel w f e d c w Projecve Trnsforons Projecve rnsforons: Affne rnsforons, nd Projecve wrps Prllel lnes do no necessrl ren prllel w h g f e d c w Tod Algnen & wrpng d rnsforons Forwrd nd nverse ge wrpng Consrucng oscs Hoogrphes Rous fng wh RANSAC

5 Ige wrpng Forwrd wrpng T(,) f(,) g(, ) T(,) f(,) g(, ) Gven coordne rnsfor nd source ge f(,), how do we copue rnsfored ge g(, ) f(t(,))? Send ech pel f(,) o s correspondng locon (, ) T(,) n he second ge Q: wh f pel lnds eween wo pels? Slde fro Alosh Efros, CMU Slde fro Alosh Efros, CMU Forwrd wrpng Inverse wrpng T(,) f(,) g(, ) T - (,) f(,) g(, ) Send ech pel f(,) o s correspondng locon (, ) T(,) n he second ge Q: wh f pel lnds eween wo pels? A: dsrue color ong neghorng pels (, ) Known s splng Slde fro Alosh Efros, CMU Ge ech pel g(, ) fro s correspondng locon (,) T - (, ) n he frs ge Q: wh f pel coes fro eween wo pels? Slde fro Alosh Efros, CMU Inverse wrpng Blner nerpolon Splng f(,): T - (,) f(,) g(, ) Ge ech pel g(, ) fro s correspondng locon (,) T - (, ) n he frs ge Q: wh f pel coes fro eween wo pels? A: Inerpole color vlue fro neghors neres neghor, lner >> help nerp Slde fro Alosh Efros, CMU Slde fro Alosh Efros, CMU

6 Algnen prole We hve prevousl consdered how o f odel o ge evdence e.g., lne o edge pons, or snke o deforng conour In lgnen, we wll f he preers of soe rnsforon ccordng o se of chng feure prs ( correspondences ). T Ige lgnen Two rod pproches: Drec (pel-sed) lgnen Serch for lgnen where os pels gree Feure-sed lgnen Serch for lgnen where erced feures gree Cn e verfed usng pel-sed lgnen Source: L. Lzenk Fng n ffne rnsforon Fgures fro Dvd Lowe, ICCV 999 Affne odel pproes perspecve projecon of plnr ojecs. Fng n ffne rnsforon Assung we know he correspondences, how do we ge he rnsforon? ), ( ), ( 4 3 An sde: Les Squres Eple S we hve se of d pons (, ), (, ), (3,3 ), ec. (e.g. person s hegh vs. wegh) We wn nce copc forul ( lne) o predc s fro s: We wn o fnd nd How n (, ) prs do we need? Wh f he d s nos? AB overconsrned n B A Source: Alosh Efros Fng n ffne rnsforon Assung we know he correspondences, how do we ge he rnsforon? ), ( ), ( 4 3 L L L L 4 3

7 Fng n ffne rnsforon L L L 3 4 L How n ches (correspondence prs) do we need o solve for he rnsforon preers? Once we hve solved for he preers, how do we copue he coordnes of he correspondng pon for, new )? ( new Wh re he correspondences? Copre conen n locl pches, fnd es ches. e.g., sples pproch: scn wh eple, nd copue SSD or correlon eween ls of pel nenses n he pch Ler n he course: how o selec regons ccordng o he geoerc chnges, nd ore rous descrpors.? Pnors On wder ngle vew conng ulple ges.... ge fro S. Sez How o sch ogeher pnor? Bsc Procedure Tke sequence of ges fro he se poson Roe he cer ou s opcl cener Copue rnsforon eween second ge nd frs Trnsfor he second ge o overlp wh he frs Blend he wo ogeher o cree osc (If here re ore ges, repe) u w, wh should hs work ll? Wh ou he 3D geoer of he scene? Wh ren we usng? Source: Seve Sez Pnors: generng snhec vews Ige reprojecon rel cer snhec cer Cn genere n snhec cer vew s long s hs he se cener of projecon! Source: Alosh Efros osc PP The osc hs nurl nerpreon n 3D The ges re reprojeced ono coon plne The osc s fored on hs plne Mosc s snhec wde-ngle cer Source: Seve Sez

8 Hoogrph How o rele wo ges fro he se cer cener? how o p pel fro PP o PP? Thnk of s D ge wrp fro one ge o noher. A projecve rnsfor s ppng eween n wo PPs wh he se cener of projecon recngle should p o rrr qudrlerl prllel lnes ren PP u us preserve srgh lnes clled Hoogrph (, ) Hoogrph w w w, w ( ), w w w p H p PP Source: Alosh Efros To ppl gven hoogrph H Copue p Hp (regulr r ulpl) Conver p fro hoogeneous o ge coordnes w w w p H p Hoogrph (, ) ( ), ( ), ( ), ( n, n ) ( n, n ) To copue he hoogrph gven prs of correspondng pons n he ges, we need o se up n equon where he preers of H re he unknowns Solvng for hoogrphes Cn se scle fcor. So, here re 8 unknowns. Se up sse of lner equons: Ah where vecor of unknowns h [,,c,d,e,f,g,h] T Need les 8 eqs, u he ore he eer Solve for h. If overconsrned, solve usng les-squres: n Ah >> help ldvde p Hp w w d e w g h c f BOARD Recp: How o sch ogeher pnor? Bsc Procedure Tke sequence of ges fro he se poson Roe he cer ou s opcl cener Copue rnsforon eween second ge nd frs Trnsfor he second ge o overlp wh he frs Blend he wo ogeher o cree osc (If here re ore ges, repe) Ige wrpng wh hoogrphes Source: Seve Sez ge plne n fron lck re where no pel ps o ge plne elow Source: Seve Sez

9 Ige recfcon Anlsng perns nd shpes Wh s he shpe of he /w floor pern? p p Hoogrph Slde fro Crns The floor (enlrged) Auocll recfed floor Anlsng perns nd shpes Anlsng perns nd shpes Auoc recfcon Fro Mrn Kep The Scence of Ar (nul reconsrucon) Wh s he (coplced) shpe of he floor pern? Auocll recfed floor Slde fro Crns S. Luc Alrpece, D. Venezno Slde fro Crns Anlsng perns nd shpes chngng cer cener Does sll work? snhec PP PP Auoc recfcon PP Fro Mrn Kep, The Scence of Ar (nul reconsrucon) Slde fro Crns Source: Alosh Efros

10 Plnr scene (or fr w) PP PP3 PP PP3 s projecon plne of oh ceners of projecon, so we re OK! Ths s how g erl phoogrphs re de Source: Alosh Efros Tod Algnen & wrpng d rnsforons Forwrd nd nverse ge wrpng Consrucng oscs Hoogrphes Rous fng wh RANSAC Oulers Oulers cn hur he qul of our preer eses, e.g., n erroneous pr of chng pons fro wo ges n edge pon h s nose, or doesn elong o he lne we re fng. Eple: les squres lne fng Assung ll he pons h elong o prculr lne re known

11 Oulers ffec les squres f Oulers ffec les squres f RANSAC RANdo Sple Consensus Approch: we wn o vod he pc of oulers, so le s look for nlers, nd use hose onl. Inuon: f n ouler s chosen o copue he curren f, hen he resulng lne won hve uch suppor fro res of he pons. RANSAC RANSAC loop:. Rndol selec seed group of pons on whch o se rnsforon ese (e.g., group of ches). Copue rnsforon fro seed group 3. Fnd nlers o hs rnsforon 4. If he nuer of nlers s suffcenl lrge, re-copue les-squres ese of rnsforon on ll of he nlers Keep he rnsforon wh he lrges nuer of nlers RANSAC Lne Fng Eple RANSAC Lne Fng Eple Tsk: Ese es lne Sple wo pons Slde cred: Jnng Ch, CMU

12 RANSAC Lne Fng Eple RANSAC Lne Fng Eple F Lne Tol nuer of pons whn hreshold of lne. RANSAC Lne Fng Eple RANSAC Lne Fng Eple Repe, unl ge good resul Repe, unl ge good resul RANSAC Lne Fng Eple RANSAC eple: Trnslon Repe, unl ge good resul Puve ches Source: Rck Szelsk

13 RANSAC eple: Trnslon RANSAC eple: Trnslon Selec one ch, coun nlers Selec one ch, coun nlers RANSAC eple: Trnslon Feure-sed lgnen oulne Fnd verge rnslon vecor Source: L. Lzenk Feure-sed lgnen oulne Feure-sed lgnen oulne Erc feures Erc feures Copue puve ches Source: L. Lzenk Source: L. Lzenk

14 Feure-sed lgnen oulne Feure-sed lgnen oulne Erc feures Copue puve ches Loop: Hpohesze rnsforon T (sll group of puve ches h re reled T) Source: L. Lzenk Erc feures Copue puve ches Loop: Hpohesze rnsforon T (sll group of puve ches h re reled T) Verf rnsforon (serch for oher ches conssen wh T) Source: L. Lzenk Feure-sed lgnen oulne Tod Erc feures Copue puve ches Loop: Hpohesze rnsforon T (sll group of puve ches h re reled T) Verf rnsforon (serch for oher ches conssen wh T) Algnen & wrpng d rnsforons Forwrd nd nverse ge wrpng Consrucng oscs Hoogrphes Rous fng wh RANSAC Mder on Tuesd n clss Source: L. Lzenk