Main questions Motivation: Recognition

Transcription

1 /6/9 hp:// de77e4py4q Algnen and Iage Warpng Tuesda, Oc 6 Announceens Mder s ne Tues, /3 In class Can rng one 8.5 shee of noes Handou: prevous ears ders Toda Algnen & warpng d ransforaons Forward and nverse age warpng Fng ransforaons Affne Projecve Applcaon: consrucng osacs Man quesons Movaon: Recognon T Warpng: Gven a source age and a ransforaon, wha does he ransfored oupu look lke? T Algnen: Gven wo ages wh correspondng feaures, wha s he ransforaon eween he? Fgures fro Davd owe

2 /6/9 Movaon: edcal age regsraon Movaon: Mosacs Geng he whole pcure Consuer caera: 5 35 Slde fro Brown & owe 3 Movaon: Mosacs Geng he whole pcure Consuer caera: 5 35 Huan Vson: Movaon: Mosacs Geng he whole pcure Consuer caera: 5 35 Huan Vson: Panorac Mosac up o 36 8 Slde fro Brown & owe 3 Slde fro Brown & owe 3 Warpng prole Gven a se of pons and a ransforaon, generae he warped age Paraerc (gloal) warpng Eaples of paraerc warps: ranslaon roaon aspec T(,) f(,) g(, ) affne perspecve Fgure Alosha Efros Source: Alosha Efros

3 /6/9 Paraerc (gloal) warpng T Scalng Scalng a coordnae eans ulplng each of s coponens a scalar Unfor scalng eans hs scalar s he sae for all coponens: p (,) p (, ) Transforaon T s a coordnae-changng achne: p T(p) Wha does ean ha T s gloal? Is he sae for an pon p can e descred jus a few nuers (paraeers) e s represen T as a ar: p Mp M Source: Alosha Efros Source: Alosha Efros Scalng Non-unfor scalng: dfferen scalars per coponen: Scalng Scalng operaon: Or, n ar for: a, Y.5 a scalng ar S Source: Alosha Efros Source: Alosha Efros Wha ransforaons can e represened wh a ar? D Scalng? s * s s * s Wha ransforaons can e represened wh a ar? D Mrror aou Y as? D Roae around (,)? cosθ* sn Θ* sn Θ* cosθ* D Shear? sh * sh * cosθ sn Θ sh sn Θ cosθ sh Source: Alosha Efros D Mrror over (,)? )? D Translaon? NO! Source: Alosha Efros 3

4 /6/9 4 D near Transforaons Onl lnear D ransforaons can e represened wh a ar. near ransforaons are conaons of d c a near ransforaons are conaons of Scale, Roaon, Shear, and Mrror Source: Alosha Efros Hoogeneous Coordnaes Q: How can we represen ranslaon as a 33 ar usng hoogeneous coordnaes? Source: Alosha Efros Hoogeneous Coordnaes Q: How can we represen ranslaon as a 33 ar usng hoogeneous coordnaes? A: Usng he rghos colun: g g Translaon Source: Alosha Efros Translaon Hoogeneous Coordnaes Source: Alosha Efros Basc D Transforaons Basc D ransforaons as 33 arces Translae s s Scale Θ Θ Θ Θ cos sn sn cos sh sh Roae Shear Source: Alosha Efros D Affne Transforaons Affne ransforaons are conaons of w f e d c a w Affne ransforaons are conaons of near ransforaons, and Translaons Parallel lnes rean parallel

5 /6/9 Projecve Transforaons a d w g e h Projecve ransforaons: Affne ransforaons, and Projecve warps c f w Parallel lnes do no necessarl rean parallel Toda Algnen & warpng d ransforaons Forward and nverse age warpng Fng ransforaons Affne Projecve Applcaon: consrucng osacs Iage warpng Forward warpng T(,) f(,) g(, ) T(,) f(,) g(, ) Gven a coordnae ransfor and a source age f(,), how do we copue a ransfored age g(, ) f(t(,))? Send each pel f(,) o s correspondng locaon (, ) T(,) n he second age Q: wha f pel lands eween wo pels? Slde fro Alosha Efros, CMU Slde fro Alosha Efros, CMU Forward warpng Inverse warpng T(,) f(,) g(, ) T - (,) f(,) g(, ) Send each pel f(,) o s correspondng locaon (, ) T(,) n he second age Q: wha f pel lands eween wo pels? A: dsrue color aong neghorng pels (, ) Known as splang Slde fro Alosha Efros, CMU Ge each pel g(, ) fro s correspondng locaon (,) T - (, ) n he frs age Q: wha f pel coes fro eween wo pels? Slde fro Alosha Efros, CMU 5

6 /6/9 Inverse warpng Blnear nerpolaon Saplng a f(,): T - (,) f(,) g(, ) Ge each pel g(, ) fro s correspondng locaon (,) T - (, ) n he frs age Q: wha f pel coes fro eween wo pels? A: Inerpolae color value fro neghors neares neghor, lnear >> help nerp Slde fro Alosha Efros, CMU Slde fro Alosha Efros, CMU Toda Algnen & warpng d ransforaons Forward and nverse age warpng Fng ransforaons Affne Projecve Applcaon: consrucng osacs Algnen prole We have prevousl consdered how o f a odel o age evdence e.g., a lne o edge pons, or a snake o a deforng conour In algnen, we wll f he paraeers of soe ransforaon accordng o a se of achng feaure pars ( correspondences ). T Fng an affne ransforaon Fng an affne ransforaon Assung we know he correspondences, how do we ge he ransforaon? (, ), ) ( Affne odel approaes perspecve projecon of planar ojecs. 3 4 Fgures fro Davd owe, ICCV 999 6

7 /6/9 7 An asde: eas Squares Eaple Sa we have a se of daa pons (, ), (, ), (3,3 ), ec. (e.g. person s hegh vs. wegh) We wan a nce copac forula (a lne) o predc s fro s: a We wan o fnd a and How an (, ) pars do we need? Wha f he daa s nos? a a a AB a overconsraned n B A Source: Alosha Efros Fng an affne ransforaon Assung we know he correspondences, how do we ge he ransforaon? ), ( ), ( Fng an affne ransforaon 4 3 How an aches (correspondence pars) do we need o solve for he ransforaon paraeers? Once we have solved for he paraeers, how do we copue he coordnaes of he correspondng pon for? ), ( new new Wha are he correspondences?? Copare conen n local paches, fnd es aches. e.g., sples approach: scan wh eplae, and copue SSD or correlaon eween ls of pel nenses n he pach aer n he course: how o selec regons accordng o he geoerc changes, and ore rous descrpors. Toda Algnen & warpng d ransforaons Forward and nverse age warpng Fng ransforaons Fng ransforaons Affne Projecve Applcaon: consrucng osacs Panoraas... Oan a wder angle vew conng ulple ages. age fro S. Sez

8 /6/9 How o sch ogeher a panoraa (a.k.a. osac)? Basc Procedure Take a sequence of ages fro he sae poson Roae he caera aou s opcal cener Copue ransforaon eween second age and frs Transfor he second age o overlap wh he frs Blend he wo ogeher o creae a osac (If here are ore ages, repea) Panoraas: generang snhec vews real caera snhec caera u wa, wh should hs work a all? Wha aou he 3D geoer of he scene? Wh aren we usng? Source: Seve Sez Can generae an snhec caera vew as long as has he sae cener of projecon! Source: Alosha Efros Iage reprojecon Iage reprojecon Basc queson How o relae wo ages fro he sae caera cener? how o ap a pel fro PP o PP Answer Cas a ra hrough each pel n PP Draw he pel where ha ra nersecs PP PP osac PP The osac has a naural nerpreaon n 3D The ages are reprojeced ono a coon plane The osac s fored on hs plane Mosac s a snhec wde-angle caera Source: Seve Sez Oservaon: Raher han hnkng of hs as a 3D reprojecon, hnk of as a D age warp fro one age o anoher. PP Source: Alosha Efros Iage reprojecon: Hoograph A projecve ransfor s a appng eween an wo PPs wh he sae cener of projecon recangle should ap o arrar quadrlaeral parallel lnes aren u us preserve sragh lnes called Hoograph PP (, ) Hoograph w w w, w ( ), w * * w * * w * * p H * * * p PP Source: Alosha Efros To appl a gven hoograph H Copue p Hp (regular ar ulpl) Conver p fro hoogeneous o age coordnaes w * w * w * p * * * H * * * p 8

9 /6/9 Hoograph (, ) ( ), ( ), ( ), ( n, n ) ( n, n ) To copue he hoograph gven pars of correspondng pons n he ages, we need o se up an equaon where he paraeers of H are he unknowns Solvng for hoographes p Hp w a w d e w g h c f Can se scale facor. So, here are 8 unknowns. Se up a sse of lnear equaons: Ah where vecor of unknowns h [a,,c,d,e,f,g,h] T Need a leas 8 eqs, u he ore he eer Solve for h. If overconsraned, solve usng leas-squares: n Ah >> help ldvde BOARD Recap: How o sch ogeher a panoraa (a.k.a. osac)? Iage warpng wh hoographes Basc Procedure Take a sequence of ages fro he sae poson Roae he caera aou s opcal cener Copue ransforaon (hoograph) eween second age and frs usng correspondng pons. Transfor he second age o overlap wh he frs. Blend he wo ogeher o creae a osac. (If here are ore ages, repea) Source: Seve Sez age plane n fron lack area where no pel aps o age plane elow Source: Seve Sez Iage recfcaon Analsng paerns and shapes Wha s he shape of he /w floor paern? p p The floor (enlarged) Slde fro Anono Crns Auoacall recfed floor 9

10 /6/9 Analsng paerns and shapes Analsng paerns and shapes cfcaon Auoac re Fro Marn Kep The Scence of Ar (anual reconsrucon) Wha s he (coplcaed) shape of he floor paern? Auoacall recfed floor Slde fro Anono Crns S. uc Alarpece, D. Venezano Slde fro Crns Analsng paerns and shapes Changng caera cener Does sll work? snhec PP PP Auoac recfcaon PP Fro Marn Kep, The Scence of Ar (anual reconsrucon) Slde fro Crns Source: Alosha Efros Recall: sae caera cener Or: Planar scene (or far awa) PP3 real caera snhec caera PP PP Can generae snhec caera vew as long as has he sae cener of projecon. Source: Alosha Efros PP3 s a projecon plane of oh ceners of projecon, so we are OK! Ths s how g aeral phoographs are ade Source: Alosha Efros

11 /6/9 Soe osac resuls fro Fall 8 Mng Jun Chen Mng Jun Chen Andrew Harp We Cheng Su

12 /6/9 We Cheng Su We Cheng Su We Cheng Su And uong And uong Cha Sheng Tsa

13 /6/9 Cha Sheng Tsa Cha Sheng Tsa Cha Sheng Tsa Ekapol Chuangsuwanch, CMU Fe Sung Ju Hwang 3

14 /6/9 Sung Ju Hwang Sung Ju Hwang Sung Ju Hwang Sung Ju Hwang HP Fraes coercals hp:// kkbc hp:// PEoQk Suar: algnen & warpng Wre d ransforaons as ar-vecor ulplcaon (ncludng ranslaon when we use hoogeneous coordnaes) Perfor age warpng (forward, nverse) Fng ransforaons: solve for unknown paraeers gven correspondng pons fro wo vews (affne, projecve (hoograph)). Mosacs: uses hoograph and age warpng o erge vews aken fro sae cener of projecon. 4