Lecture 3 Camera Models 2 & Camera Calibration. Lecture 3 Camera Models 2 & Camera Calibration. Projective camera

Transcription

1 Lectre Cer Models Cer Clrton rofessor Slo Srese Copttonl Vson nd Geoetry L Slo Srese Lectre - - Jn- 5 Lectre Cer Models Cer Clrton In ths lectre, e ll dscss the topc of cer clrton. We ll strt th recp of cer odels. Next, e ll forlte the cer clrton prole nd nestgte ho to estte the nknon preters. We ll see ho rdl dstorton ffects or clrton process, nd ho e cn odel t. Fnlly, e ll end th n exple clrton sesson n MLB. Recp of cer odels Cer clrton prole Cer clrton th rdl dstorton Exple Redng: [F] Chpter Geoetrc Cer Clrton [HZ] Chpter 7 Coptton of Cer Mtrx Soe sldes n ths lectre re cortesy to rofs. J. once, F-F L Slo Srese Lectre - 4- Jn- 5 rojecte cer nhole perspecte projecton Let's rest or projecte cer odel. lese refer to the lectre notes for detls. f f focl length

2 rojecte cer nhole perspecte projecton f y y c x c f focl length o, o offset note dfferent conenton.r.t. lectre C [ o, o ] x rojecte cer f Unts: k,l [pxel/] f [] Non-sre pxels α, β [pxel] f focl length o, o offset α, β non-sre pxels rojecte cer f y y c θ C [ o, o ] x c x f focl length o, o offset α, β non-sre pxels θ ske ngle

3 rojecte cer f α α cotθ o β snθ o ' K hs 5 degrees of freedo * + ' x y z * f focl length o, o offset α, β non-sre pxels θ ske ngle rojecte cer f R, j O k ' R 4 4 f focl length o, o offset α, β non-sre pxels θ ske ngle R, rotton, trnslton rojecte cer f R, j O k M K R [ ] Internl preters Externl preters f focl length o, o offset α, β non-sre pxels θ ske ngle R, rotton, trnslton

4 rojecte cer f R, j O k hs slde srzes the n eton tht descres the projecte trnsforton fro pont n the orld coordnte syste n hoogenos coordntes nto pont E n the ge pxels n Eclden coordnte systes. hs eton s lso presented n lectre. ' M K R W W W W 4 4 E ' E, M heore Fgers, 99 M K[ R ] [ K R K ] [ ] [E.; lect. ] In 99 Oler Fgers dered poerfl theore tht reltes the propertes of generc projecte trnsforton M th the propertes of the trx x s defned n E.. s defned s K R nd, nd re the frst, second nd thrd ro ectors of, respectely. hese reltonshps re srzed n the necessry nd sffcent condtons llstrted n the slde. f Exercse R, j Let s consder no the follong exercse: sppose tht the to reference systes concde RI x nd x nd the cer odel hs focl length f, zero- ske, no offset, sre pxels. Cn e rte splfed expresson for E? Notce tht the reslt s exctly ht e dered for the pnhole cer n lectre. k O à M K R ' E, K I f x z, f y z f f x y z

5 rojecte cer So fr, e he dscssed projecte cer odel. Gen pont n the orld, ts ge coordntes depends on ts depth. p r f O Q R Wek perspecte projecton When the relte scene depth s sll copred to ts dstnce fro the cer f R p R_ r O Q Q_ zo _ π We ll no descre spler odel knon s the Wek erspecte rojecton odel. In the ek perspecte ponts re: frst projected to reference plne sng orthogonl projecton nd then projected to the ge plne sng projecte trnsforton. s the slde shos, gen reference plne π t dstnce ζ fro the center of the cer, the ponts, Q nd R re frst projected to the plne π sng n orthogonl projecton; ths genertes the ponts R_, Q_ nd _. ths s elent to ssgnng the z- coordnte of the ponts, Q nd R to z o hs s resonle pproxton hen detons n depth fro the plne re sll copred to the dstnce fro the cer. Wek perspecte projecton f zo R p R_ r O Q Q_ hen, the ponts R_, Q_ nd _ re projected to the ge plne sng reglr projecte trnsforton prodcng the ponts p,, r. Notce, hoeer, tht ecse e he pproxted the depth of ech pont to ζ, the projecton hs een redced to sple constnt gnfcton. he gnfcton s el to the focl length f hch s constnt s dded y ζ hch s lso constnt. x' f ' z x y' f ' z y x' f ' z x y' f ' z y _ π Mgnfcton

6 M [ ] R K M O f zo p r Q R_ R Q π à rojecte perspecte Wek perspecte Wek perspecte projecton hs lso splfes the projecton trx M. Note tht the lst ro of M s no [ ] rtten s [ ] sng the lock trx notton oe. We do not proe ths reslt nd lee t to stdents s n exercse. We cn osere ths splfcton lgerclly n the next slde. W W W, E M M gnfcton M M W W W W, E erspecte Wek perspecte In the etons oe,,, nd denote the ros of the projecton trx. In or fll perspecte odel, s el to [ ], here s soe non- zero x ector. On the other hnd, [ ] for the ek perspecte odel. hs reslts n the denontor ter to e. s reslt, the non lnerty of the projecte trnsforton dsppers nd the ek perspecte trnsforton cts s ere gnfer. Orthogrphc ffne projecton Dstnce fro center of projecton to ge plne s nfnte x' f ' z x y' f ' z y x' x y' y Frther splfcton leds to the orthogrphc or ffne projecton odel. In ths cse, the optcl center s locted t nfnty. he projecton rys re no perpendclr to the retnl plne prllel to the optcl xs. s reslt, ths odel gnores depth ltogether. It s often sed for rchtectre nd ndstrl desgn.

7 ros nd Cons of hese Models Wek perspecte ch spler th. ccrte hen oject s sll nd dstnt. Most sefl for recognton. nhole perspecte ch ore ccrte for scenes. Used n strctre fro oton or SLM. Wek perspecte projecton nted y Wng H 6-77 nd ssstnts, t s exected efore Western perspecte s ntrodced nto Chnese rt. In the deo D. Hockney coents the propertes of the prospecte geoetry depcted n the pntng. he Kngx Eperor's Sothern Inspecton or By Wng H Wek perspecte projecton he Kngx Eperor's Sothern Inspecton or By Wng H

8 Lectre Cer Clrton In ths secton, e ll ntrodce the prole of cer clrton nd dscss ts portnce. Next, e ll look t soe coon ethods for solng the clrton prole. Recp of cer odels Cer clrton prole Cer clrton th rdl dstorton Exple Redng: [F] Chpter Geoetrc Cer Clrton [HZ] Chpter 7 Coptton of Cer Mtrx Soe sldes n ths lectre re cortesy to rofs. J. once, F-F L Slo Srese Lectre - 4- Jn- 5 rojecte cer K[ R ] M Recll tht or cer s odeled sng nternl, or ntrnsc, preters K nd externl, or extrnsc, preters [R ]. her prodct ges s the fll x4 projecton trx, shon oe n the expnded for. It hs degrees of freedo e dscssed tht n lectre. Internl preters Externl preters 4 K α α cotθ β snθ o o r R r r t x t y t z Gol of clrton Estte ntrnsc nd extrnsc preters fro or ltple ges We no pose the follong prole: gen one or ore ges tken y cer, estte ts ntrnsc nd extrnsc preters. Notce tht fro ths pont on e chnge the notton for hch no s denoted s, nd hch s no denoted s p. K[ R ] M 4 K α α cotθ β snθ o o r R r r t x t y t z Chnge notton: p

9 Clrton role Clrton rg We cn descre ths prole ore precsely sng clrton rg, sch s the one shon oe. he rg slly conssts of pttern slly checkerord th knon densons. he rg lso estlshes or orld reference coordnte fre [O,,j,k ] s shon n the fgre. j C n th knon postons n [O,,j,k ] Clrton role No, sppose e re gen n ponts on the clrton rg, n, long th ther correspondng coordntes, p, p n n the ge. Usng these correspondences, or gol s to estte oth ntrnsc nd extrnsc preters. Clrton rg ge p j C n th knon postons n [O,,j,k ] p, p n knon postons n the ge Gol: copte ntrnsc nd extrnsc preters Clrton role Clrton rg ge Ho ny of sch correspondences old e need to copte oth ntrnscs nd extrnscs? Or projecton trx hs degrees of freedo 5 for the ntrnscs nd 6 for the extrnscs, so e need t lest etons. Ech correspondence ges s to etons one for x nd y ech. herefore, e old need t lest 6 correspondences. p j C Ho ny correspondences do e need? M hs nknon We need etons 6 correspondences old do t

10 Clrton role ge j C Clrton rg In prctce, sng ore thn 6 correspondences enles ore rost reslts p s lys, or correspondences re not perfect nd ssceptle to nose. Hng ore thn the nl ner of correspondences llos s to e rost to these precsons. Clrton role j C M p M n pxels [E. ] In the next sldes, e ll set p lner syste of etons fro n of sch correspondences nd propose procedre for solng t gnst the nknons ntrnsc nd extrnsc preters. Let s consder the th correspondence pr defned y pont n the clrton rg n the orld reference syste nd ts oserton p n the ge. nd re the coordntes of p esred n pxels nd n Eclden coordntes; the reltonshp eteen nd nd s expressed y E. s lso llstrted n the follong slde. Clrton role [E. ] [Es. ] Fro E., e cn dere pr of etons tht relte th nd th the syste of Es.

11 Clrton role ssng tht e he n of sch correspondences, e cn set p syste of n etons s llstrted oe [Es. ]. n n n n n n [Es. ] Block Mtrx Mltplcton B B B B B Before e oe forrd, let s refly dscss the concept of lock trx ltplcton fro trx theory. For concseness nd effcency, lrge trces cn e prttoned nto sller locks or strces. Gen to sch lock trces, e cn defne ther prodct n ters of ther strces ssng tht the prttonng s sch tht the prodct s fesle s shon. hs y of hndlng trces s ery copct nd helps s ke the ensng derton eser to descre. Wht s B? B B B + + B B B B + + B B Clrton role + + n n n + n n + n knon nknon [E. 4] Hoogenos lner syste Retrnng to the cer clrton prole, the n etons otned fro the correspondences consttte hoogeneos lner syste [Es. ]. herefore, e cn express t n ters of trx eton of the for [E. 4]. coprses the knon coeffcents fro or correspondences hle coprses the nknon preters e sh to estte. Usng lock trx notton, e cn concsely rte don nd s shon oe. Note tht s ectorzed erson of the x4 projecton trx, here ech 4x lock corresponds to ro n the orgnl trx. x4 def n n n n n n ' n x def 4x x

12 M Hoogeneos M x N Lner Systes N Mner of etons n Nner of nknon Rectnglr syste M>N s lys solton o fnd non-zero solton Mnze nder the constrnt When n>, or hoogeneos lner syste s oerdeterned. For sch syste, zero s lys trl solton. Frtherore, f s solton, then so s k, here k s n rtrry sclng fctor. herefore, to constrn or optzton, e nze sject to the constrnt tht see lner lger ree sesson for detls. he oxes rond, nd ge ltte llstrton of the densonlty of the trx ssocted to ech of the rles. he dshed sdes llstrte stton here MN. Clrton role Recll tht for hoogeneos lner syste sch s, the SVD ges s the lest- sres solton, sject to. Ho do e sole ths hoogenos lner syste? V SVD decoposton Clrton role We cn otn the lest- sres solton for y fctorzng to UDV sng SVD nd then tkng the lst coln of V. he derton s for hy ths s tre goes eyond the scope of ths lectre. lese refer to Sec. 5. HZ g. 59, 59 for detls. Once s copted, e cn repckge t nto M nd copte ll the cer preters s e ll see next; SVD decoposton of U n D V Lst coln of V ges Why? See pg 59 of HZ def M

13 Extrctng cer preters Once e he estted the coned preters n, the next step s to extrct the ctl ntrnscs nd extrnscs fro t. M ρ Reeer, tht M s kno p scle. We explctly dentfy the scle preter s ρ. Notce tht ρ t s not rel nknon. he constrnt tht llos to estte ths ddtonl nknon s gen y the fct tht st e or, elently, the Froens nor of M. M ρ Box Extrctng cer preters Estted les ± ρ cosθ Intrnsc o ρ o ρ K[ R ] α α cotθ o β K o snθ It s reltely strghtforrd to dere ll the ntrnscs nd extrnscs fro the x4 trx M hch e estted n the preos sldes. We refer to [F], Sec..., for the detls or lee t s n exercse. o od oercoplctng the notton, e rene M/ρ s [ ], nd prode solton for the ntrnscs nd extrnscs s fncton of the eleents of nd s defned n the ox n the slde. solton for the scle ρ, offset, ske re reported n the slde. heore Fgers, 99 Interestngly, the eton tht prodes solton for the ske ngle thet, spples n esy y for erfyng the second cl of the Fgers theore tht e ntrodced erler.

14 Extrctng cer preters [ ] K R Estted les Intrnsc θ ρ α sn θ ρ β sn ρ Here e he solton for lph nd et nd, ths, for the focl length. Extrctng cer preters [ ] K R Estted les Extrnsc r r ± r r r K ρ ρ Here e he soltons for the extrnscs R nd. Degenerte cses s cnnot le on the se plne onts cnnot le on the ntersecton cre of to drc srfces

15 Lectre Cer Clrton We ll no dscss the ore coplex scenro here rel orld lenses ntrodce rdl dstorton. We ll nlyze fe coon types of rdl dstortons nd ncorporte the n or odel. Recp of projecte cers Cer clrton prole Cer clrton th rdl dstorton Exple Redng: [F] Chpter Geoetrc Cer Clrton [HZ] Chpter 7 Coptton of Cer Mtrx Soe sldes n ths lectre re cortesy to rofs. J. once, F-F L Slo Srese Lectre - 4- Jn- 5 Rdl Dstorton Ige gnfcton ndecreses th dstnce fro the optcl xs Csed y perfect lenses Detons re ost notcele for rys tht pss throgh the edge of the lens So fr, e he een orkng th del lenses hch re free fro ny dstorton. Hoeer, rel lenses cn dete fro rectlner projecton. he resltng dstortons re often rdlly syetrc, hch cn e ttrted to the physcl syetry of the lens. he ge oe exhts rrel dstorton. hs occrs hen the ge gnfcton decreses th dstnce fro the optcl xs. Fsheye lenses slly prodce ths type of dstorton. No dstorton n cshon Brrel d Rdl Dstorton Ige gnfcton decreses th dstnce fro the optcl center S λ λ M p λ he rdl dstorton cn e odeled sng n sotropc trnsforton S λ s shon n the slde. hs trnsforton s reglted y the dstorton fctor λ. Notce tht nlke trdtonl scle trnsforton of the type [s ; s ; ], here s s constnt, λ s fncton of the dstnce d fro the center snce the dstorton s rdlly syetrc ot the center nd ths fncton of the coordntes,. hs cses the dstorton trnsforton to ntrodce non- lnerty to the ppng fro to p We pproxte λ sng polynol expnson E. 5. he resltng coeffcents, κ p, re knon s the dstorton coeffcents. In order to odel the rdl ehor, the dstnce cn e expressed s drtc fncton of the coordntes, n the ge plne E. 6. Dstorton coeffcent λ ± κ d p p d + + c p [E. 5] o odel rdl ehor [E. 6] olynol fncton

16 p Q p M λ λ Q Is ths lner syste of etons? Rdl Dstorton No hy? [Es.7] We cn rerte or projecton etons y sng the lock trx notton nd defnng Q s S λ M. We follo the se procedre tht led to E. nd Es. see preos sldes. Unlke for the syste defned y Es., hoeer, the syste of etons Es. 7 s no longer lner one. X f Q esreents preters f s the nonlner ppng -Neton Method -Leenerg-Mrrdt lgorth Iterte, strts fro ntl solton My e slo f ntl solton fr fro rel solton Estted solton y e fncton of the ntl solton Neton reres the coptton of J, H Leenerg-Mrrdt doesn t rere the coptton of H Generl Clrton role n [E.8] If n correspondences re lle, ll these constrnts cn e pckged nto non- lner lt- rle fncton f hch reltes ll the nknon preters Q th the osertons/esreents [E.8]. coon y to sole ths s to resort to non- lner optzton technes. o coon ones nclde the Neton s ethod nd the Leenerg- Mrrdt lgorth. he slde lsts soe of the dntges nd trde- offs of Leenerg- Mrrdt. J nd H refer to the Jcon nd Hessn, respectely. For ore detls ot optzton ethods, plese refer to [F] Sec.. pge or ny reference text ooks. possle lgorth. Sole lner prt of the syste to fnd pproxted solton. Use ths solton s ntl condton for the fll syste. Sole fll syste sng Neton or L.M. Generl Clrton role X f Q n esreents preters [E.8] f s the nonlner ppng possle sple lgorth tht people se n prctce s: - sole the syste sng n correspondences s e dd for E. 4 y ssng tht there s no dstorton. - Use ths solton s ntl condton for solng the prole n E Sole the fll prole n E. 8 sng Neton or L.M.

17 ypcl ssptons: - zero-ske, sre pxel - o, o knon center of the ge Generl Clrton role X f Q esreent preter f s nonlner We cn splfy the clrton prole f e ke certn ssptons. Soe typcl ssptons nclde zero ske nd sre pxels hch s resonle for ny odern cers, knon ge center, nd neglgle dstorton. Under these ssptons, the densonlty of the prole s redced nd the optzton prole s splfed. Cn e estte nd nd gnore the rdl dstorton? Rdl Dstorton p λ d Hnt: slope n lternte nd ery elegnt pproch, tht doesn t rere the se of non- lner optzton, follos next. We note tht the rto eteen to coordntes nd of the pont p n the ge s not ffected y the dstorton reeer the rrel dstorton cts rdlly. Esttng nd Rdl Dstorton p λ n n n n L n n s [87] Get nd y SVD [E.9] [E.] [E.] hs, let s copte ths rto of the ech p. hs leds s to E.9. By ssng tht n correspondences re lle, e cn set p the syste n E.. Slrly to the derton e sed to copte E. 4, e cn otn the hoogenos syste n E.. hs cn e soled sng SVD nd llos solton for nd.

18 Rdl Dstorton Once tht nd re estted Once tht nd re estted cn e expressed non lner fncton of nd nd ld. hs stll reres to sole n non- optzton prole hose coplexty, hoeer, s ery ch splfed. p λ s non lner fncton of,, λ here re soe degenerte confgrtons for hch nd cnnot e copted Lectre Cer Clrton We ll no go oer n exple of cer clrton sng MLB. Recp of projecte cers Cer clrton prole Cer clrton th rdl dstorton Exple Redng: [F] Chpter Geoetrc Cer Clrton [HZ] Chpter 7 Coptton of Cer Mtrx Soe sldes n ths lectre re cortesy to rofs. J. once, F-F L Slo Srese Lectre - 4- Jn- 5 Clrton rocedre Cer Clrton oolox for Mtl J. Boget [998-]

19 Clrton rocedre Clrton rocedre Clrton rocedre

20 Clrton rocedre Clrton rocedre Clrton rocedre

21 Clrton rocedre Next lectre Sngle e reconstrcton