CAMERA GEOMETRY. Ajit Rajwade, CS 763, Winter 2017, IITB, CSE department

Transcription

1 CAMERA GEOMETRY Aji Rajwade, CS 763, Wine 7, IITB, CSE depamen

2 Conens Tansfomaions of poins/vecos in D Tansfomaions of poins/vecos in 3D Image Fomaion: geome of pojecion of 3D poins ono D image plane Vanishing Poins Camea Calibaion

3 Tansfomaions of poins and vecos in D

4 Tanslaion and Roaion Le us denoe he coodinaes of he oiginal poin as (, ) and hose of he ansfomed poin as as (, ). Tanslaion: Roaion abou poin (,) ani-clockwise hough angle q cos θ sin θ cos θ sin θ sin θ cos θ sin θ cos θ D Roaion mai (ohonomal mai and wih deeminan )

5 Tanslaion and oaion Roaion abou poin ( c, c ) ani-clockwise hough angle q c c c c c c θ ) ( θ ) ( θ ) ( θ ) ( cos sin sin cos -Pefom anslaion such ha ( c, c ) coincides wih he oigin. -Roae abou he new oigin. -Tanslae back. Noe: we ae inoducing a hid (ficiious) coodinae even hough he poins ae in D. This enables anslaion opeaions o be epessed in he mai muliplicaion famewok. This new coodinae is called as he homogeneous coodinae. cos sin sin cos cos sin sin cos θ θ θ θ θ θ θ θ c c c c c c c c

6 Tanslaion and oaion Roaion and anslaion: Tanslaions and oaions ae igid ansfomaions he peseve angles, lenghs and aeas. cos sin sin cos cos sin sin cos θ θ θ θ θ θ θ θ

7 Tanslaions/oaion o Affine Tansfomaions Affine ansfomaion: oaion, anslaion, scaling and sheaing A A A A Assumpion: he sub-mai A is NOT ank deficien, ohewise i will ansfom wo-dimensional figues ino a line o a poin. The mai A in geneal is NOT ohonomal. If i is ohonomal, his educes o eihe a oaion (deeminan ) o eflecion (deeminan = -)

8 Affine Tansfomaions Scaling If c = - and c =, hen he scaling opeaion is called eflecion acoss he Y ais. If c = and c = -, hen i is called eflecion acoss X ais. Reflecions can occu acoss an abia ais.

9 Affine ansfomaions Sheaing: k k k k ' ; ' ' ' ', ' ' ' hp://cs.fi.edu/~wds/classes/cse555/hesis/shea/shea.hml k= k=

10 Affine ansfomaion The D affine ansfomaion model (including anslaion in X and Y diecion) includes 6 degees of feedom. The affine ansfomaions geneall do no peseve lenghs, angles and aeas. Bu he peseve collineai of poins, i.e. poins on one line emain on a line afe affine ansfomaion. The peseve aios of lenghs of collinea line segmens and aios of aeas. In fac aea of ansfomed objec = A aea of oiginal objec, whee A is he ansfomaion mai.

11 Composiion of affine ansfomaions Composiion of muliple pes of affine ansfomaions is given b he muliplicaion of hei coesponding maices. Eample: eflecion acoss an abia diecion hough (,). (,) Line L ϴ (, )

12 Composiion of affine ansfomaions Appl a oaion R such ha line L coincides wih eihe he X ais (o he Y ais). Appl a eflecion R abou he X ais (o he Y ais). Appl a evese oaion R =R - such ha he oiginal oienaion of L is esoed. ' ' cosq sinq sinq cosq cosq sinq sinq cosq R R R

13 Composiion of affine ansfomaions How will ou change his if he diecion L didn pass hough he oigin? (,) Line L ϴ (, )

14 Composiion of affine ansfomaions In geneal, affine ansfomaions ae no commuaive, i.e. T A T B T B T A. When composing ansfomaions, emembe ha successive anslaions ae addiive (and commuaive). u u u u u u

15 Composiion of affine ansfomaions When composing ansfomaions, emembe ha successive oaions ae addiive (and commuaive). Successive scalings ae muliplicaive (and commuaive). ) cos( ) sin( ) sin( ) cos( cos sin sin cos cos sin sin cos θ θ θ θ θ θ θ θ s w w s s s w w

16 Tansfomaions of poins/vecos in 3D

17 Tanslaion and Roaion in 3D ' ' z' z z ' ' R z' z 3D Roaion mai, size 3 3. R is ohonomal (R T = R - )and has a deeminan of. Roaion in 3D is associaed wih an ais. We efe o oaion in 3D as oaion abou an ais. The following ae he oaion maices fo an angle abou he X,Y,Z aes especivel. Y Righ-handed coodinae ssem (X ais o he igh, Y ais X poins up) Z Noe: Roaion abou X ais leaves X coodinae unchanged, oaion abou Y ais leaves Y coodinae unchanged, oaion abou Z ais leaves Z coodinae unchanged. All hese 3 maices epesen ani-clockwise oaion of column vecos, assuming he ais of oaion poins owad he obseve and we have a igh-handed coodinae ssem.

18 hp:// hp://paulbouke.ne/geome/oae/ Roaion abou abia ais Le he abia ais be given as a uni veco = (,, 3 ) passing hough he oigin. We wan o oae abou his ais hough angle ρ.

19 hp:// hp://paulbouke.ne/geome/oae/ Roaion abou abia ais To do his, we shall appl some geomeic ansfomaions o he objec o align = (,, 3 ) wih he Z ais. We will hen oae he objec abou he Z ais hough angle ρ. We will hen appl he evese ansfomaion fom he fis sep o he objec.

20 Roaion abou an abia ais Roae abou he X ais so ha i now aligns wih he XZ plane (i.e. we wan o nullif is Y coodinae). Fo his, we oae abou X ais hough an angle ϴ given b sinq ansfoming, cosq 3 3 o 3,, ) ' (,, ) ( 3 3 Noe: As he pojecion of ono he ZY planes aligns iself wih he Z ais (ha s how he angle θ is calculaed), he veco ges ino he XZ plane.

21 Roaion abou an abia ais Now oae abou Y ais hough angle (-ϴ ) (o align i wih he Z ais) given b: sinq, cosq ansfoming ' (,, 3 ) o '' (,,)

22 Roaion abou an abia ais Now oae abou he Z ais hough angle ρ. Now evese oae abou Y-ais hough angle ϴ. Now evese oae abou X-ais hough angle ϴ. The final oaion mai is given as: R ( ) R( q ) R( q ) Rz( ) R( q ) R( q) Noe: in his deivaion, we could have also applied ansfomaions o bing ino he YZ plane b oaion abou Y ais (insead of binging i ino he XZ plane b oaing abou X ais), and hen aligned i wih Z ais b oaion abou X ais. Also hee is no specific eason wh we chose Z ais o align veco wih. We could have chosen he X o Y aes as well.

23 Roaion abou an abia ais The final oaion mai can be shown o have he following fom (ais is uni veco L = [l,m,n], i.e. l +m +n =, passing hough he oigin):

24 Roaion maices Regadless of ais, an oaion mai is alwas ohonomal. The do poduc of an wo (diffeen) ows is, he do poduc of an wo (diffeen) columns is. The magniude of an ow (o column) is. Thus RR T = R T R = I, R T =R - A oaion mai is ohonomal, bu no eve ohonomal mai is a oaion mai (wh?)

25 Affine ansfomaions in 3D 3D affine ansfomaions ae epesened as follows: z A A A A A A A A A z z Thee ae degees of feedom. This subsumes oaions, scalings, sheaing, eflecions and anslaions. Rigid ansfomaions (oaions and anslaions) peseve lenghs, aeas and angles. Affine ansfomaions peseve collineai of poins, and aios of disances, bu no lenghs o aeas o angles.

26 Reflecions in 3D Acoss XY plane: Z coodinae changes Acoss YZ plane: X coodinae changes z z z z

27 Composiion of affine ansfomaions Composiion of muliple pes of affine ansfomaions is given b he muliplicaion of hei coesponding maices. Eample: Reflecion acoss an abia plane P:. Tanslae so ha a known poin in plane P coincides wih he oigin.. Roae such ha he nomal veco of he plane coincides wih he Z ais. 3. Pefom eflecion acoss XY plane (Z ais is nomal o he XY plane). 4. Appl invese ansfom fo sep. 5. Appl invese ansfom fo sep.

28 Composiion of affine ansfomaions Jus as in D, affine ansfomaions in 3D do no commue in geneal. Successive anslaions ae addiive, successive scaling abou he same ais ae muliplicaive, successive oaions abou he same ais ae addiive.

29 Degees of feedom (DoF) in 3D oaion mai The numbe of DoF efes o he numbe of independen paamees equied o chaaceize he oaion mai. A 3D oaion mai has size 3 3, bu i has onl 3 DoF. This is because he oaion in 3D is paameeized b an ais (which is a uni veco and has DoF) and he angle of oaion (which gives he 3 d DoF). Anohe wa of seeing his: he fis column of he ohonomal mai accouns fo DoF (as i is a uni veco), he second column being pependicula o he fis one accouns fo one moe DoF (wh jus one moe? *), and he hid column is a coss poduc of he fis wo columns (aking cae o choose he sign such ha he deeminan of he mai is ). * If ou conside a plane pependicula o he veco given b he fis column, hen he second veco is given b a oaion b angle (sa) ϒ inside his plane ha s wh jus one DoF.

30 Image Fomaion in a Camea

31 Pinhole Camea No pinhole Baie wih pinhole: blocks mos of he ligh as Objec Objec Phoo Film Phoo Film

32 Pinhole Camea wih Non-Inveed Plane

33 Image Fomaion in a Pinhole Camea O = cene of pojecion of he camea (pinhole), also eaed as oigin of he coodinae ssem fo he objec ( camea coodinae ssem o camea fame ) P = (X,Y,Z) = objec poin (3D veco) p = (,,z=f) image of P on he image plane π, Z ais is nomal o π (i.e. π is paallel o XOY plane) Noe: p is obained b he inesecion of line OP wih π o = image cene- given b he poin whee he veco nomal o π and passing hough O (his veco is he Z ais!) inesecs π. Line Oo = opical ais of he camea lens (coincides wih he Z ais) f = focal lengh = pependicula disance fom O ono he image plane π = lengh of segmen Oo

34 hp://en.wikipedia.og/wiki/pinhole_camea_model Y ais Z ais P=(X,Y,Z) p=(,) Z B similai of iangles: p ais o ais X Y X ais X f, Z X f, Z f f Y Z Y Z Fo non-inveed image plane: Which iangles ae simila? Tiangle Oop and Tiangle OO P (analogousl Tiangle Oop and Tiangle OO P ae simila) p = pojecion of p ono XZ plane, O = Conside plane passing hough P and paallel o image plane. O is he inesecion of he opical ais Oo wih his plane. P = pojecion of P ono XZ plane.

35 hp://en.wikipedia.og/wiki/pinhole_camea_model o p O Z O X ais of image plane X ais P X Z ais Viewing down he Y ais f Which iangles ae simila? Tiangle Oop and Tiangle OO P (analogousl Tiangle Oop and Tiangle OO P ae simila) p = pojecion of p ono XZ plane, O = Conside plane passing hough P and paallel o image plane. O is he inesecion of he opical ais Oo wih his plane. P = pojecion of P ono XZ plane. X Z Viewing down he X ais will ield a simila figue and lead ou o deive: f Y Z

36 Image Fomaion in a Pinhole Camea The pojecion p of poin P is called a pespecive pojecion. The elaion beween p = (,,z=f) and P = (X,Y,Z) is non-linea, as he Z coodinae will va fom poin o poin. This pe of a ansfomaion does no peseve angles, lenghs o aeas. I is also non-linea. In mai fom, we have: f / Z f / Z X Y Z Noe ha he enies in his mai hemselves depend on he Z coodinae of he objec poin(s). So his is a non-linea elaionship!

37 Image Fomaion in a Pinhole Camea: Homogeneous Coodinaes The elaionship beween (,) and (X,Y,Z) can also be epessed as follows using homogeneous coodinaes: ' ', ' ', ' ' ' z z Z Y X f f z

38 Piels and Image poins The objec poins pojec ono poins in an image. The image epesened in a camea is in disceized foma, in he fom of a D aa. Thus he coodinaes p = (,,z=f) ae conveed ino discee spaial coodinaes in ems of piels. The eac elaionship depends upon he sampling ae of he camea (piel esoluion) and he aspec aio. ( o ) s, ( o ) s im im Coodinaes of opical cene in ems of piels Piel aspec aio

39 Weak-pespecive pojecion If he vaiaion in deph (Z coodinae) of diffeen poins in he scene is negligible compaed o he aveage deph Z of he scene, hen he pojecion is called weakpespecive: ', ' ) ) ' / ( ' / ( ' ' Z Y f Z X f Z Z Z X f Z X f

40 Ohogaphic o Paallel Pojecion If he diecions of pojecion ae paallel o each ohe and ohogonal o he image plane, he pojecion of he 3D poins ono D peseves angles, lenghs and aeas. Such a pojecion is called as ohogaphic o paallel pojecion. The weak pespecive pojecion is a scaled fom of an ohogaphic pojecion. Diecions of pojecion ae paallel implies infinie focal lengh (cene of pojecion is ve fa awa fom he image plane). In such cases, = X, = Y.

41 Ohogaphic o Paallel Pojecion hp:// /546-s9/www/lec/6/lec6.pdf

42 Vanishing Poins

43 Vanishing Poins P=(X,Y,Z) Line L = (l,l,lz) p=(,,z=f) o VL f O Z Line paallel o L passing hough O inesecs he image plane a poin VL his is called as he vanishing poin fo line L. The vanishing poin of line L unde pespecive pojecion is basicall he pojecion of an infiniel disan poin on L ono he image plane. π Two paallel lines in objec space will have he same vanishing poin. I is fo his eason ha images of paallel lines unde pespecive pojecion appea o inesec.

44 Vanishing Poins P=(X,Y,Z) p=(,,z=f) π O f Z o Line L = (l,l,l z ) VL z VL z VL z z z l l f l l f l Z l Y f l Z l X f Z X f l Z Z l Y Y l X X ) ( lim, ) ( lim ) (, ) ( ) ( ) ( ) (, ) (, ) (

45 hp:// hp://

46 z VL z VL z z z l l f l l f l Z l Y f l Z l X f Z X f l Z Z l Y Y l X X ) ( lim, ) ( lim ) (, ) ( ) ( ) ( ) (, ) (, ) ( If l z =, i means he se of oiginal paallel lines in 3D space lie in a plane paallel o he image plane (which is ofen he XY plane). In such cases, hee is no vanishing poin, o we have a vanishing poin a infini. hp://plus.mahs.og/conen/geing-picue

47 Vanishing Lines Conside wo o moe ses of paallel lines in 3D space ling on he same plane. The vanishing poins of all hese ses of lines ae collinea (wh?). This line conaining all hese vanishing poins is called he vanishing line. If he plane involved is he gound plane, hen he vanishing line is called he hoizon.

48 Fom slides b Fank Dallae

49 Camea Calibaion

50 Fom D o 3D? Given piel coodinaes of some poins in an image, we wan o infe he 3D coodinaes of he undeling objec poins. Fo his, we need o know () he elaionship beween he camea coodinae ssem and he wold coodinae ssem, and () he elaionship beween piel coodinaes and he acual coodinaes of he D image poins (in he camea coodinae ssem).

51 Fom D o 3D? Wha is he wold coodinae ssem? I is a coodinae ssem chosen b he use (sa he designe of an envionmen fo a obo o avel, he achiec of a building, ec). I is picall diffeen fom he camea coodinae ssem.

52 Fom D o 3D? The oienaion and posiion of he camea coodinae ssem wih espec o he wold coodinae ssem is given b he einsic camea paamees. The elaionship beween he image coodinaes and hei epesenaion in ems of piels is given b he ininsic camea paamees.

53 Fom D o 3D? The einsic and ininsic paamees ae unknown. The can be deemined using a pocess called as (geomeic) camea calibaion.

54 Camea calibaion: wh? Given picues fom a calibaed camea (o a se of calibaed cameas), we can econsuc pas of he undeling 3D scene. We can esimae vaious dimensions in 3D jus given he D image(s). This is all apa fom he fac ha we ge o esimae he camea esoluion, focal lengh, ec. (in seveal cases, hese paamees ma be unknown!).

55 Camea calibaion: how? To pefom calibaion, we ake phoogaphs of an objec whose geome is known and which will poduce salien feaue poins in an image. Eample: checkeboad cube. Such an objec is called calibaion objec.

56 We will measue he 3D coodinaes of vaious juncion poins (maked ellow) of such a calibaion objec. Ideall, we would diecl like o epess hese coodinaes in ems of he oigin of he camea coodinae ssem (i.e. wih espec o he poin O in he pevious slides). Bu ha is no phsicall possible, so we epess he coodinaes wih espec o an oigin O ha we choose. Eample: i could be he boom lef cone of he checkeboad paen (maked hee as a ed cicle). This is he wold coodinae ssem. Secondl, we will measue he coodinaes of he piel locaions of he vaious juncion poins fom he image ha we capued wih he camea. Thus fo each of he N juncion poins, we have he piel coodinaes as well as he wold coodinaes. Fom his, we have o deive he einsic and ininsic paamees of he camea.

57 Einsic paamees w w w c c c z Z Y X Z Y X w c w c P P R P P,, ), ( The einsic camea paamees ae picall unknown. The accoun fo 6 degees of feedom (i.e. 6 unknowns) wh? Coodinaes of phsical poin P in ems of wold coodinae ssem (his is measued phsicall, e.g. wih a measuing ape) Coodinaes of phsical poin P in ems of camea coodinae ssem

58 Einsic paamees Noe: he wold coodinae ssem and he camea coodinae ssem ae no aligned. R and epesen he oaion and anslaion (especivel) ha akes ou fom he wold coodinae ssem o he camea coodinae ssem.

59 Ininsic paamees The elaionship beween image coodinaes and piel coodinaes is given as: ( o ) s, ( o ) s im im Coodinaes of opical cene in ems of piels Piel aspec aio The ohe ininsic camea paamee is he focal lengh f. Ininsic paamees ae also unknown! The accoun fo 5 degees of feedom.

60 Thee main equaions and a new one im im s o s o ) (, ) ( z w w w c c c Z Y X Z Y X R c c c c Z Y f Z X f, ) ( ) ( ) ( ) ( ) ( ) ( 3 3 P R P R P R P R w w w w f s o f s o im im R R R R Row of he mai R c c im c c im Z Y f s o Z X f s o ) ( ) (

61 Pojecion equaion in mai fom z z Z Y X o s f o s f z im im w w w ˆ ˆ, ˆ ˆ, ˆ ˆ ˆ R R R in M e M ) ( ) ( ) ( ) ( ) ( ) ( 3 3 P R P R P R P R w w w w f s o f s o im im Ininsic camea mai (3 3) Einsic camea mai (3 4) M = 3 4 camea mai

62 Camea Calibaion i i im i i i im i w i w i w i i i i z z Z Y X o s f o s f z N i i ˆ ˆ, ˆ ˆ, ˆ ˆ ˆ,,,,,,, R R R unknowns.,i.e.,,,,,,,,esimae and Given,,, i,, R N i w i w i w i N im i im i o o s s f Z Y X

63 Camea Calibaion Conside he equaions, wo fo each of he N poins: i i im, i im, i o o f s f s R P R P 3 R P R P 3 w, i w, i w, i w, i z Refe o lecue scans and ebook b Tucco and Vei (secion 6. and 6..) fo he wo diffeen calibaion pocedues. z Noe: In he above equaions, we have = -R., = -R., z = -R 3. f f 3 3 X X X X w, i w, i w, i w, i 3 Y Y 3 w, i w, i Y Y w, i w, i 3 33 Z 3 Z 33 w, i Z w, i Z w, i w, i z z

64 Camea calibaion: Image cene The fis calibaion pocedue menioned in he lecues ields all paamees ecep fo he coodinaes of he image cene, given b (o,o ). To find he image cene, we compue he vanishing poins of hee muuall pependicula ses of lines. The hee vanishing poins fom a iangle, and he image cene uns ou o be he ohocene of ha iangle (i.e. poin whee he hee aliudes of he iangle mee)! The second calibaion pocedue deemines he image cene as well.

65 Fis pocedue (b Tsai, secions 6.. o 6..3 of Tucco and Vei) I eplicil finds he camea paamees duing calibaion. Given N 8 pais of poins, i ses up an equaion of he fom Av = whee A has size N 8 and conains funcions of known coodinaes, and v is an 8 veco of unknown values. The soluion v is obained (up o an unknown scala and sign) as he eigenveco of A T A wih he leas (o zeo) eigenvalue. (In he noise-fee case, A has ank 7.) The camea paamees ae deived fom v as menioned in he book. The opical cene (o,o ) is sepaael esimaed using he ohocene pope of vanishing poins, and his esimae mus be given as inpu o his mehod.

66 Second pocedue (b Faugeas and Toscani, secion 6.3 of Tucco and Vei) I seeks o diecl find he camea mai M (which is a 3 4 mai). The camea mai M has onl degees of feedom (wh?). The camea mai is found b seing up an equaion of he fom Am = whee m is a veco conaining he enies of M, and A is a N mai. Veco m is compued (up o an unknwon scale and sign) as he eigenveco of A T A wih he leas (ideall zeo) eigenvalue.

67 Second pocedue (b Faugeas and Toscani, secion 6.3 of Tucco and Vei) Veco m is eshaped o ge a 3 4 mai M. The unknown scale and sign need no bohe us (wh?) Wha if we waned o find he ininsic and einsic paamees? The book b Tucco and Vei gives one mehod fo his. In he ne slide, we will see anohe mehod!

68 Second pocedue (b Faugeas and Toscani) using RQ facoizaion Recall ha he camea mai is given as: f s M M M i n e M f s i n o o ( R R) 3 3 ( M i n R M R R R3 R) Conside he RQ decomposiion of he 3 3 submai G, G = G u G o whee G u is uppe iangula and G o is ohonomal. i n ( G g) Uppe iangula!

69 Second pocedue (b Faugeas and Toscani) This RQ decomposiion alwas eiss and is unique fo a full-ank mai (which G is). The RQ decomposiion is deived fom he moe popula QR decomposiion which also alwas eiss fo an mai, and is unique fo a full-ank mai. Code (Te): ReveseRows = [ ; ; ]; [Q R] = q((reveserows * A)'); R = ReveseRows * R' * ReveseRows; Q = ReveseRows * Q'; hps://

70 Second pocedue (b Faugeas and Toscani) The afoemenioned RQ decomposiion can be used o deemine M in and R. The anslaion veco = R T (M in ) - g (see pevious slides). This mehod gives us f, f, o and o as well as R and. Noice ha (o, o ) need no be sepaael esimaed unlike mehod.

71 Using Calibaed Cameas

72 Deemining deph Le p be he image of poin P capued using a calibaed camea C wih 3 4 known pojecion mai M. Hence we have: Given jus he image poin coodinaes, we can onl deemine he 3D diecion on which he objec poin lies. This diecion is given as (u,v,f), whee: ˆ ˆ, ˆ ˆ, ˆ ˆ ˆ z v z u Z Y X z w w w w M M P p ˆ ˆ, ˆ ˆ z v z u

73 Deemining deph To deemine he eac locaion, we will need anohe calibaed camea. Le p be he image of he same poin P capued using calibaed camea C wih 3 4 known pojecion mai M. Hence we have: We again deemine he 3D diecion on which he objec poin lies. This diecion is given as (u,v,f), whee: The inesecion of he wo diecions ields he 3D coodinae of he poin. ˆ ˆ, ˆ ˆ, ˆ ˆ ˆ z v z u Z Y X z w w w w M M P p ˆ ˆ, ˆ ˆ z v z u

74 P p p O O

75 Coss-Raios: Pojecive Invaians

76 Measuing Heigh using Coss-Raios Conside fou collinea poins A, B, C, D (in ha ode). The following quani is called he coss-aio of A, B, C, D: coss AC BD aio( A, B, C, D) AD BC AC AD / / BC BD A B C D

77 Coss-Raio: Pojecive invaian Coss-aios ae invaian unde pespecive pojecion, i.e. he coss aio of fou collinea in 3D = coss aio of hei pojecions (i.e., images) ono a D image plane! We will pove his algebaicall in class! A geomeic poof is on he ne slide. Recall lenghs, aeas, aios of lenghs, aios of aeas ae no peseved unde pespecive pojecion (aios of lenghs and aeas ae peseved unde affine ansfomaion).

78 Coss aio: pojecive invaian Thee eiss a beauiful heoem in geome which sas ha he coss aio of he poins (A,B,C,D) and ha of he poins (A,B,C,D ) in he figue below ae equal. hp:// Raio.shml

79 Coss aio: pojecive invaian Now conside 4 collinea poins A,B,C,D in 3D space whose images on he image plane as A,B,C,D. Then cleal lines AA, BB, CC, DD inesec a poin M which is he cene of pojecion. Invoking he heoem poves ou esul!

80 v p O c Image plane b C R B Gound plane Poin whose heigh BC is o be deemined Refeence poin whose heigh BR is known coss aio( B, R, C, ) BC BR RC RC RC bc vp bv c Measuing Heigh using Coss-Raios p BC R B RC RC is unknown es ae known

81 Slide aken fom Deek Hoiem (UIUC) Inspied b he PhD hesis of Anonio Ciminisi Accuae Visual Meolog fgom Single and Muliple Uncalibaed Images, Univesi of Ofod, 999

82 Slide aken fom Deek Hoiem (UIUC) Inspied b he PhD hesis of Anonio Ciminisi Accuae Visual Meolog fgom Single and Muliple Uncalibaed Images, Univesi of Ofod, 999 v z Noe: Refe o ne slide fo eplanaion. Line bb is NOT necessail paallel o lines in G. L v v v H R H G vanishing line (hoizon) Refeence diecion G b b vz ( R H) b v R Z R H R

83 Inpu: an image of a peson sanding ne o a building o a sucue whose heigh R is known. To deemine: heigh of he peson (H) fom he image. Assumpion : he building is in he so called efeence diecion wih vanishing poin (called v z - which will lie a if his diecion is paallel o camea plane). Assumpion : We ae able o find wo ses of paallel lines (called G and G ) on he gound plane. Thei coesponding vanishing poins fom he hoizon. Poins b and b ae boh on he gound plane. Conside a line (sa L) passing hough and paallel o line bb. The images of L and bb will inesec a a poin on he hoizon (wh? Because bb is coplana wih G and G, and vanishing poins of coplana lines ae collinea). Le s call his poin v.

84 Now line v (i.e. L) is paallel o bb in 3D space. Also line v inesecs he efeence diecion a poin. Hence in 3D space, b = b = unknown heigh H (on he image plane, of couse hei lenghs ae diffeen). Now use he coss-aio invaiance pope o find H: b v v Z Z R H R

85 Algoihm Find he vanishing poin in he efeence diecion v z. Find he hoizon line b joining he vanishing poins of wo pais of paallel lines on he gound plane. Find he poin of inesecion (v) of he hoizon wih he line joining he base of he building (b) and he poin whee he peson is sanding (b ). Find he poin of inesecion () of he line v ( = peson s head) wih efeence diecion. Use he coss-aio fomula o deemine he heigh of he peson.

86 Plana Homogaph

87 Plana Homogaph Conside a plana objec in 3D space, imaged b wo cameas. The coodinaes of coesponding poins in he wo images ae said o be elaed b a plana homogaph. This elaionship can be deemined wihou calibaing eihe camea.

88 P Plane π p p O O

89

90 Plana Homogaph (wih homogeneous coodinaes) Le equaion of plane π be ax + by +cz + =, i.e. [N ]P = whee N = (a,b,c) is he veco (in 3D) nomal o he plane and P = (X,Y,Z,) in he coodinae ssem of he fis camea. The coodinaes of is image in he fis camea (in coodinae ssem of fis camea) ae given b: X u Y p v M MP Z fw M is he einsic (34) mai fo he fis camea. Noe ha he image coodinaes coesponding o P ae (u /w,v /w,f) which in a diecional sense is equivalen o (u /(fw ), v /(fw ),).

91 Plana Homogaph (wih homogeneous coodinaes) Cleal P lies on a a fom he pinhole (oigin) o he poin p in 3D space. Hence we can sa ha P u v fw k au bv cfw k Np k I P N p I has size 3 3, N has size 3, so i is a 4 3 mai The (X,Y,Z) coodinaes of P can be epessed as (u /k,v /k,fw /k). The homogeneous coodinae k can be inepeed as a paamee which ells ou eacl whee on he line fom he pinhole o p, ou poin P lies. In geneal, his k would be unknown, bu hee we have moe infomaion ha P lies on a plane whose equaion is known. This would ield au /k + bv /k + cfw /k + =, i.e. au + bv + cfw + k =.

92 Plana Homogaph (wih homogeneous coodinaes) Fo he second camea, we have:, Hp p N I M p p N I M M P M M M p ~ ~ ~ ' ' ' Z Y X Z Y X fw v u H is a 3 3 mai, as M~ is a 3 4 mai, and (I;-N) is a 4 3 mai. Noe: Le (X,Y,Z,) be he coodinaes of poin P in he coodinae fame of he second camea. p is he coodinae of is image again in he ssem of he second camea. Mai M is he einsic (34) mai fo he second camea and 4 4 mai M, conains suiable einsic paamees o align he coodinae ssems of he fis camea o ha of he second camea.

93 Plana Homogaph (wih homogeneous coodinaes) The pevious elaionship was beween he coodinaes of he images of P: p in he fis camea s coodinae ssem, and p in he second camea s coodinae ssem. Wha abou a simila elaionship in he especive image coodinae ssems? p M p Hp -,in, im p, im M HM,in HM,in,in p p, im, im Hp ˆ, im M,in and M,in ae ininsic (33) camea maices giving he elaion beween he coodinaes of a poin in camea coodinae ssem and image coodinae ssem. This is also a 3 3 mai which elaes he image coodinaes in he wo images (in hei especive image coodinae ssems bu wih homogeneous coodinaes)

94 Plana Homogaph Given wo images of a coplana scene aken fom wo diffeen (uncalibaed) cameas, how will ou deemine he plana homogaph mai H (ahe H^)? How man poin coespondences will ou equie? The answe is 4 so ha we have a leas 8 equaions fo he 8 degees of feedom. p ˆ, im v Hp, im, im, im u w Hˆ Hˆ Hˆ Hˆ 3 3 u u u u Hˆ Hˆ Hˆ Hˆ 3 3 v v v v Hˆ Hˆ ˆ H3 Hˆ 3w Hˆ w Hˆ Hˆ w w Hˆ Hˆ Hˆ 3 Hˆ Hˆ Hˆ Hˆ 3 3 Hˆ Hˆ Hˆ, im, im, im, im u v w Hˆ Hˆ Hˆ Hˆ 3 3, im, im, im, im Hˆ Hˆ Hˆ Hˆ ,, im u w,, im v w

95 hp:// demo.hml

96 , im, i, im, i, im, i, im, i, im, i Ah, A Hˆ Hˆ 3 3, im, i, im, i, im, i, im, i, im, i Hˆ Hˆ 3 Hˆ Hˆ Plana Homogaph, 3, im, i, im, i, im, i 33 33, im, i Reaanging hese equaions, we ge: Hˆ has size N 9, h has size 9, im, i, im, i Hˆ Hˆ, im, i, im, i, im, i, im, i, im, i, im, i Hˆ Hˆ Hˆ Hˆ, im, i, im, i 3 3, im, i, im, i, im i, im, i Hˆ Hˆ ˆ H ˆ H Hˆ ˆ H ˆ H ˆ H Thee will be N such pais of equaions (i.e. oall N equaions), given N pais of coesponding poins in he wo images The equaion Ah = will be solved b compuing he SVD of A, i.e. A = USV T. The veco h will be given b he singula veco in V, coesponding o he null singula value (in he ideal case) o he leas singula value. This is equivalen o he eigenveco of A T A wih he leas eigenvalue. In he ideal case, A has ank 8 and hence he eac soluion eiss. In he non-ideal case, hee is no h such ha Ah =, so we find an appoimae soluion, i.e. a soluion fo which Ah is small in magniude.

97 Plana Homogaph Alhough hee ae 9 enies in he mai, hee ae onl 8 degees of feedom. You can see his if ou divide he numeao and denominao of he following equaions b H^33. Hˆ Hˆ Hˆ Hˆ 3 3 Hˆ Hˆ Hˆ Hˆ 3 3 Hˆ Hˆ Hˆ Hˆ ~ ~ ~ H H H ~ ~ H3 H3 ~ ~ ~ H H H ~ ~ H H 3 The numbe of poin coespondences equied is a leas

98 Plana homogaph I is a moion model ha is moe geneal han D affine ansfomaions! Noe: in he deivaion, make sue ou undesand ha he planai of he objec being imaged, is ciical. Ohewise ou need a moion model moe complicaed han a homogaph! Again noe: ou do no need calibaed cameas (i.e. he coesponding poins ae in ems of image coodinaes, i.e. in piels), and ou do no need o know he ininsic o einsic camea paamees in ode o deemine he homogaph. The homogaph ansfomaion has 8 degees of feedom and is moe geneal han he affine ansfomaion which has onl 6 degees of feedom. Obseve he homogaph mai on he pevious slides in an affine ansfomaion, H 3 and H 3 would be.

99 Adding a Lens

100 Pinhole Camea o Camea wih Lens Wih a lage pinhole, he image spo is lage, esuling in a blu image. Wih a small pinhole, ligh is educed esuling in a shap bu nois image Wih a simple lens, much moe ligh can be bough ino shap focus. hp://en.wikipedia.og/wiki/camea_lens

101 Thin convegen lens: Opical ais Sos passing hough he opical cene O of he lens and nomal o he plane of he lens All as of ligh fom objec poin (sa) P paallel o he opical ais pass hough he pincipal focus F afe efacion. The a of ligh fom P hough O passes wihou efacion. Bu all as of ligh emanaing fom P passing hough he lens ae focused ono a single poin p he image of P (if poin P was in focus).

102 Camea Lens: Focal Lengh, Apeue hp://en.wikipedia.og/wiki/cicle_of_confusion A lens focusses ligh ono a senso (o phoo-film). The disance beween he cene of he lens and he pincipal focus is called focal lengh. This disance is changed as he shape of he lens changes. Defocussed poins map ono a cicle of confusion on he senso. Focal lengh affecs magnificaion. Lage focal lenghs ield smalle fields of view bu moe magnificaion. Small focal lenghs ield lage fields of view bu less magnificaion.

103 Camea: Apeue A wide apeue (above) allows fo a lage shue speed han a smalle apeue (below) fo he same amoun of eposue (o he same amoun of ligh eneing he camea). Lowe shue speed = suscepibili o moion blu

104 Deph of field The images of some objec poins can appea blu. The amoun of blu depends on he disance beween he objec poin and he camea plane (i.e. deph of he poin). The ange of dephs fo which he level of blu is accepable o he human ee (o which appea accepabl shap) is called as deph of field.

105

106 Apeue and deph of field Lage apeue = shallowe DOF, as adii of cicle of confusion incease apidl Smalle apeue = moe eensive DOF, as adii of cicle of confusion incease moe slowl hp://

107 Downside of using a lens I causes adial disoion saigh lines in 3D can ge mapped ono cuved lines in D, and his effec is much moe jaing nea he image bodes.

108 Downside of using a lens The elaionship beween he disoed and undisoed coodinaes is appoimaed b: k and k ae (ea!) ininsic paamees of he camea. The associaed calibaion pocedue is now moe complicaed. 4 4 ) ( ) ( ) )( ( ) )( ( u u u d u d o o k k o o k k o o ( u, u ) = undisoed coodinaes ( d, d ) = disoed coodinaes

109 The Human Ee as a Camea Phoo-film Lens Camea apeue Appaaus o adjus widh of camea apeue hp://en.wikipedia.og/wiki/file:blausen_388_eeanaom_.png

110 Summa Affine Tansfomaions in D and 3D Pinhole camea, pespecive pojecions Vanishing poins Camea Calibaion Coss-aios Plana homogaph Adding in a lens