EECS 442 Computer vision. Multiple view geometry Affine structure from Motion

Size: px

Start display at page:

Download "EECS 442 Computer vision. Multiple view geometry Affine structure from Motion"

Miranda Wiggins
6 years ago
Views:

EECS 442 Computer visio Multiple view geometry Affie structure from Motio - Affie structure from motio problem - Algebraic methods - Factorizatio

1 EECS 442 Computer visio Multiple view geometry Affie structure from Motio - Affie structure from motio problem - Algebraic methods - Factorizatio methods Readig: [HZ] Chapters: 6,4,8 [FP] Chapter: 2 Some slides of this lectures are courtesy of prof. J. Poce, prof FF Li, prof S. Lazebik & prof. M. Hebert

2 Structure from motio problem X j j M m M mj 2j M 2 Give m images of fied 3D poits ij M i X j, i,, m, j,,

3 Structure from motio problem X j j M m M mj 2j M 2 From the m correspodeces ij, estimate: m projectio matrices M i 3D poits X j motio structure

4 Applicatios Courtesy of Oford Visual Geometry Group

5 Affie structure from motio (simpler problem) Image World Image From the m correspodeces ij, estimate: m projectio matrices M i (affie cameras) 3D poits X j

6 Fiite cameras p q r R Q P O [ ]X T K R T R K M 3 3 M Caoical perspective projectio matri Affie homography (i 3D) Affie Homography (i 2D) y s K o y o α α

7 Trasformatio i 2D Affiities: y H y t A y' ' a -Preserve: - Parallel lies - Ratio of areas - Ratio of legths o colliear lies - others - 6 DOF

8 Weak perspective projectio Whe the relative scee depth is small compared to its distace from the camera P ~ ' m y' my where m f z ' is the magificatio.

9 Weak perspective projectio Whe the relative scee depth is small compared to its distace from the camera ' m y' my Scalig fuctio of the distace (magificatio)

10 Orthographic (affie) projectio Whe the camera is at a (roughly costat) distace from the scee ' y' y Distace from ceter of projectio to image plae is ifiite

11 Orthographic (affie) projectio Whe the camera is at a (roughly costat) distace from the scee

12 Affie cameras [ ]X T K R T R K M y s K o y o α α Projective case Affie case

13 Trasformatio i 2D Projective: y H y b v t A y' ' p Affiities: y H y t A y' ' a

14 Affie cameras [ ]X T K R y α s α o y o K T R K M T R K M y s K o y o α α Projective case Affie case Parallel projectio matri (poits at ifiity are mapped as poits at ifiity) Magificatio (scalig term)

15 Weak perspective projectio Qigmig Festival by the Riverside Zhag Zedua ~9 AD

16 Affie cameras [ ]X T K R y K α α T R K M b A 4affie] [4 3affie] 3 [ b a a a b a a a M X b AX Euc M b b Z Y X a a a a a a y [Homogeeous] [o-homogeeous image coordiates] [ ] b A M M Euc ; P M Euc

17 Affie cameras p P p M camera matri To recap: from ow o we defie M as the camera matri for the affie case p u v AP + b M P ; M [ A b]

18 The Affie Structure-from-Motio Problem Give m images of fied poits P j (X i ) we ca write N of cameras N of poits Problem: estimate the m 2 4 matrices M i ad the positios P j from the m correspodeces p ij. How may equatios ad how may ukow? 2m equatios i 8m+3 ukows Two approaches: - Algebraic approach (affie epipolar geometry; estimate F; cameras; poits) - Factorizatio method

19 Algebraic aalysis (2-view case) - Derive the fudametal matri F A for the affie case - Compute F A - Use F A to estimate projectio matrices - Use projectio matrices to estimate 3D poits

20 . Derivig the fudametal matri F A p P v p u Homogeeous system Dim? 44

21 Derivig the fudametal matri F A where The Affie Fudametal Matri!

22 Affie Epipolar Geometry Note: the epipolar lies are parallel.

24 Estimatig F A From at least 4 correspodeces, we obtai a liear system o the ukow alpha, beta, etc Measuremets: u, u, v, v v u v u v u v u f M M M M M Computed by least square ad by eforcig f SVD

25 Estimatig projectio matrices from epipolar costraits p P p

26 Affie ambiguity Affie PX ( PQ - )( Q X) A A

27 Estimatig projectio matrices from epipolar costraits p P p

28 Estimatig projectio matrices from epipolar costraits Choose Q such that A ~ ~ b [ ] T A ~ a ~ b b [ d] T c Caoical affie cameras Fuctio of the parameters of F

29 Estimatig projectio matrices from epipolar costraits Choose Q such that By re-eforcig the epipolar costrait, we ca compute a, b, c, d directly from the measuremets

30 Remider: epipolar costrait p P v p u Homogeeous system

31 Estimatig projectio matrices from epipolar costraits Choose Q such that ~ A ~ b Re-eforce the Epipolar costrait

32 Estimatig projectio matrices from epipolar costraits Choose Q such that A b

33 Estimatig projectio matrices from epipolar costraits Liear relatioship betwee measuremets ad ukow Ukow: a, b, c, d Measuremets: u, u, v, v From at least 4 correspodeces, we ca solve this liear system ad compute a, b, c, d (via least square) The cameras ca be computed How about the structure?

34 4. Estimatig the structure from epipolar costraits A b Ca be solved by least square agai

35 A factorizatio method Tomasi & Kaade algorithm C. Tomasi ad T. Kaade. Shape ad motio from image streams uder orthography: A factorizatio method. IJCV, 9(2):37-54, November 992. Ceterig the data Factorizatio

36 Ceterig: subtract the cetroid of the image poits ( ) j i k k j i k i k i i j i k ik ij ij ˆ ˆ A X X X A b A X b A X + + A factorizatio method - Ceterig the data X k ik i ^

37 Ceterig: subtract the cetroid of the image poits ( ) j i k k j i k i k i i j i k ik ij ij ˆ ˆ A X X X A b A X b A X + + A factorizatio method - Ceterig the data

38 Ceterig: subtract the cetroid of the image poits ( ) + + k k j i k i k i i j i k ik ij ij ˆ X X A b A X b A X j i ij X A ˆ A factorizatio method - Ceterig the data Assume that the origi of the world coordiate system is at the cetroid of the 3D poits After ceterig, each ormalized poit ij is related to the 3D poit X i by

39 A factorizatio method - Ceterig the data X ˆ A ij i X j

40 Let s create a 2m data (measuremet) matri: m m m D ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ L O L L cameras (2m ) poits ( ) A factorizatio method - factorizatio

41 Let s create a 2m data (measuremet) matri: [ ] m m m m X X X A A A D L M L O L L ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ cameras (2m 3) poits (3 ) The measuremet matri D M S has rak 3 (it s a product of a 2m3 matri ad 3 matri) A factorizatio method - factorizatio (2m ) M S

42 Factorizig the measuremet matri Source: M. Hebert

43 Factorizig the measuremet matri Sigular value decompositio of D: Source: M. Hebert

44 Factorizig the measuremet matri Sigular value decompositio of D: Sice rak (D)3, there are oly 3 o-zero sigular values Source: M. Hebert

45 Factorizig the measuremet matri Obtaiig a factorizatio from SVD: Motio (cameras) structure What is the issue here? D has rak>3 because of - measuremet oise - affie approimatio

46 Factorizig the measuremet matri Obtaiig a factorizatio from SVD: structure D D

47 Affie ambiguity The decompositio is ot uique. We get the same D by usig ay 3 3 matri C ad applyig the trasformatios M MC, S C - S We ca eforce some Euclidea costraits to resolve this ambiguity (more o et lecture!)

48 Algorithm summary. Give: m images ad features ij 2. For each image i, ceter the feature coordiates 3. Costruct a 2m measuremet matri D: Colum j cotais the projectio of poit j i all views Row i cotais oe coordiate of the projectios of all the poits i image i 4. Factorize D: Compute SVD: D U W V T Create U 3 by takig the first 3 colums of U Create V 3 by takig the first 3 colums of V Create W 3 by takig the upper left 3 3 block of W 5. Create the motio ad shape matrices: M M U 3 ad S W 3 V 3 T (or U 3 W 3½ ad S W 3½ V 3T ) 6. Elimiate affie ambiguity

49 Recostructio results C. Tomasi ad T. Kaade. Shape ad motio from image streams uder orthography: A factorizatio method. IJCV, 9(2):37-54, November 992.

50 Net lecture Multiple view geometry Perspective structure from Motio

Affine Structure from Motion

Affine Structure from Motion Affie Structure from Motio EECS 598-8 Fall 24! Foudatios of Computer Visio!! Istructor: Jaso Corso (jjcorso)! web.eecs.umich.edu/~jjcorso/t/598f4!! Readigs: FP 8.2! Date: /5/4!! Materials o these slides