Global Motion. Estimate motion using all pixels in the image. Parametric flow gives an equation, which describes optical flow for each pixel.

Transcription

1 Globl Flow

2 Globl Motion Estimte motion using ll piels in the imge. Prmetric low gives n eqution, which describes opticl low or ech piel. Aine Projective Globl motion cn be used to generte mosics Object-bsed segmenttion

3 Bergn et l Aine

4 Aine (0,0) (,) (,) (0,0) (,) (, ) Imge t t Imge t t- 4 3 ), ( ), ( b v b u X X U v u

5 Aine 4 3 ), ( ), ( b v b u 4 3 b b v u

6 Sptil Trnsormtions trnsltion rottion sher Rigid (rottion nd trnsltion) ine

7 Bergn et l Aine 4 3 ), ( ), ( b v b u ), ( ), ( b b v u X u ) ( ) (

8 Bergn et l Opticl low constrint eq X u ) ( ) ( t v u ), ( ) ( ) ( u u T t E ), ( ) ( ) ( X() T t E ), ( ) ( ) ( E T t X X

9 Bergn et l min Liner sstem ), ( ) ( ) ( E T t X t X T T X X T X X X ) ( ( A B Homework

10 Bsic Components Prmid construction Motion estimtion Imge wrping Corse-to-ine reinement

11 Corse-to-ine globl low estimtion u=.5 piels u=.5 piels u=5 piels imge H u=0 piels imge I Gussin prmid o imge H Gussin prmid o imge I

12 Corse-to-ine globl low estimtion Compute Globl Flow Itertivel wrp & upsmple Compute Globl Flow Itertivel... imge JH Gussin prmid o imge H imge I Gussin prmid o imge I

13 0 Level= W M W * + * W Level= W M + W * * W Level=0 (,,t-) W + 3 (,,t-) M 3 (,,t)

14 Estimtion o Globl Flow Single Itertion Compute A nd B Solve A B Imge t Imge t+ Wrp b

15 Estimtion o Globl Flow Initil Estimte T b 3 4 b Itertive Imge t Imge t+ Wrp b Compute A nd B Solve A B Wrp b

16 Estimtion o Globl Flow Initil Estimte T b 3 4 b Itertive Iterte Imge t Imge t+ Wrp b Compute A nd B Solve A B Updte

17 Imge Wrping Wrping n imge into imge h using some trnsormtion g, involves mpping intensit t ech piel (,) in imge to piel (g(),g()) in imge h such tht (, ) ( g( ), g( )) In cse o ine trnsormtion, (, ) is trnsormed to s: A b (, ) U A b Displcement model Instntneous model

18 Imge Wrping (Bergn et l) ( X, t ) ( X, t) wrp ( X, t ) A ( ) ( X b) X A ( ) ( X b) X

19 Imge Wrping Imge t time t: X Imge t time t-: X X X X U ( AX b) ( A) X X ( X b) X

20 Imge Wrping How bout vlues in re not integer. But imge is smpled onl t integer rows nd columns Insted o converting to nd coping t we cn convert integer vlues to nd cop t

21 Imge Wrping ( X, t ) ( X, t) wrp ( X, t )

22 Imge Wrping But how bout the vlues in re not integer. Perorm biliner interpoltion to compute t non-integer vlues.

23 Wrping

24 Video Mosic

25 Video Mosic

26 mosic Video Mosic

27 Sprite

28 Mnn & Picrd Projective

29 Projective Flow (weighted) u u v T t t 0 0 Opticl Flow const. eqution A b C T A b u T C Projective trnsorm

30 Projective Flow (weighted) low ( u T X t A b T (( ) T t C ) ) T T T (( A b ( C ) ) ( C ) t ) minimize Homework

31 Projective Flow (weighted) ],,,,,,, [ t t t ) ( ) ( t T T T c c b b 4 3,,,,,,,

32 Projective Flow (unweighted)

33 Pseudo-Perspective Tlor Series T C b A v u

34 Biliner Tlor Series & remove Squre terms T C b A v u

35 Projective Flow (unweighted) low ( u T X t ) Minimize

36 Biliner nd Pseudo-Perspective ( T )q t T (,,,),,,, biliner c c T (,,),,,, c, c Pseudo perspective Homework

37 Algorithm- Estimte q (using pproimte model, e.g. biliner model). Relte q to p select our points S, S, S3, S4 ppl pproimte model using q to compute estimte ect p : (, ) k k

38

39 True Projective c 3 c c c 4 b b

40 Perorm lest squres it to compute. A P k k k k k k k k k k k k

41 Finl Algorithm A Gussin prmid o three or our levels is constructed or ech rme in the sequence. The prmeters p re estimted t the top level o the prmid, between the two lowest resolution imges, g nd h, using lgorithm-.

42 Finl Algorithm The estimted p is pplied to the net higher resolution imge in the prmid, to mke imges t tht level nerl congruent. The process continues down the prmid until the highest resolution imge in the prmid is reched.

43 Video Mosics Mosic ligns dierent pieces o scene into lrger piece, nd semlessl blend them. High resolution imge rom low resolution imges Incresed iled o view

44 Steps in Generting A Mosic Tke pictures Pick reerence imge Determine trnsormtion between rmes Wrp ll imges to the sme reerence view

45 Applictions o Mosics Virtul Environments Computer Gmes Movie Specil Eects Video Compression

46 Steve Mnn

47 Sequence o Imges

48 Projective Mosic

49 Aine Mosic

50 Building

51 Wl-Mrt

52 Scientiic Americn Frontiers

53 Scientiic Americn Frontiers

54 Hed-mounted Cmer t Resturnt

55 MIT Medi Lb

56 COCOA: A Sstem or Processing o Aeril Videos

57 COCOA Sstem Flow Aeril Video Telemetr* Ego Motion Compenstion Feture bsed + Grdient Bsed Motion Detection Accumultive Frme Dierencing + Bckground Modeling + Object Segmenttion Object Trcking Kernel Trcking + Blob Trcking + Occlusion Hndling COCOA Registered Imges Motion Detection Trcks Event Detection & Indeing

58 Registrtion Result - I Aeril Video - EO Mosic Alignment Msk

59 Registrtion Result - II Aeril Video - IR Mosic Alignment Msk

60 Detection Result

61 Trcking Results

62 Reerences (C code or generting mosics) The Lplcin Prmid s compct code, Burt nd Adelson, IEEE Trns on Communiction, 983. J. Bergen, P. Anndn, K. Hnn, nd R. Hingorni, Hierrchicl Model-Bsed Motion Estimtion, ECCV-9, pp 37-.

63 Reerences s/mosics.html ( Eicient representtions o video sequences nd their pplictions, Michl Irni, P. Anndn, Jim Bergen, Rkesh Kumr, nd Steve Hsu) R. Szeliski. Video mosics or virtul environments, IEEE Computer Grphics nd Applictions, pges,-30, Mrch 996.