Feature Selection & Dynamic Tracking F&P Textbook New: Ch 11, Old: Ch 17 Guido Gerig CS 6320, Spring 2013

Transcription

1 Feature Selecton & Dynamc Trackng F&P Textbook New: Ch 11, Old: Ch 17 Gudo Gerg CS 6320, Sprng 2013 Credts: Materal Greg Welch & Gary Bshop, UNC Chapel Hll, some sldes modfed from J.M. Frahm/ M. Pollefeys, and R. Klette Course Materals

2 Materal Feature selecton: SIFT Features Trackng: Kalman Flter: F&P Chapter 17 Greg Welch and Gary Bshop, UNC: Web-ste (electronc and prnted references, book lsts, Java demo, software etc.) Course materal SIGGRAPH:

3 Trackng Rgd Objects

4 Trackng Rgd Objects

5 Trackng Rgd Objects

6 Trackng Rgd Objects

7 Feature Trackng Trackng of good features & effcent search for subsequent postons. What are good features? Requred propertes: Well-defned (.e. negborng ponts should all be dfferent) Stable across vews (.e. same 3D pont should be extracted as feature for neghborng vewponts)

8 Lowe s SIFT features (Lowe, ICCV99) SIFT: Scale Invarant Feature Transform Recover features wth change of poston, orentaton and scale

9 SIFT features Scale-space DoG maxma Verfy mnmum contrast and cornerness Orentaton from domnant gradent Descrptor based on gradent dstrbutons

10 Dynamc Feature Trackng Trackng s the problem of generatng an nference about the moton of an object gven a sequence of mages. The key techncal dffculty s mantanng an accurate representaton of the posteror on object poston gven measurements, and dong so effcently.

11

12 Kalman Flter The Kalman flter s a very powerful tool when t comes to controllng nosy systems. Apollo 8 (December 1968), the frst human spaceflght from the Earth to an orbt around the moon, would certanly not have been possble wthout the Kalman flter (see &d=293 ). Applcatons: Trackng Economcs Navgaton Depth and velocty measurements..

13 14 What s t used for? Trackng mssles Trackng heads/hands/drumstcks Extractng lp moton from vdeo Fttng Bezer patches to pont data Lots of computer vson applcatons Economcs Navgaton

14 Key Concept Nosy process data Estmate average trajectores Smoothng: Sldng wndow for averagng (here sze 64) But: If horzontal axs s tme? We know the past but not the future! Tme-dependent process: Modelng of process tself ncludng nose estmates.

15 Model for trackng Object has nternal state X Captal ndcates random varable Small represents partcular value X x Obtaned measurements n frame are Value of the measurement y Y

16 Lnear Dynamc Models State s lnearly transformed plus Gaussan nose x ~ N D x, 1 d Relevant measures are lnearly obtaned from state plus Gaussan nose y ~ N M Suffcent to mantan mean and standard devaton x, m

17 General Steps of Trackng 1. Predcton: What s the next state of the object gven past measurements P X Y y Y y 0 0,, Data assocaton: Whch measures are relevant for the state? 3. Correcton: Compute representaton of the state from predcton and measurements. P X Y y, Y y Y y 0 0, 1 1,

18 Concept Kalman Flterng predct correct

19 Independence Assumptons Only mmedate past matters P X X X PX X 1,, 1 1 Measurements depend only on current state P Y, Y,, Y X P Y X P Y,, Y X j k j k Important smplfcatons Fortunately t doesn t lmt to much!

20 Sprt of Kalman Flterng: A really smple example We are on a boat at nght and lost our poston We know: star poston

21 Fxed Poston p s poston of boat, v s velocty of boat p p 1 state s X p X D X 1 D I We only measure poston so M I, Y M X X

22 Observer 1 makes a measurement y0, m 0 Condtonal Densty Functon x0 y 0 N ( y 0, m 0 ) 0 m

23 Then: Observer 2 makes a measurement m y1, 1 Condtonal Densty Functon x1? N ( y 1, m 1 ) 1? How does second measurement affect estmate of frst measurement?

24 x K Combne measurements & varances: Kalman 2 x1 K2 2 x ( y ) 1 y y 2 2 Combne Varances (statstcs)

25 Combne measurements & varances: Kalman x x Condtonal Densty Functon N ( x 2, 2 ) Orgnal estmates updated (corrected) n the presence of a new measurement.

26 33 Predct Correct KF operates by 1. Predctng the new state and ts uncertanty 2. Correctng wth the new measurement predct correct

27 34 A really smple example We are on a boat at nght and lost our poston We know: move wth constant velocty star poston

28 35 But suppose we re movng Not all the dfference s error. Some may be moton KF can nclude a moton model Estmate velocty and poston

29 36 Process Model Descrbes how the state changes over tme The state for the frst example was scalar The process model was nothng changes A better model mght be constant velocty moton X p v p p v v 1 t 1 ( t) v 1

30 37 Measurement Model What you see from where you are not Where you are from what you see

31 38 Constant Velocty p s poston of boat, v s velocty of boat p p 1 ( t) v 1 state s X [ p v] t X D X 1 D 1 t 0 1 We only measure poston so M t t [1 0], Y [1 0] [ p v] t p

32 Multdmensonal Statstcs To be seen as a generalzaton of the scalar-valued mean and varance to hgher dmensons. 2 varables: Source: Wkpeda

33 40 State and Error Covarance Frst two moments of Gaussan process Process State (Mean) x Error Covarance d

34 41 The Process Model Process dynamcs X D X 1 w State transton Uncertanty over nterval w ~ N 0, d Dffcult to determne

35 42 Measurement Model Measurement relatonshp to state Y M X Measurement matrx Measurement uncertanty ~ N 0, m

36 Predct (Tme Update) X D X 1 D 1 Y M X D T x 1 d x 43 1

37 Measurement Update (Correct) a posteror state and error covarance X X K Y M I K M X Kalman gan Mnmzes posteror error covarance x x 44

38 The Kalman Gan 1 m T T M M M K Weghts between predcton and measurements to posteror error covarance For no measurement uncertanty: 0 m T T M M M M K y M M x y M x x 1 1 State s deduced only from measurement

39 46 The Kalman Gan Smple unvarate (scalar) example m K a posteror state and error covarance x y K x x K 1

40 47 Summary PREDICT CORRECT 1 x D x d T D D 1 x M y K x x M K I 1 m T T M M M K

41 48 Example: Estmatng a Constant The state transton matrx D I x Dx 1 w x 1 w The measurement matrx M I y Mx x Predcton x x 1 1 d

42 49 Measurement Update x x K y x 1 K K m

43 Setup/Intalzaton Generate 50 samples centered around wth standard devaton of 0.1 (var 0.01). d 10 5 x

44 51 State and Measurements m =0.1 2 =0.01 m 0.1 Flter was told the correct measurement varance.

45 52 Error Covarance (ntally 1) ( 2 )

46 State and Measurements m = 1 53 Flter was told that the measurement varance was 100 tmes greater (.e. 1) so t was slower to beleve the measurements.

47 State and Measurements m = = Flter was told that the measurement varance was 100 tmes smaller (.e ) so t was very quck to beleve the nosy measurements.

48 Demonstraton own experments

49

50

51

52

53 2D Poston-Velocty (PV)

54 2D Poston-Velocty (PV)

55 Example: Hand Gesture Recognton and Trackng

56 63 Kalman Flter Web Ste Electronc and prnted references Book lsts and recommendatons Research papers Lnks to other stes Some software News

57 64 Java-Based KF Learnng Tool On-lne 1D smulaton Lnear and non-lnear Varable dynamcs

58 65 KF Course Web Page ( ) Java-Based KF Learnng Tool KF web page

59 66 Relevant References Azarbayejan, Al, and Alex Pentland (1995). Recursve Estmaton of Moton, Structure, and Focal Length, IEEE Trans. Pattern Analyss and Machne Intellgence 17(6): Dellaert, Frank, Sebastan Thrun, and Charles Thorpe (1998). Jacoban Images of Super- Resolved Texture Maps for Model-Based Moton Estmaton and Trackng, IEEE Workshop on Applcatons of Computer Vson (WACV'98), October, Prnceton, NJ, IEEE Computer Socety.

60 Extensons: Partcle Flterng, Condensaton A. Blake, B. Bascle, M. Isard, and J. MacCormck, Statstcal models of vsual shape and moton, n Phl. Trans. R. Soc. A., vol. 356, pp , 1998 B. Isard, M., Blake, and A., Condensaton condtonal densty propagaton for vsual trackng, n Int. J. Computer Vson, vol. 28, no. 1, pp. 5 28, 1998 C. Blake, A., Isard, M.A., Reynard, and D., Learnng to track the vsual moton of contours, n J. Artfcal Intellgence, vol. 78, pp , 1995

61 Condensaton Algorthm (Blake et al. )

62 Extensons: Partcle Flterng, Condensaton

63 Extensons: Partcle Flterng, Condensaton

64 Extensons: Partcle Flterng, Condensaton