Tracking. Many slides adapted from Kristen Grauman, Deva Ramanan

Transcription

1 Tracking Man slides adaped from Krisen Grauman Deva Ramanan

2 Coures G. Hager

3 Coures G. Hager J. Kosecka cs3b

4 Adapive Human-Moion Tracking Acquisiion Decimaion b facor 5 Moion deecor Grascale convers. Image differencing Moion hisor im. Segmenaion Validaion Skin color deecor Moion presence Skin color presence RGB o HSV convers. Hue-saura. Limier Big conour presence Skin color binar im. Image closing Average ravelled disance Segmenaion Adapaion Coninuous adapaion Moion iniializaion J. Kosecka cs3b Tracking Disance scoring Conour o arge assignmen

5 Coures G. Hager

6 Tracking wih dnamics Ke idea: Given a model of epeced moion predic where objecs will occur in ne frame even before seeing he image Resric search for he objec Improved esimaes since measuremen noise is reduced b rajecor smoohness

7 General model for racking The moving objec of ineres is characerized b an underling sae Sae gives rise o measuremens or observaions Y A each ime he sae changes o and we ge a new observaion Y Y Y Y

8 Seps of racking redicion: Wha is he ne sae of he objec given pas measuremens? Y Y

9 Seps of racking redicion: Wha is he ne sae of he objec given pas measuremens? Y Y Correcion: Compue an updaed esimae of he sae from predicion and measuremens Y Y Y

10 Seps of racking redicion: Wha is he ne sae of he objec given pas measuremens? Y Y Correcion: Compue an updaed esimae of he sae from predicion and measuremens Y Y Y Tracking can be seen as he process of propagaing he poserior disribuion of sae given measuremens across ime

11 Simplifing assumpions Onl he immediae pas maers dnamics model

12 Simplifing assumpions Onl he immediae pas maers Measuremens depend onl on he curren sae Y Y Y Y dnamics model observaion model

13 Simplifing assumpions Onl he immediae pas maers Measuremens depend onl on he curren sae Y Y Y Y dnamics model observaion model - Y Y Y - Y

14 Tracking as inducion Base case: Assume we have iniial prior ha predics sae in absence of an evidence: A he firs frame correc his given he value of Y

15 Tracking as inducion Base case: Assume we have iniial prior ha predics sae in absence of an evidence: A he firs frame correc his given he value of Y Y

16 Tracking as inducion Base case: Assume we have iniial prior ha predics sae in absence of an evidence: A he firs frame correc his given he value of Y Given correced esimae for frame : redic for frame + Correc for frame + predic " correc "

17 redicion redicion involves represening given

18 redicion redicion involves represening given d d d Law of oal probabili

19 redicion redicion involves represening given d d d Condiioning on

20 redicion redicion involves represening given d d d Independence assumpion

21 redicion redicion involves represening given d d d dnamics model correced esimae from previous sep

22 Correcion Correcion involves compuing given prediced value

23 Correcion Correcion involves compuing given prediced value d Baes rule

24 Correcion Correcion involves compuing given prediced value d Independence assumpion observaion depends onl on sae

25 Correcion Correcion involves compuing given prediced value d Condiioning on

26 Correcion Correcion involves compuing given prediced value d observaion model prediced esimae normalizaion facor

27 Summar: redicion and correcion redicion: Correcion: d d dnamics model correced esimae from previous sep observaion model prediced esimae

28 Baes Filer Definiion Environmen sae Measuremen z Can we calculae p z z z u u u?

29 Baes Filers Illusraed Localizaion problem

30 sae ime z observaion u acion η consan Baes Filers u z p Bel z z u p z z u z p η z z u p z p η d z u p z u p z p η d Bel u p z p η u z u z p Markov Baes Markov d u u z p u p z p η

31 Baes Filers sae ime z observaion u acion Bel η p z p u Bel d

32 Baes Filers Illusraed

33 Baes Filers Iniial Esimae of Sae Ierae Receive measuremen updae our belief uncerain shrinks redic updae our belief uncerain grows

34 Eample of Baesian Inference psaircase Slow.8 Down! Sensor model pimage saircase.7 pimage no saircase. Environmen prior psaircase.? Baesian inference psaircase image pimage saircase psaircase pim sair psair + pim no sair pno sair.7. /

35 Mehods Baes Filer Kalman Filer aricle Filer Unscened Kalman Filer Eended Kalman Filer

36 Overview The Tracking roblem Baes Filers Kalman Filers Using Kalman Filers aricle Filers

37 The Kalman filer Linear dnamics model: sae undergoes linear ransformaion plus Gaussian noise Observaion model: measuremen is linearl ransformed sae plus Gaussian noise The prediced/correced sae disribuions are Gaussian You onl need o mainain he mean and covariance The calculaions are eas all he inegrals can be done in closed form

38 Tracking wih KFs: Gaussians! iniial esimae predicion measuremen updae

39 Gaussians : ~ σ πσ σ e p N p -σ σ Univariae / / : ~ ì Ó ì Ó Ó ì e p Í p d π Mulivariae

40 ~ ~ σ σ a b a N Y b a Y N + # $ % + roperies of Univariae Gaussians!! " # $ $ % & * + ~ ~ ~ σ σ σ σ σ σ σ σ σ σ N p p N N

41 We sa in he Gaussian world as long as we sar wih Gaussians and perform onl linear ransformaions. ~ ~ T A A B A N Y B A Y N Σ + # $ % + Σ roperies Mulivariae Gaussians Esseniall he same as in he -D case bu wih more general noaion ~ ~ ~ + Σ Σ Σ + Σ Σ + Σ + Σ Σ % & ' Σ Σ N p p N N

42 Linear Kalman Filer Esimaes he sae of a discree-ime conrolled process ha is governed b he linear sochasic difference equaion A + B u + ε wih a measuremen z C + δ

43 Componens of a Kalman Filer A B C ε δ Mari n n ha describes how he sae evolves from o + wihou conrols or noise. Mari n i ha describes how he conrol u changes he sae from o +. Mari k n ha describes how o map he sae o an observaion z. Random variables represening he process and measuremen noise ha are assumed o be independen and normall disribued wih covariance R and Q respecivel.

44 Kalman Filer Algorihm. Algorihm Kalman_filer - Σ - u z :. redicion: Σ A + B u A Σ A + 5. Correcion: T 6. K ΣC Reurn Σ T Σ I K C Σ R T C ΣC + Q K z C

45 Kalman Filer Updaes in D measuremen belief belief new belief measuremen old belief

46 Kalman Filer Updaes in D wih + Σ Σ # $ % Σ Σ + T T Q C C C K K C I C z K bel wih obs K K z K bel σ σ σ σ σ + " # $ + old belief new belief measuremen

47 Kalman Filer Updaes in D! " # + Σ Σ + T R A A B u A bel! " # + + ac a b u a bel σ σ σ old belief new belief measuremen new belief newes belief

48 Kalman Filer Updaes belief laes belief belief measuremen belief

49 The redicion-correcion-ccle! " # + Σ Σ + T R A A B u A bel! " # + + ac a b u a bel σ σ σ redicion

50 The redicion-correcion-ccle + Σ Σ # $ % Σ Σ + T T Q C C C K K C I C z K bel obs K K z K bel σ σ σ σ σ + " # $ + Correcion

51 The redicion-correcion-ccle + Σ Σ # $ % Σ Σ + T T Q C C C K K C I C z K bel obs K K z K bel σ σ σ σ σ + " # $ +! " # + Σ Σ + T R A A B u A bel! " # + + ac a b u a bel σ σ σ Correcion redicion

52 Kalman Filer Summar Highl efficien: olnomial in measuremen dimensionali k and sae dimensionali n: Ok n Opimal for linear Gaussian ssems! Man roboics ssems are nonlinear!

53 Overview The Tracking roblem Baes Filers aricle Filers Kalman Filers Using Kalman Filers

54 aricle Filers: Basic Idea p p... p z equali for n se of n paricles

55 aricle Filer Eplained

56 Basic aricle Filer Algorihm Iniializaion: n paricles [i] ~ p pariclefilers - { for i o n [i] ~ p [i] - } w [i] pz [i] endfor for i o n include [i] in wih probabili w [i] predicion imporance weighs resampling p z... u... η p z p u p z... u... d p p z... u...

57 Imporance Sampling Weigh samples: w f / g

58 Some Roboics Eamples Tracking Hands eople Mobile Robo localizaion eople localizaion Car localizaion Mapping

59 aricle Filer B Frank Dellaer

60 aricle Filers

61 aricle Filers

62 aricles Robusness

63 Facored sampling Represen he sae disribuion non-paramericall redicion: Sample poins from prior densi for he sae Correcion: Weigh he samples according o Y M. Isard and A. Blake CONDENSATION -- condiional densi propagaion for visual racking IJCV 9: d

64 aricle filering We wan o use sampling o propagae densiies over ime i.e. across frames in a video sequence A each ime sep represen poserior Y wih weighed sample se revious ime sep s sample se - Y - is passed o ne ime sep as he effecive prior M. Isard and A. Blake CONDENSATION -- condiional densi propagaion for visual racking IJCV 9:

65 aricle filering Sar wih weighed samples from previous ime sep Sample and shif according o dnamics model Spread due o randomness; his is prediced densi Y - Weigh he samples according o observaion densi Arrive a correced densi esimae Y M. Isard and A. Blake CONDENSATION -- condiional densi propagaion for visual racking IJCV 9:

66 Eample Mike Isard and Andrew Blake

67 Eample Mike Isard and Andrew Blake

68 Eample Mike Isard and Andrew Blake

69 aricle filering resuls hp://

70 Tracking issues Iniializaion Manual Background subracion Deecion

71 Tracking issues Iniializaion Obaining observaion and dnamics model Generaive observaion model: render he sae on op of he image and compare Discriminaive observaion model: classifier or deecor score Dnamics model: learn ver difficul or specif using domain knowledge

72 Tracking issues Iniializaion Obaining observaion and dnamics model redicion vs. correcion If he dnamics model is oo srong will end up ignoring he daa If he observaion model is oo srong racking is reduced o repeaed deecion

73 Tracking issues Iniializaion Obaining observaion and dnamics model redicion vs. correcion Daa associaion Wha if we don know which measuremens o associae wih which racks?

74 Daa associaion So far we ve assumed he enire measuremen o be relevan o deermining he sae In reali here ma be uninformaive measuremens cluer or measuremens ma belong o differen racked objecs Daa associaion: ask of deermining which measuremens go wih which racks

75 Daa associaion Simple sraeg: onl pa aenion o he measuremen ha is closes o he predicion

76 Daa associaion Simple sraeg: onl pa aenion o he measuremen ha is closes o he predicion Doesn alwas work

77 Daa associaion Simple sraeg: onl pa aenion o he measuremen ha is closes o he predicion More sophisicaed sraeg: keep rack of muliple sae/observaion hpoheses Can be done wih paricle filering This is a general problem in compuer vision here is no eas soluion