Computer Vision 2 Lecture 6

Transcription

1 Compuer Vision 2 Lecure 6 Beond Kalman Filers ( ) leibe@vision.rwh-aachen.de, sueckler@vision.rwh-aachen.de RWTH Aachen Universi, Compuer Vision Group hp://

2 Conen of he Lecure Single-Objec Tracking Baesian Filering Kalman Filers, EKF Paricle Filers Muli-Objec Tracking Visual Odomer Visual SLAM & 3D Reconsrucion 2

3 Toda: Beond Gaussian Error Models 3 Figure from Isard & Blake

4 Topics of This Lecure Recap: Kalman Filer Basic ideas Limiaions Exensions Paricle Filers Basic ideas Propagaion of general densiies Facored sampling Case sud Deecor Confidence Paricle Filer Role of he differen elemens 4

5 Recap: Tracking as Inference Inference problem The hidden sae consiss of he rue parameers we care abou, denoed X. The measuremen is our nois observaion ha resuls from he underling sae, denoed Y. A each ime sep, sae changes (from X - o X ) and we ge a new observaion Y. Our goal: recover mos likel sae X given All observaions seen so far. Knowledge abou dnamics of sae ransiions. X X 2 X Y Y 2 Y 5 Slide credi: Krisen Grauman

6 Slide credi: Svelana Lazebnik Recap: Tracking as Inducion Base case: Assume we have iniial prior ha predics sae in absence of an evidence: P(X 0 ) A he firs frame, correc his given he value of Y 0 = 0 Given correced esimae for frame : Predic for frame + Correc for frame + predic correc 6

7 7 Recap: Predicion and Correcion Predicion: Correcion: 0 0,,,, dx X P X X P X P Dnamics model Correced esimae from previous sep Slide credi: Svelana Lazebnik dx X P X P X P X P X P 0 0 0,,,,,, Observaion model Prediced esimae

8 Recap: Linear Dnamic Models Dnamics model Sae undergoes linear ransformaion D plus Gaussian noise x ~ N D x, d Observaion model Measuremen is linearl ransformed sae plus Gaussian noise ~ N M x, m 8 Slide credi: Svelana Lazebnik, Krisen Grauman

9 9 Recap: Consan Veloci (D Poins) Sae vecor: posiion p and veloci v Measuremen is posiion onl ) ( v v v p p v p x noise v p noise D x x 0 (greek leers denoe noise erms) noise v p noise Mx 0 Slide credi: Svelana Lazebnik, Krisen Grauman

10 0 Recap: Consan Acceleraion (D Poins) Sae vecor: posiion p, veloci v, and acceleraion a. Measuremen is posiion onl 2 2 ) ( ) ( ) ( a a a v v a v p p a v p x noise a v p noise D x x (greek leers denoe noise erms) noise a v p noise Mx 0 0 Slide credi: Svelana Lazebnik, Krisen Grauman

11 Recap: General Moion Models Assuming we have differenial equaions for he moion E.g. for (undampened) periodic moion of a linear spring 2 d p d 2 p Subsiue variables o ransform his ino linear ssem p p 2 dp d p p2 p3 2 d d Then we have p p ( ) p 2 ( ) p x p p p, 2, 3, p p, 2, 3,, p p 2,, ( ) p 2, 3, 2 3, D

12 Recap: The Kalman Filer Know correced sae from previous ime sep, and all measuremens up o he curren one Predic disribuion over nex sae. Receive measuremen Know predicion of sae, and nex measuremen Updae disribuion over curren sae. P X 0,, Time updae ( Predic ) Measuremen updae ( Correc ) P X,, 0 Mean and sd. dev. of prediced sae:, Time advances: ++ Mean and sd. dev. of correced sae:, 2 Slide credi: Krisen Grauman

13 Recap: General Kalman Filer (>dim) PREDICT CORRECT x D x D K T M M T D x x K Mx d I KM M T m residual Kalman gain 3 for derivaions, see F&P Chaper 7.3 Slide credi: Krisen Grauman More weigh on residual when measuremen error covariance approaches 0. Less weigh on residual as a priori esimae error covariance approaches 0.

14 Resources: Kalman Filer Web Sie hp:// Elecronic and prined references Book liss and recommendaions Research papers Links o oher sies Some sofware News Java-Based KF Learning Tool On-line D simulaion Linear and non-linear Variable dnamics 4 Slide adaped from Greg Welch

15 Remarks Tr i! No oo hard o undersand or program Sar simple Experimen in D Make our own filer in Malab, ec. Noe: he Kalman filer wans o work Debugging can be difficul Errors can go un-noiced 5 Slide adaped from Greg Welch

16 Topics of This Lecure Recap: Kalman Filer Basic ideas Limiaions Exensions Paricle Filers Basic ideas Propagaion of general densiies Facored sampling Case sud Deecor Confidence Paricle Filer Role of he differen elemens 6

17 Exension: Exended Kalman Filer (EKF) Basic idea Sae ransiion and observaion model don need o be linear funcions of he sae, bu jus need o be differeniable. x = g x, u + ε = h x + δ The EKF esseniall linearizes he nonlineari around he curren esimae b a Talor expansion. Properies Unlike he linear KF, he EKF is in general no an opimal esimaor. If he iniial esimae is wrong, he filer ma quickl diverge. Sill, i s he de-faco sandard in man applicaions Including navigaion ssems and GPS 7

18 Recap: Kalman Filer Deailed Algorihm Algorihm summar Assumpion: linear model Predicion sep Correcion sep 8

19 Exended Kalman Filer (EKF) Algorihm summar Nonlinear model wih he Jacobians Predicion sep Correcion sep 9

20 Kalman Filer Oher Exensions Unscened Kalman Filer (UKF) Used for models wih highl nonlinear predic and updae funcions. Here, he EKF can give ver poor performance, since he covariance is propagaed hrough linearizaion of he non-linear model. Idea (UKF): Propagae jus a few sample poins ( sigma poins ) around he mean exacl, hen recover he covariance from hem. More accurae resuls han he EKF s Talor expansion approximaion. Ensemble Kalman Filer (EnKF) Represens he disribuion of he ssem sae using a collecion (an ensemble) of sae vecors. Replace covariance marix b sample covariance from ensemble. Sill basic assumpion ha all prob. disribuions involved are Gaussian. EnKFs are especiall suiable for problems wih a large number of variables. 20

21 Even More Exensions Swiching Linear Dnamic Ssem (SLDS) Use a se of k dnamic models A (),...,A (k), each of which describes a differen dnamic behavior. Hidden variable z deermines which model is acive a ime. A swiching process can change z according o disribuion. 2 Figure source: Erik Sudderh

22 Topics of This Lecure Recap: Kalman Filer Basic ideas Limiaions Exensions Paricle Filers Basic ideas Propagaion of general densiies Facored sampling Case sud Deecor Confidence Paricle Filer Role of he differen elemens Toda: onl main ideas Formal inroducion nex lecure 22

23 When Is A Single Hpohesis Too Limiing? Iniial posiion Predicion Measuremen Updae x x x x Consider his example: sa we are racking he face on he righ using a skin color blob o ge our measuremen. Video from Jojic & Fre 23 Slide credi: Krisen Grauman Figure from Thrun & Kosecka

24 Propagaion of General Densiies 24 Figure from Isard & Blake

25 25 Facored Sampling Idea: Represen sae disribuion non-paramericall Predicion: Sample poins from prior densi for he sae, P(X) Correcion: Weigh he samples according o P(Y X) dx X P X P X P X P X P 0 0 0,,,,,, Slide credi: Svelana Lazebnik

26 Paricle Filering (Also known as Sequenial Mone Carlo Mehods) Idea We wan o use sampling o propagae densiies over ime (i.e., across frames in a video sequence). A each ime sep, represen poserior P(X Y ) wih weighed sample se. Previous ime sep s sample se P(X - Y - ) is passed o nex ime sep as he effecive prior. 26

27 Paricle Filering Man variaions, one general concep: Represen he poserior pdf b a se of randoml chosen weighed samples (paricles) Poserior Sample space Randoml Chosen = Mone Carlo (MC) As he number of samples become ver large he characerizaion becomes an equivalen represenaion of he rue pdf. 27 Slide adaped from Michael Rubinsein

28 Paricle Filering Sar wih weighed samples from previous ime sep Sample and shif according o dnamics model Spread due o randomness; his is prediced densi P(X Y - ) Weigh he samples according o observaion densi Arrive a correced densi esimae P(X Y ) 28 M. Isard and A. Blake, CONDENSATION -- condiional densi propagaion for visual racking, IJCV 29():5-28, 998 Slide credi: Svelana Lazebnik

29 Paricle Filering Visualizaion 29 Code and video available from hp://

30 Paricle Filering Resuls 30 hp://

31 Videos from Isard & Blake Paricle Filering Resuls Some more examples hp:// 3

32 Obaining a Sae Esimae Noe ha here s no explici sae esimae mainained, jus a cloud of paricles Can obain an esimae a a paricular ime b quering he curren paricle se Some approaches Mean paricle Weighed sum of paricles Confidence: inverse variance Reall wan a mode finder mean of alles peak 32

33 Condensaion: Esimaing Targe Sae From Isard & Blake, 998 Sae samples (hickness proporional o weigh) Mean of weighed sae samples 33 Slide credi: Marc Pollefes Figures from Isard & Blake

34 Summar: Paricle Filering Pros: Able o represen arbirar densiies Converging o rue poserior even for non-gaussian and nonlinear ssem Efficien: paricles end o focus on regions wih high probabili Works wih man differen sae spaces E.g. ariculaed racking in complicaed join angle spaces Man exensions available 34

35 Summar: Paricle Filering Cons / Caveas: #Paricles is imporan performance facor Wan as few paricles as possible for efficienc. Bu need o cover sae space sufficienl well. Wors-case complexi grows exponeniall in he dimensions Mulimodal densiies possible, bu sill single objec Ineracions beween muliple objecs require special reamen. No handled well in he paricle filering framework (sae space explosion). 35

36 Topics of This Lecure Recap: Kalman Filer Basic ideas Limiaions Exensions Paricle Filers Basic ideas Propagaion of general densiies Facored sampling Case sud Deecor Confidence Paricle Filer Role of he differen elemens 36

37 Challenge: Unreliable Objec Deecors Example: Low-res webcam fooage (320240), MPEG compressed Deecor inpu Tracker oupu 37 How o ge from here? o here?

38 Tracking based on Deecor Confidence (using ISM deecor) Deecor oupu is ofen no perfec Missing deecions and false posiives Bu coninuous confidence sill conains useful cues. (using HOG deecor) Idea pursued here: Use coninuous deecor confidence o rack persons over ime. 38

39 Main Ideas Deecor confidence paricle filer Iniialize paricle cloud on srong objec deecions. Propagae paricles using coninuous deecor confidence as observaion model. Disambiguae beween differen persons Train a person-specific classifier wih online boosing. Use classifier oupu o disinguish beween nearb persons. 39 [Breiensein, Reichlin, Leibe e al., ICCV 09]

40 Deecor Confidence Paricle Filer Sae: Moion model (consan veloci) Observaion model Discree deecions Deecor confidence Classifier confidence 40

41 When Is Which Term Useful? Discree deecions Deecor confidence Classifier confidence 4

42 Each Observaion Term Increases Robusness! Deecor onl CLEAR MOT scores 42

43 Each Observaion Term Increases Robusness! Deecor + Confidence CLEAR MOT scores 43

44 Each Observaion Term Increases Robusness! Deecor + Classifier CLEAR MOT scores 44

45 Each Observaion Term Increases Robusness! Deecor + Confidence + Classifier False negaives, false posiives, and ID swiches decrease! CLEAR MOT scores 45

46 Qualiaive Resuls 46

47 Remaining Issues Some false posiive iniializaions a wrong scales Due o limied scale range of he person deecor. Due o boundar effecs of he person deecor. 47

48 References and Furher Reading A good uorial on Paricle Filers M.S. Arulampalam, S. Maskell, N. Gordon, T. Clapp. A Tuorial on Paricle Filers for Online Nonlinear/Non-Gaussian Baesian Tracking. In IEEE Transacions on Signal Processing, Vol. 50(2), pp , The CONDENSATION paper M. Isard and A. Blake, CONDENSATION - condiional densi propagaion for visual racking, IJCV 29():5-28,