Object Tracking. Computer Vision Jia-Bin Huang, Virginia Tech. Many slides from D. Hoiem

Transcription

1 Objec Tracking Compuer Vision Jia-Bin Huang Virginia Tech Man slides from D. Hoiem

2 Adminisraive suffs HW 5 (Scene caegorizaion) Due :59pm on Wed November 6 oll on iazza When should we have he final exam? Dec 6 Dec 3

3 Toda s class Explain HW 5 in deail Review/finish objec deecion Tracking Objecs Examples and Applicaions Overview of probabilisic racking Kalman Filer aricle Filer

4 HW 5 Color hisogram and k-neares neighbor (knn) classifier Bag of visual words model and neares neighbor classifier Bag of visual words model and a discriminaive classifier Spaial pramid model and a discriminaive classifier

5 Review: Saisical emplae Objec model = log linear model of pars a fixed posiions = -.5? > 7.5 Non-objec =.5? > 7.5 Objec

6 Review: Saisical emplaes A. ropose Window B. Exrac Feaures C. Classif D. osprocess Sliding window: scan image pramid HOG SVM Boosed subs Neural nework Non-max suppression Segmen or refine localizaion Fas randomized feaures Region proposals: edge/region-based resize o fixed window CNN feaures

7 Review: ar-based Models ar. Sar-shaped model Example: Deformable ars Model Felzenswalb e al. 2 ar ar Roo ar ar 2. Tree-shaped model Example: icorial srucures Felzenszwalb Huenlocher Sequenial predicion models

8 Deformable Laen ars Model (DM) Deecions Templae Visualizaion Felzenszwalb e al. 28 2

9 icorial Srucures Model Appearance likelihood Geomer likelihood

10 icorial Srucures Model Opimizaion is rick bu can be efficien For each l find bes l 2 : Remove v 2 and repea wih smaller ree unil onl a single par For k pars n locaions per par his has complexi of O(kn 2 ) bu can be solved in ~O(kn) using generalized disance ransform

11 Tracking Objecs Goal: Locaing a moving objec/par across video frames Examples and Applicaions Overview of visual racking Moion models: probabilisic racking Kalman Filer aricle Filer

12 Tracking examples Traffic Spors Face Bod

13 Furher applicaions Moion capure Augmened Reali Acion Recogniion Securi raffic monioring Video Compression Human-compuer ineracion Video Summarizaion Medical Screening

14 Tracking Examples Traffic: hps:// Soccer: hp:// Face: hp:// Bod: hps:// Ee: hp:// Gaze: hp://

15 Things ha make visual racking difficul Small few visual feaures Erraic movemens moving ver quickl Occlusions leaving and coming back Surrounding similar-looking objecs

16 Sraegies for racking Tracking b repeaed deecion Works well if objec is easil deecable (e.g. face or colored glove) and here is onl one Need some wa o link up deecions Bes ou can do if ou can predic moion

17 Tracking wih dnamics Ke idea: Based on a model of expeced moion predic where objecs will occur in nex frame before even seeing he image Resric search for he objec Measuremen noise is reduced b rajecor smoohness Robusness o missing or weak observaions

18 Sraegies for racking Tracking wih moion predicion redic he objec s sae in he nex frame Kalman filering: nex sae can be linearl prediced from curren sae (Gaussian) aricle filering: sample muliple possible saes of he objec (non-parameric good for cluer)

19 General model for racking sae : The acual sae of he moving objec ha we wan o esimae Sae could be an combinaion of posiion pose viewpoin veloci acceleraion ec. observaions Y: Our acual measuremen or observaion of sae. Observaion can be ver nois A each ime he sae changes o and we ge a new observaion Y

20 Seps of racking redicion: Wha is he nex sae of he objec given pas measuremens? Y Y

21 Seps of racking redicion: Wha is he nex sae of he objec given pas measuremens? Correcion: Compue an updaed esimae of he sae from predicion and measuremens Y Y Y Y Y

22 Simplifing assumpions Onl he immediae pas maers dnamics model

23 Simplifing assumpions Onl he immediae pas maers dnamics model Measuremens depend onl on he curren sae Y Y Y Y observaion model

24 Simplifing assumpions Onl he immediae pas maers dnamics model Measuremens depend onl on he curren sae Y Y Y Y observaion model 2 - Y Y Y 2 Y - Y

25 roblem saemen We have models for Likelihood of nex sae given curren sae: Likelihood of observaion given he sae: Y We wan o recover for each :

26 robabilisic racking Base case: Sar wih iniial prior ha predics sae in absence of an evidence: ( ) For he firs frame correc his given he firs measuremen: Y =

27 robabilisic racking Base case: Sar wih iniial prior ha predics sae in absence of an evidence: ( ) For he firs frame correc his given he firs measuremen: Y = ) ( ) ( ) ( ) ( ) ( ) ( Y

28 robabilisic racking Base case: Sar wih iniial prior ha predics sae in absence of an evidence: ( ) For he firs frame correc his given he firs measuremen: Y = Given correced esimae for frame -: redic for frame Observe ; Correc for frame predic correc

29 redicion redicion involves represening given d d d Law of oal probabili

30 redicion d d d Condiioning on redicion involves represening given

31 redicion d d d Independence assumpion redicion involves represening given

32 redicion d d d dnamics model correced esimae from previous sep redicion involves represening given

33 Correcion Correcion involves compuing given prediced value

34 Correcion d Baes Rule Correcion involves compuing given prediced value

35 d Correcion Independence assumpion (observaion direcl depends onl on sae ) Correcion involves compuing given prediced value

36 d Correcion Condiioning on Correcion involves compuing given prediced value

37 Correcion observaion model prediced esimae normalizaion facor d Correcion involves compuing given prediced value

38 Summar: redicion and correcion redicion: Correcion: d d dnamics model correced esimae from previous sep observaion model prediced esimae

39 The Kalman filer Linear dnamics model: sae undergoes linear ransformaion plus Gaussian noise Observaion model: measuremen is linearl ransformed sae plus Gaussian noise The prediced/correced sae disribuions are Gaussian You onl need o mainain he mean and covariance The calculaions are eas (all he inegrals can be done in closed form)

40 Example: Kalman Filer Ground Truh Observaion Correcion redicion Nex Frame Updae Locaion Veloci ec.

41 Comparison Ground Truh Observaion Correcion

42 ropagaion of Gaussian densiies Curren sae Expeced change Decen model if here is jus one objec bu localizaion is imprecise Uncerain Observaion and Correcion

43 aricle filering Represen he sae disribuion non-paramericall redicion: Sample possible values - for he previous sae Correcion: Compue likelihood of based on weighed samples and ( ) M. Isard and A. Blake CONDENSATION -- condiional densi propagaion for visual racking IJCV 29():

44 aricle filering Sar wih weighed samples from previous ime sep Sample and shif according o dnamics model Spread due o randomness; his is prediced densi ( Y - ) Weigh he samples according o observaion densi Arrive a correced densi esimae ( Y ) M. Isard and A. Blake CONDENSATION -- condiional densi propagaion for visual racking IJCV 29():

45 ropagaion of non-parameric densiies Curren sae Expeced change Good if here are muliple confusable objecs (or cluer) in he scene Uncerain Observaion and Correcion

46 aricle filering resuls eople: hp:// Hand: hp:// Localizaion (similar model): hps:// hp:// Good informal explanaion: hps://

47 Tracking issues Iniializaion Manual Background subracion Deecion

48 Tracking issues Iniializaion Geing observaion and dnamics models Observaion model: mach a emplae or use a rained deecor Dnamics model: usuall specif using domain knowledge

49 Tracking issues Iniializaion Obaining observaion and dnamics model Uncerain of predicion vs. correcion If he dnamics model is oo srong will end up ignoring he daa If he observaion model is oo srong racking is reduced o repeaed deecion Too srong dnamics model Too srong observaion model

50 Tracking issues Iniializaion Geing observaion and dnamics models redicion vs. correcion Daa associaion When racking muliple objecs need o assign righ objecs o righ racks (paricle filers good for his)

51 Tracking issues Iniializaion Geing observaion and dnamics models redicion vs. correcion Daa associaion Drif Errors can accumulae over ime

52 Drif D. Ramanan D. Forsh and A. Zisserman. Tracking eople b Learning heir Appearance. AMI 27.

53 Sae-of-he-ar objec racking VOT 26 Objec racking benchmark AMI5

54 Things o remember Tracking objecs = deecion + predicion robabilisic framework redic nex sae Updae curren sae based on observaion Two simple bu effecive mehods Kalman filers: Gaussian disribuion aricle filers: mulimodal disribuion