Automatic Object Trajectory- Based Motion Recognition Using Gaussian Mixture Models

Transcription

1 Automatc Object Trajectory- Based Moton Recognton Usng Gaussan Mxture Models Fasal I. Bashr, Ashfaq A. Khokhar, Dan Schonfeld Electrcal and Computer Engneerng, Unversty of Illnos at Chcago. Chcago, IL, USA.

2 Motvaton - Importance of trajectory-based processng - In poor qualty vdeos (survellance), features such as face, appearance, color, etc are not vsble. - Need to process data from Non- vdeo sensors: e.g., wred- gloves, radar, GPS, CNS. - Major Applcaton areas - Vdeo Survellance. (Intellgent Vdeo) - Sgn Language Recognton. - Sports Vdeo Analyss for Teams and Vewers. - Anmal Moblty Experments. - Movng Object Databases.

3 Problem Statement Develop Scalable Classfcaton Algorthms for Recognton of of Trajectores obtaned from Vdeo or or Non-Vdeo sources. Optmally- Compact Representaton at Low- Easly Extendble to Hgh- Level Not ted to Vdeo Sensors Level

4 Related Work Trajectory Indexng and Retreval Sahoura & Zakhor [ICIP 99] X and Y- separately processed usng Haar Wavelets. Frst 8 coeffs. stored as ndex. Eucldean Dstance. W. Chen, S.F. Chang [SPIE 00] Wavelet- based segmentaton; Feature vector wth acceleraton, velocty, subtrajectory length, etc. Le-Chen & Ora [ACM MIR 04] X and Y- transformed to Movement Sequence quantzed nto 8x8 bns; Normalzed Edt Dstance.

5 Related Work Trajectory Modelng and Recognton Rao, Shah [IJCV 04] Vew- nvarant representaton of actons based on curvature; For each dynamc nstant, frame number, locaton of hand and sgn of nstant stored; Matchng done on trajectores wth same number of nstants and same sgn permutatons. Vaswan, Chellapa [IEEE TIP, to appear] Model Actvty performed by a group of movng and nteractng objects; Objects n vdeo taken as ponts and shape formed by these ponts s tracked over tme; Abnormalty detected as perturbaton n ths shape.

6 Challenges & Proposed Solutons Compact Indexng Use Prncpal Component Analyss (PCA)- based representaton. (Optmal Energy Compacton) Partal Trajectores Due to Occluson, Nose, etc. Segment trajectores nto small chunks of of subtrajectores. Estmate Hgh- Dmensonal Multmodal PDFs of Actvty Classes Use Gaussan Mxture Models to to Estmate arbtrarly Complex PDFs.

7 Outlne Trajectory Segmentaton usng Hypothess Testng on Curvature PCA-Based Representaton Gaussan Mxture Models for Class Densty Estmaton

8 Trajectory Segmentaton - Motvaton Partal Queres can be answered. If one porton of trajectory s not avalable due to occluson, etc. If two objects follow the same pattern of moton for a whle and then go ther dfferent ways. Implct Dmensonalty Reducton

9 Segmentaton usng Hypothess Testng based on Curvature Segmentaton s based on Curvature: x ( n) y( n) x( n) y ( n) κ ( n) = 3 ( x ( n) 2 + y ( n) 2 ) 2 Two non-overlappng wndows from curvature Lkelhood Rato Test: The two wndows come from same dstrbuton? Fnd DstnctMaxmas n Dstance measure to locate Segmentaton Ponts. Segmented Subtrajectores are normalzed for spatal nvarance.

10 Hypothess Testng on Curvature Lkelhood Rato Test 1 ( x µ ) 1 ( x µ ) LX ( ; ) exp( ). ( ; ) exp( ) θ1 = LYθ 2 2 = 2 2πσ 2σ 1 1 2πσ 2σ ( x µ 3) ( ; θ3) = exp( ). θ (, ) 2 = µ σ 2πσ 2σ ( x µ 3) 0 = exp( ) 2 2πσ 2σ 3 3 LZ L L λ = πσ 2 σ 1σ 2 1 σ2 L = L L ( x µ ) ( x µ ) exp( ) σσ ( x µ 3) ( x µ 1) ( x µ 2) d( X, Y) = log( λl ) = log 2π + [ ] σ 2 σ σ σ

11 Segmentaton Results Segmentaton of trajectores from dfferent sgners. Norway (a) (b); Alve (c) (d)

12 Prncpal Component Analyss Data-dependant Orthonormal bases (PCs) as opposed to generc bases n DFT,DWT etc. Let X be a vector of p-random varables: lnear functon α 1 x of the elements of x wth maxmum varance. lnear functon α 2 x, uncorrelated wth α 1 x, wth maxmum varance, and so on. If Covarance matrx s known then k th PC s ts egenvector correspondng to k th largest egenvalue.

13 Prncpal Component Analyss Projecton: y = Φ q x Y s maxmally uncorrelated: det( Σ y ) s maxmzed. How many PCs to be retaned? t m = 100 m j = 1 p j = 1 λ λ j j

14 PCA Based Combned X- and Y- Representaton Segment based on 2-D spato-temporal curvature Represent both x- and y- usng sngle set of PCA Coeffcents Trajectory data from x- and y- projectons for each segment s stacked to form one vector per subtrajectory PCA s performed on these stacked vectors PCA feature vectors used to Tran GMMs.

15 Gaussan Mxture Models N c P( y Θ ) = π (y; µ, ) = 1 (y; µ, ) µ : M-dmensonal Gaussan densty : Mean Vector : Covarance Matrx π : Mxng parameters of the Gaussan components, satsfyng π = 1

16 Expectaton Maxmzaton for GMM Parameter Estmaton E-Step: M-Step: π (y; µ, ) k t k k k = Nc k t k k π j µ j j = 1 h(t) (y;, ) N T k 1 t 1 π + = = N N c T = 1 t= 1 h(t) k h(t) k N T k 1 t 1 µ + = = N T t = 1 h(t)y k h(t) k t = N T k + 1 t = 1 h(t)(y µ )( y µ ) k t k 1 t k 1 T N T t = 1 h(t) k

17 Tranng and Test Data Sets ASL I: 207 Trajectores from 3 classes n Australan Sgn Language. Tranng on half; Testng on rest half. ASL II: Same as above. Tranng on half; Testng on all. HJSL I: 108 Trajectores from Hgh Jump and Slalom Skng Dataset. Tranng on half; Testng on rest half. HJSL II: Same as above. Tranng on half; Test on all.

18 Results - GMM Learnng 1-Sgma contours of GMM s learnt from three classes n Australan Sgn Language Dataset. (a) Norway. (b) Alve. (c) Crazy.

19 Results - Classfcaton ROC curves for Three Classfers usng ASL II dataset for: (a) Class 1 Norway. (b) Class 2 Alve. (c) Class 3 Crazy. (d) Average performance across all classes.

20 Results - Accuracy accuracy =1- false alarms test set Method ASL I ASL II HJSL I HJSL II GMM PCA Densty Estmaton GMM Global Classfcaton Accuracy Results for Three classfers n Four expermental setups.

21 Questons??? Contact Informaton : Fasal I. Bashr. fbashr@ece.uc.edu