Detection and Classification of Interacting Persons

Transcription

1 Deecon and Classfcaon of Ineracng Persons Sco Blunsden (1), Rober Fsher (2) (1) European Commsson Jon Research Cenre, Ialy (2) Unversy of Ednburgh, Scoland ABSTRACT Ths chaper presens a way o classfy neracons beween people. Examples of he neracons we nvesgae are; people meeng one anoher, walkng ogeher and fghng. A new feaure se s proposed along wh a correspondng classfcaon mehod. Resuls are presened whch show he new mehod performng sgnfcanly beer han he prevous sae of he ar mehod as proposed by [Olver e al., 2000]. INTRODUCTION Ths chaper presens an nvesgaon no classfcaon of mulple person neracons. There has been much prevous work upon denfyng wha acvy ndvdual people are engaged n. [Davs and Bobck, 2001] used a momen based represenaon based on exraced slhouees and [Efros e al., 2003] modeled human acvy by generang opcal flow descrpons of a person s acon. Descrpons were generaed by frs hand-rackng an ndvdual, re-scalng o a sandard sze and hen akng he opcal flow of a persons acons over several frames. A daabase of hese descrpons was creaed and mached o novel suaons. Ths mehod was exended by [Roberson, 2006] who also ncluded locaon nformaon o help gve conexual nformaon o a scene. Locaon nformaon s of asssance when ryng o deermne f someone s loerng or merely wang a a road crossng. Followng on from flow based feaures [Dollar e al., 2005] exraced spaal-emporal feaures o denfy sequences of acons. Rbero and Sanos-Vcor [Rbero and Sanos-Vcor, 2005] ook a dfferen approach o classfy an ndvdual s acons n ha hey used mulple feaures calculaed from rackng (such as speed, egenvecors of flow) and seleced hose feaures whch bes classfed he persons acons usng a classfcaon ree wh each branch usng a mos 3 feaures o classfy he example. The classfcaon of neracng ndvduals was suded by [Olver e al., 2000] who used rackng o exrac he speed, algnmen and dervave of he dsance beween wo ndvduals. Ths nformaon was hen used o classfy sequences usng a coupled hdden Markov model (CHMM). [Lu and Chua, 2006] expanded he wo person classfcaon o hree person sequences usng a hdden Markov model (HMM) wh an explc role arbue. Informaon derved from rackng was used o provde feaures such as he relave angle beween wo persons o classfy complee sequences. Xang and Gong [Gong and Xang, 2003] agan used a CHMM o model

2 neracons beween vehcles on an arcraf runway. These feaures are calculaed by deecng sgnfcanly changed pxels over several frames. The correc model for represenng he sequence s deermned by he connecons beween he separae models. Goodness of f s calculaed by he Bayesan nformaon creron. Usng hs mehod a model represenng he sequences acons s deermned. Mul-person neracons whn a rgd formaon was also he goal of [Khan and Shah, 2005] who used a geomerc model o deec rgd formaons beween people, such an example would be a marchng band. [Inlle and Bobck, 2001] used a pre-defned Bayesan nework o descrbe planned moons durng Amercan fooball games. Ohers such as [Perse e al., 2007] also use a pre-specfed emplae o evaluae he curren acon beng performed by many ndvduals. Prespecfed emplaes have been used by [Van Vu e al., 2003, Hongeng and Nevaa, 2001] whn he conex of survellance applcaons. SPECIFICALLY WHAT ARE WE TRYING TO DO? Gven an npu vdeo sequence he goal s o auomacally deermne f any neracons beween wo people are akng place. If any are akng place hen we wan o denfy he class of he neracon. Here we lm ourselves o pre-defned classes whch have been prevously labeled. To make he suaon more realsc here s also a no neracon class. We seek o gve each frame a label from a predefned se. For example a label may be ha person 1 and person 2 are walkng ogeher n frame 56. The ably o auomacally classfy such neracons would be useful n cases whch are ypcal of many survellance suaons. Such an ably o auomacally recognze neracons would also be useful n vdeo summarzaon where could be possble o focus only on specfc neracons. FEATURES AND VARIABLES Vdeo daa s rch n nformaon wh he resoluon of modern survellance cameras capable of delverng megapxel resoluon a a susaned frame rae of greaer han 10fps. Such daa s overwhelmng and mosly unnecessary for classfcaon of neracons. As a frs sep, rackng of he ndvduals s employed. There s a rch body of work upon rackng of people and obecs whn he leraure. See he revew by Ylmaz e al. for a good survey of curren rackng echnology [Ylmaz, A e al. 2006]. For all expermens performed n hs chaper he boundng box mehod of denfyng a person was used. The posonal nformaon was calculaed based upon he cenre of hs box. Ths process s llusraed n Fgure 1. Such rackng nformaon s ypcal of he oupu of many rackng procedures and wll be assumed ha such a racker s avalable hroughou all expermens carred ou n hs chaper.

3 Fgure 1 - Boundng box rackng. Colored lnes show he prevous poson of he cenre of he racked obec. Here hree ypes of feaures are used as npu o a classfer. We make use of movemen, algnmen and dsance based feaures. More deals are gven n he followng secons. Movemen Based feaures Movemen plays an mporan role n recognzng neracons. The speed of an ndvdual s calculaed as shown n equaon (1.1). The double vercal bar (.. ) represens a vecor L2 norm as gven by x = sqr( x1+ x x n ) where x n refers o he nh componen of vecor x. s w = p p (1.1) w Here p refers o he poson of he racked obec a me for obec. Whn hs work only he wo dmensonal ( p, = x y ) case s consdered due o rackng nformaon only beng avalable n wo dmensons. The emporal offse w s nroduced due o he hgh raes whch ypfy many modern vdeo cameras. Wh frame raes of around 25 frames per second hs can mean ha dfferences beween he curren and las frame (w=1) are very small and may be domnaed by nose. The absolue dfference n speed ( ε ) beween wo racks s also calculaed ( s s ). The vorcy ( v ) s measured as a devaon from a lne. Ths lne s calculaed by fng a lne o a w se of prevous posons of he raecory P = p,.., p. The wndow sze w s he same as ha used n equaon (1.1). A each pon he orhogonal dsance o he lne s found. The oal dsance of all pons are hen summed and normalzed by wndow lengh and so gve a measure of he vorcy.

4 Algnmen Based feaures The algnmen of wo racks can gve valuable nformaon as o how hey are neracng. The degree of algnmen s common o [Ggerenzer e al., 1999; Olver e al., 2000; and Lu and Chua, 2006] who all make use of such nformaon when classfyng raecory nformaon. To calculae he do produc he headng (h) of he obec s aken as n equaon (1.2) and he do produc (1.3) s calculaed from he drecons of racks and. hˆ p p w = w p p a ˆ ˆ = (1.2) h h (1.3) In addon o he algnmen beween wo people he poenal nersecon ( ) of wo γ Traecores s also calculaed. Such feaures are suggesed n [Ggerenzer e al., 1999] and [Lu and Chua, 2006]. We frs es for an nersecon of he headngs. Ths s acheved as shown n Algorhm 1. d = p p c f c δ l = h h c elsef c f / /dsance beween arges and a me / /cross produc of he headngs of and a me 0 / /non zero cross produc d = h = p + δ h / /hey nesec a hs pon = 0 / /parallel bu may be n oppose drecon d > 0 and h = h and h h > 0 = h + h r δ l d = h r c = p + δ h / /hey nesec a hs pon else hey do no mee end Algorhm 1- Algorhm o deermne he meeng of wo raecores

5 Dsance Based feaures Dsance s a good measure for many ypes of neracon, for example meeng s no possble whou beng n close physcal proxmy. Frs he Eucldean dsance s aken and s used as gven n equaon (1.4) below. d = p p (1.4) The dervave of he dsance was also calculaed. Ths s he dfference n dsance a conguous me seps. I s calculaed as shown n equaon (1.5) below. dˆ d w (1.5) = ( w) An nsananeous measure such as he dsance and he dervave of he dsance can boh be prone o shor erm rackng errors. In an effor o remove hs effec a wndow sze conanng w pons was averaged (as n P n he prevous secon). The dsance was calculaed for every pon (as n equaon (1.4)) n hs wndow. Fnal feaure vecor The fnal feaure vecor for each par of people s gven n equaon (1.6) below: ˆ ˆ ˆ r = s s, s s, ε, a, d, d, v v, γ (1.6) The vecor beween persons and a me s made up of he speed of each person ( s s ) and he change n speed ˆ, ˆ. The algnmen, dsance and change n dsance a a parcular s s pon n me s gven by and a, d ˆ. The vorcy a a parcular pon n me for each d person s gven by whls he possbly of an nersecon beween wo raecores s gven v v by. Ths gves a fnal feaure vecor conanng 11 elemens. A furher processng sep of γ, normalzng he ranng daa o have zero mean and un varance was also aken.

6 Observaon Wndow Sze Throughou hese expermens we nvesgaed he role of varyng he number of vdeo frames used before makng a decson as o wha s happenng whn he frame. Fgure 2 (below) shows how hs s acheved. Throughou hs work we used nformaon from before and afer he curren frame n order o classfy. Typcally he frames relae o a small quany of me (1-2 seconds) and help wh he lag problem when your decson s based by he large amoun of prevous nformaon used n he classfcaon. The wndow sze varaon hroughou hs work s equvalen o a few seconds delay. Ths was no foreseen as a problem f such an approach was aken n a real survellance applcaon. The fac ha here would be a lag n classfcaon f makng use of only prevous nformaon seems an approprae rade-off for an ncrease n accuracy. Fgure 2 - The frame o classfy () uses nformaon from w frames around he curren frame n order o classfy he frame. CLASSIFICATION Ths secon nroduces he classfers whch are used hroughou subsequen expermens. We make use of a smple lnear dscrmnan classfer (LDA) whch s non-probablsc and provdes a baselne for performance. Ths s nroduced n he nex secon. We hen brefly nroduce he hdden Markov model (HMM) whch s wdely used hroughou he leraure. We hen nroduce a newer model, he condonal random feld (CRF). Fnally he prevous bes mehod as suggesed by [Olver, 2000] s brefly revewed.

7 Lnear Dscrmnan Classfer Lnear dscrmnan analyss (LDA) seeks o maxmze he obecve funcon gven n equaon (1.7) below. Here SW s he whn class scaer marx wh S B beng he beween class scaer marx. T B J ( w) = wsw T ws w (1.7) The obecve funcon (equaon (1.7)) s ofen referred o as he sgnal o nose rao. Wha we are ryng o acheve s a proecon (w) whch maxmzes he dsance of he class means relave o he (sum of) varances of a parcular class. To generae a soluon s noed ha equaon (1.7) has a propery whereby s nvaran wh respec o scalng of he w vecors (eg T w αwwhere α s some arbrary scalng). Therefore w can be chosen such ha wsw w = 1. I s also common (and ndeed he case here) o perform a whenng sep (zero mean and un varance) on he daa pror o npu no hs mehod. Ths maxmzaon can be urned no a regular egenvalue problem. Proecons are found for each class (e one class vs all ohers) and he mean and varance (C) of he class proecon are found. In order o classfy a novel pon he new pon s proeced wh S. The class label s deermned by akng he smalles Mahalonobs dsance beween he calculaed class model s mean and varance and he new es pon. Hdden Markov Model Hdden Markov models (HMM s) have been nroduced by (among ohers) Rabner [Rabner,1990]. The model s parameerzed by he pror dsrbuon wh each elemen π represenng π = p( x= ) across all hdden saes [1,.., N]. The saonary sae ranson marx A s used o represen he probably of a ranson from one sae () o anoher () hrough me. An enry whn he sochasc marx A s referenced by a = p( x= x 1 = ). Whn hs work we are concerned wh connuous real valued npus ( r ) whch can be accommodaed whn he model by usng a Gaussan mxure model o represen he npu dsrbuon p( r x = ). W 1 W M b ( r) = c N( r, μ, C )(1.8), m, m, m m= 1 Here he observed daa R s he vecor beng modeled, c, m mh mxure n sae. N s a Gaussan dsrbuon wh mean vecor for he mh mxure componen n sae. The mxure coeffcen s he mxure coeffcen for he c μ, m and covarance mus sum o 1. C, m

8 The hdden Markov model s parameers can hus be represened as λ = ( Π, A, Θ) where Θ represen he parameers of he mxure model. Condonal Random Feld In hs secon he workngs of a condonal random feld (CRF) are explaned and hen he specfc formulaon as appled n hs chaper s gven. The srucure of he CRF we use s shown n Fgure 3. The CRF can be confgured o resemble HMM lke models. However hey can be more expressve n ha arbrary dependences on he observaons are allowed. Usng he feaure funcons of he CRF allows any par of he npu sequence o be used a any me durng he nference sage. I s also possble ha dfferen sae s (classes) can have dfferng feaure funcons (hough we do no make use of hs here). The feaure funcons descrbe he srucure of he model. The CRF s also a dscrmnave model where as he HMM s a generave one. A poenal advanage of he dscrmnave CRF over generave models s ha hey have been shown o be more robus o volaons of ndependence assumpons [Laffery e al., 2001]. Fgure 3- CRF model. The observaons (r) are shown for each mesep. The class label x s also shown. The dscree emporal sae a a parcular mesep s gven by x whch akes a value from he se of all possble class labels x X = {1,2,.., C}. Here C s he maxmum number of class labels whls represens he me wh T beng he maxmum lengh of he sequence. Observaons a me are denoed as r wh he on observaons gven as R = ( r1,.., r ). Lkewse he on sae s gven by X = ( x1,.., x). For noaonal compacness we shall refer o X as X and R as R n accordance wh oher auhor s [Smnchsescu e al.,2005, and Wallach, 2004]. The dsrbuon over on labels X gven observaons R and parameers Θ are gven by: 1 c pθ( X R) = φθ( Xc, Rc) Z ( R) θ c C( XR, ) (1.9)

9 Whn hs equaon φ s a real valued poenal funcon of he clque c and ( R ) s he c θ observaon dependen normalzaon (somemes referred o as a paron funcon). C(X,R) s he se of maxmal clques. Here a frs order lnear chan s used (as show n Fgure 3). The clques are pars f neghborng saes ( x, x + 1), whls he connecvy among observaons s resrced o ha shown n he graph (Fgure 3- CRF model. The observaons (r) are shown for each mesep. The class label x s also shown.). A CRF model wh T meseps, as used here, can be re-wren n erms of exponenal feaure funcons F θ compued n erms of weghed sums over he feaures of he clques. Ths exponenal feaure formulaon s gven below n equaon (1.10) Z θ T 1 pθ( X R) = exp( Fθ( x, x 1, R)) Z ( R) θ = 1 (1.10) The condonal log lkelhood of he CRF s gven below. Assumng ha he ranng daa s X d d, R he parameers of he model are obaned by opmzaon of he fully labeled { } 1,.., followng funcon: d= D D D T L = log p ( X R ) = ( F ( x, x, R ) log Z ( R )) (1.11) d d d d d d θ θ θ 1 θ d= 1 d= 1 = In order o make parameer opmzaon more sable he problem ofen makes use of a regularzed erm ( R θ ). The problem hen becomes one of opmzng he penalzed lkelhood 2 ( Lθ + Rθ ). The regularzng erm used here was chosen o be R θ = θ. Once raned novel npu s gven o a CRF model and a probably dsrbuon s gven hroughou all meseps for all classes. In hs case we choose he hghes probably as beng he classfcaon label of he new example. A Gaussan pror over he npu daa was used hroughou all expermens. Olver s Coupled Hdden Markov Model Here we brefly cover Olver s mehod ([Olver e al., 2000]) of classfyng neracng ndvduals. Ths work s revewed as provdes a sae of he ar mehod o compare our resuls wh. Olver used coupled hdden Markov models o model fve dfferen neracve behavors. Olver s work s used for comparson wh he work presened here. Olver e al use wo feaure vecors (one for each chan). These feaure vecors are made up of he velocy of he person, he change n dsance beween he wo people and he algnmen beween he wo people. Ths gves wo feaure vecors, one for each chan. For each class he parameers of a wo chan coupled hdden Markov model are raned. When classfyng he model whch produces he hghes lkelhood for a es sequence s aken as he class label.

10 RESULTS Ths secon presens he resuls obaned by usng he mehods descrbed n he precedng chapers. Resuls usng a condonal random feld (CRF), hdden Markov model (HMM) and s coupled varaon (CHMM) are presened. Resuls usng a lnear dscrmnan model (LDA) are also presened and used as a baselne non-probablsc mehod o whch resuls are compared. We presen he resul of classfcaon over many ranng and esng subses o gve an ndcaon of he sandard devaon and he expeced performance when usng a mehod. Resuls are presened over many dfferen wndow szes. The graphs show he averaged performance of he classfer over 50 runs. The sandard devaon s gven by he shaded regons. Expermenal seup The CAVIAR daase has been prevously used n [Dee and Hogg, 2004a, Wu and Nevaa, 2007] however here s no a unversally agreed ranng and esng se. Therefore was deemed ha n order o characerse an algorhm s rue performance upon a daase should be esed wh dfferen subses of he enre daa. Ths wll gve an ndcaon of he expeced performance of he algorhm raher han fndng a parcularly good (n erms of classfcaon accuracy) subse. We are neresed n comparng he four mehods as descrbed n he prevous secon. Furhermore he role of me s nvesgaed. We seek o nvesgae wha s he opmal lengh of me a sequence should be wached before a decson s made. Resuls comparng each mehod and he effecs of me are gven n he followng secons. Throughou he ranng procedure he esng se was kep separae from he ranng daa and was unseen by he learnng algorhm unl esng me. Therefore only he ranng se was used when deermnng he parameers of he learnng model. Tranng and esng ses were spl 50/50 on a per class bass. Paronng was done per-class raher han over he whole daase due o he uneven dsrbuon of classes. Such a sep means ha n he ranng sage he learnng algorhm wll have examples of every ype of class. We would no expec a correc classfcaon on unseen classes and so hs measure can sop msleadng resuls. In order o show he average performance hs procedure was repeaed over 50 dfferen parons of he ranng and es daa. The daase conans examples of complee sequences, for example a sequence conssng of wo people walkng ogeher may be hundreds of frames long. Our goal s o classfy each frame correcly. If we were o ake hs sequence and spl up as ranng and esng frames hen he classfcaon ask would be much smpler as ranng and esng pons would be smply a maer of nerpolaon beween hghly smlar pons. I s for hs reason ha when decdng on he ranng and esng daa we paron based on he complee sequences. Ths means ha an enre walkng ogeher sequence wll be assgned o he ranng se whls anoher complee walkng ogeher sequence wll be assgned o he esng se. Ths should avod he pfall of havng ranng and esng daa whch s essenally he same. Ths s especally rue as he daa has a very srong emporal couplng.

11 Wha we are amng o do s ry o gve a class label o wo neracng ndvduals. Ths s shown n Fgure 4-. The wo people are approachng one anoher. Beween he frs and second frame he dsance beween hem decreases. The classfer would assgn one label for hs neracon (coverng boh people) o ndcae approachng. Classfcaon Resuls Fgure 4-Two people approachng one anoher. CAVIAR Daase The CAVIAR daase [EC Funded CAVIAR.. (2004)] conans 11,415 frames n whch some have labeled examples of neracons. Whn hs se here are 5 dsnc classes whch we seek o denfy and classfy. The 5 classes conss of examples of people: walkng ogeher (2,543), approachng (942), gnorng (4,916), meeng one anoher (1,801), splng up (879) and fghng (334). The numbers n brackes ndcaes how many frames conan hs behavor. Overall resuls for he daase are shown n Fgure 5. These resuls are he furher broken down and shown for each class n Fgure 6, Fgure 7 and Fgure 8.

12 Fgure 5- Overall resuls on he CAVIAR daase for each mehod. Lnes show averaged resuls (over 50 runs) whls he shaded regons show one sandard devaon. I s vsble ha for small wndow szes he CRF mehod offers he bes performance (excep perhaps when classfyng he approachng behavor). However n hs daase he HMM model gves he bes possble average performance. Boh he HMM and he CRF usng he new proposed feaures show superor performance o Olver s mehod. A parcular problem for all mehods s n he classfcaon of fghng behavor. A small sample sze and he shor mescale where any fghng acually occurs conrbue owards hs. We see ha for some cases he wndow s oo small (such as walkng ogeher and gnore) or oo large (such as gnore and approach). A per class me wndow or an enhanced feaure se would help hs problem

13 Fgure 6 Resuls on he approachng and fghng class, for he CAVIAR daa. Fgure 7 Resuls on he gnore and mee class, for he CAVIAR daa.

14 Fgure 8 Resuls on he spl and walk ogeher class, for he CAVIAR daa One of he oher feaures of hs daase s ha for approachng, splng and fghng here were perhaps no enough examples o ge a buld a suffcenly generalsable model. A smple answer would be o say ha more daa s requred. However n many real survellance applcaons such an approach s no possble so showng how a mehod performs usng only lmed daa s sll of value. BEHAVE Daase The BEHAVE daase [Blunsden e al. 2007] conans 134,457 frames whch have labeled examples of neracons. Whn hs se here are 5 dsnc classes whch we seek o denfy and classfy. The 8 classes conss of examples of people: In a group (91,421), Approachng (8,991), walkng ogeher (14,261), splng (11,046), gnorng (1,557), fghng (4,772), runnng (1,870) and chasng (539) one anoher. The numbers n brackes ndcaes how many frames conan hs behavor.

15 Fgure 9- Overall performance on he BEHAVE daase for each mehod. Lnes show averaged resuls (over 50 runs) whls he shaded regons show one sandard devaon. The overall averaged classfcaon s shown n Fgure 9. The CRF clearly performs beer han all oher mehods on hs daase for all wndow szes. There s a slgh ncrease n performance when he wndow sze s ncreased when usng a CRF. However he effec s more dramac for boh he CHMM, HMM and he LDA mehod. All hree of hese mehods (HMM, CHMM, LDA) ncrease n performance as he wndow sze s ncreased. Sgnfcan performance ncreases are observed beween wndow szes of 1 and 20. Per class resuls (fgures Fgure 10, Fgure 11 and Fgure 12) dsplay a smlar sory where ncreasng he wndow sze has lle effec upon CRF classfcaon. When classfyng splng, approachng (Fgure Fgure 10) and fghng (Fgure 11) ncreasng he wndow sze mproves he performance of he HMM, CHMM and he LDA classfers. The HMM classfer gves he bes performance of all mehods when classfyng fghng (Fgure 6). Increasng wndow sze also gves a smlar ncrease n performance for boh he n group classfcaon and he walkng ogeher class (Fgure 12). However when classfyng people n a group he LDA mehod decreases n performance.

16 Fgure 10-Resuls on he spl and approach classes for he BEHAVE daase. Fgure 11 - Resuls on he fgh and gnore classes for he BEHAVE daase.

17 Fgure 12- Resuls on he walk ogeher and n group classes for he BEHAVE daase. CONCLUSION Over all daases he CRF classfer performs well (80-95%) when usng lmed nformaon. The prevous bes mehod suggesed by Olver [Olver, 2000] s mproved upon usng he CRF classfer n conuncon wh he new proposed feaure se. The new proposed feaure se also ouperforms Olver s mehod when usng a HMM model for a grea many cases. The CRF classfer dsplays an ably o beer classfy daa n he shor erm compared o he HMM. In conras he HMM model mproves more rapdly when he observaon wndow sze s ncreased suggesng s beer a smoohng he sgnal over longer sequences. The forward algorhm used o deermne he lkelhood seems o smooh he sgnal much beer han he CRF. The case of he CHMM he more gradual mprovemen could be arbuable o he larger number of parameers whch requres more daa n order o represen he daa adequaely. A suggeson for fuure work would be herefore o mprove he long erm emporal model of he CRF ha we are usng. I should be noed ha hese commens abou he CRF model apply for he sngle chan srucure whch s used here. There are many confguraons whch can be used whn he CRF framework. A hgher order CRF may produce a beer emporal model and so we would expec o see larger mprovemens when he observaon wndow sze s ncreased. However a CRF model wll always be dscrmnave compared o he HMM and CHMM s generave ably. The ably o generae samples may be mporan n ceran cases (such as esmang a model s complexy [Gong and Xang, 2003]) however hs ably s no requred for he classfcaon asks as presened here. Throughou all expermens on all of he daases s vsble ha here seems o be an opmal wndow sze for classfcaon of a parcular class. For some acves such as fghng n he CAVIAR daase (Fgure 6) he wndow sze s que shor (due o he speed of a fgh) where as

18 for oher classes such as he gnore behavor (Fgure 7) a longer wndow sze mproves performance. FUTURE WORK The sgnfcance of he role of me whn hs work has been demonsraed. Fuure work should seek o explo hs n a prncpled way. I would also be fruful o nvesgae n a prncpled way auomac feaure selecon. I would be envsaged ha such a procedure could ncorporae los of feaures and chose hem auomacally raher han, as s he case here, where s only possble o use a lmed number of feaures. By havng a comparave sudy of he classfers on he daase s also possble o know how much you should rely on hem, f for nsance you were usng some classfcaon crera n feaure selecon or o esablsh an opmal me wndow. REFERENCES S. Blunsden, E. Andrade, A. Laghaee, and R. Fsher. Behave neracons es case scenaros, epsrc proec gr/s98146, On Lne, Sepember URL: hp://groups.nf.ed.ac.uk/vson/behavedata/interactions/ndex. EC Funded CAVIAR proec/ist found a url: hp://homepages.nf.ed.ac.uk/rbf/cavar/, J. W. Davs and A. F. Bobck. The represenaon and recognon of acon usng emporal emplaes. In IEEE Transacons on Paern Analyss and Machne Inellgence, volume 23, pages IEEE Compuer Socey, H. M. Dee and D. C. Hogg. Deecng nexplcable behavour. In Brsh Machne Vson Conference, pages , Sepember P. Dollar, V. Rabaud, G. Corell, and S. Belonge. Behavor recognon va sparse spaoemporal feaures. In PETS, pages 65 72, Chna, A. Efros, A. Berg, G. Mor and J. Malk. Recognsng acon a a dsance. In In 9 h Inernaonal Conference on Compuer Vson, volume 2, pages , G. Ggerenzer, P. M. Todd, and ABC Research Group. Smple Heurscs Tha Make Us Smar. Evoluon and Cognon Seres. Oxford Unversy Press, S. Gong and T. Xang. Recognon of group acves usng a dynamc probablsc nework. In IEEE Inernaonal Conference on Compuer Vson, pages , Ocober 2003b. S. S. Inlle and A. F. Bobck. Recognzng planned, mulperson acon. CVIU, 81(3): , March X. Lu and C. S. Chua. Mul-agen acvy recognon usng observaon decomposed hdden markov models. Image and Vson Compung, pages , 2006.

19 N. M. Olver, B. Rosaro, and A. P. Penland. A Bayesan compuer vson sysem for modelng human neracons. IEEE Transacons on Paern Analyss and Machne Inellgence, 22(8): , M. Perse, M. Krsan, J. Pers, and S. Kovacc. A emplae-based mul-player acon recognon of he baskeball game. In J. Pers and D. R. Magee, edors, ECCV Workshop on Compuer Vson Based Analyss n Spor Envronmens, pages 71 82, Graz, Ausra, May L. R. Rabner. A uoral on hdden markov models and seleced applcaons n speech recognon. Readngs n speech recognon, pages , P. Rbero and J. Sanos-Vcor. Human acves recognon from vdeo: modelng, feaure selecon and classfcaon archecure. In Workshop on Human Acvy Recognon and Modellng (HAREM n conuncon wh BMVC 2005), pages 61 70, Oxford, Sepember N. M. Roberson. Auomac Causal Reasonng for Vdeo Survellance. PhD hess, Hearford College, Unversy of Oxford, June C. Smnchsescu, A. Kanaua, Z. L and D. Meaxas. Condonal models for conexual human moon recognon. In Inernaonal Conference on Compuer Vson, T. Van Vu, F. Bremond, and M. Thonna. Auomac vdeo nerpreaon: A recognon algorhm for emporal scenaros based on precompled scenaro models. In ICVS, H. M. Wallach. Condonal random felds: An nroducon. Techncal repor, Unversy of Pennsylvana, CIS Techncal Repor MS-CIS Bo Wu and Ram Nevaa. Deecon and rackng of mulple, parally occluded humans by bayesan combnaon of edgele based par deecors. Inernaonal Journal of Compuer Vson, 75(2): , Ylmaz, A., Javed, O., and Shah, M. (2006). Obec rackng: A survey. ACM Compu. Surv. 38, 4 (Dec. 2006)