Soccer Player Tracking across Uncalibrated Camera Streams

Transcription

1 EEE nernainal rkshp n Visual Surveillance and erfrmance Evaluain f Tracking and Surveillance ETS 3 n cnjuncin wih V Ocber 3 Nice France. Sccer layer Tracking acrss Uncalibraed amera Sreams Jinman Kang saac hen and Gerard Medini RS mpuer Visin Grup Universiy f Suhern alifrnia s Angeles A {jinmankaichenmedini}@iris.usc.edu Absrac This paper presens a nvel apprach fr cninuus deecin and racking f mving bjecs bserved by muliple sainary cameras. e address he racking prblem by simulaneusly mdeling min and appearance f he mving bjecs. The bjec s appearance is represened using clr disribuin mdel invarian D rigid and scale ransfrmain. prvides an efficien blbs similariy measure fr racking. The min mdels are bained using a Kalman Filer KF prcess which predics he psiin f he mving bjec in D and 3D. The racking is perfrmed by he maximizain f a jin prbabiliy mdel reflecing bjecs min and appearance. The nvely f ur apprach cnsiss in inegraing muliple cues and muliple views in a JDAF fr racking a large number f mving peple wih parial and al cclusins. e demnsrae he perfrmances f he prpsed mehd n a sccer game capured by w sainary cameras.. nrducin Tracking players in a sccer field frm muliple cameras pses very ineresing prblems fr visin-based racking sysems. The players unifrms d n prvide sufficien clr infrmain fr racking he players in he field. Appearance based racking sysems are n efficien in handling hese siuains. ndeed clr-based racking sysems are rapidly faced wih he prblem where he clr is n sufficien fr characerizing he rajecries f independenly he mving players in he scene. The large number f clr ambiguiies bserved is due he lack f saliency f he clr feaures. The grup behaviur f he players requires als a racking mehd rbus dynamic cclusins. These cclusins ccur beween a large number f players and hey can be parial r cmplee and f varying durain. The small number f pixels represening he mving player makes he prblem mre challenging as i is difficul build an accurae appearance-based mdel f he invlved players. ndeed muliple players cclude each her in very small regins and smeimes he human bserver has difficulies in racking he varius players when hey rush ward he ball. layers size in he image can be increased by using a mving camera ha racks he players r muliple sainary cameras. Muliple sainary cameras can guaranee a gd balance beween sccer field cverage and image size f he sccer players. This hwever requires a racking sysem capable f inegraing infrmain acrss views fr guaraneeing cnsisency f he rajecries f he mving players. Using efficienly he knwledge f he cameras lcains and characerizing areas f he field ha verlap in several views increases he racking perfrmances by reslving ambiguiies ha may appear in a paricular view. n he fllwing secins we review relaed wrk and presen an verview f he prpsed apprach. Secin describes he mehd fr regisering muliple sainary cameras. Deecing mving bjecs in vide sequences is briefly described in Secin 3. Secin 4 inrduces he appearance-based mdel and he jin prbabiliy mdel used fr racking mving bjecs. The iniializain f bjecs appearance is addressed in Secin 5. Obained resuls are presened and discussed in Secin 6. Finally Secin 7 cncludes ur paper wih a discussin n fuure wrk... revius rk Several algrihms fr racking mving bjecs acrss muliple sainary cameras have been prpsed recenly and ms f hem use clr disribuin as he main cue fr racking bjecs acrss views. mmn mehds such as maching clr hisgram f he blbs acrss he views fr cameras handver [][3] cnsrucing blbs in 3D space using shr-base line sere maching wih muliple sere cameras [] r using vlume inersecin []. Since clr infrmain can be easily biased by several facrs such as illuminain shadw blbs segmenain appearance change r differen camera cnrls clr cue is n very reliable fr racking mving bjecs acrss large scenes such as halls. The use f such mehds is limied he case f synchrnized cameras fr ensuring crrespndence acrss views. n [5] he auhr prpsed an apprach fr space and ime selfcalibrain f cameras bu he prpsed apprach is limied small mving bjecs and p dwn views where he bserved shapes are similar acrss views and he deph f he bjec is n significan. Muliple views racking sysem frm unsynchrnized vide sreams is prpsed in Errr! Reference surce n fund.. The auhrs prpsed a cmbinain wih grund plane hmgraphy and spai-empral hmgraphy fr regisering unsynchrnized muliple sainary views. Aumaic

2 EEE nernainal rkshp n Visual Surveillance and erfrmance Evaluain f Tracking and Surveillance ETS 3 n cnjuncin wih V Ocber 3 Nice France. camera hand ff cnrl has been prpsed in [4][6] using hese principles. n [6] he auhr prpsed a cmplee muli-sensr surveillance sysem limied nly muliple sainary cameras... Overview f he rpsed Apprach n his paper we prpse a nvel apprach fr inegraing infrmain frm muliple sainary cameras wih a prir knwledge f he scene. e use a hmgraphy fr regisering sainary cameras n p f a knwn grund plane. This apprach cninuusly racks mving bjecs acrss w r mre uncalibraed cameras. The deecin f mving bjecs in he vide sream is perfrmed by a mde-based backgrund mehd. Each deeced mving blb is prjeced n he grund plane fr guaraneeing cnsisency f he labels acrss he cameras view pins. The racking f deeced mving bjecs in each view is frmulaed as a maximizain f a jin prbabiliy mdel. The jin prbabiliy mdel cnsiss f an appearance mdel D and 3D min mdels. The appearance prbabiliy mdel is a clr similariy measure beween deeced blbs. The appearance mdel is a clr disribuin mdel derived by segmening he blb in a cllecin f plar bins. The appearance mdel is defined as a cmbinain f he plar disribuins guaraneeing invariance ranslain rain and scale f he bjec. The D and 3D min prbabiliy mdels are inferred frm a Kalman Filer KF. These mdels are calculaed by a Gaussian disribuin beween he prediced bunding bx psiin and he bunding bx psiin f he bserved blbs in D. Fr 3D a prjecin n he grund f he deeced bunding bx characerizes he bserved player s f psiin. The prediced psiin f he f n he grund plane is esimaed using a KF. The nvely f ur apprach cnsiss in handling muliple cluered and verlapped rajecries bserved by muliple sainary cameras using JDAF. derives a simulaneus min measuremen in D and 3D fr bjecs viewed by he w cameras and allws fr an aumaic handling f cclusins deecin errrs and cameras handff.. Regisrain f Muliple ameras Muliple sainary cameras prvide a gd cverage f he scene bu require he inegrain f infrmain acrss cameras. The gemeric regisrain f cameras viewpin is perfrmed using a hmgraphy frm a se f 4 maching pins bained frm a grund plane. f we dene as he hmgraphy frm view H g he grund plane mdel we can regisered all her available views by he same cnveninal hmgraphy. n Figure we shw he regisrain f w sainary views by a knwn grund plane mdel e.g. a sandard f sccer field. n addiin he regisrain f muliple cameras here are several advanages if we use prir knwledge f he grund plane. The firs advanage f he prpsed regisrain mehd is we can rughly esimae he psiin f each camera and each mving bjecs in 3D s we can inegrae 3D min racking cmpnen wihu explici recvery f 3D infrmain such as SFM. The regisered f psiin f each mving bjec by he hmgraphy is used fr cnsrucing a cmpnen f 3D min mdel laer. Furher mre he rugh heigh esimain f mving bjecs is used fr seing hyphesis f uncerainy beween racked bjecs and deeced bjecs. The uncerainy hyphesis is used fr iniializain f each mving bjecs. The secnd advanage f he suggesed apprach is we can simulaneusly label each mving bjecs acrss views wih unique id using regisered f psiin f deeced mving bjecs. Figure. amera regisrain using he grund plane. 3. Deecing Mving Regins The deecin f mving regins frm a saic backgrund has been sudied exensively fr a lng ime and here are ls f well knw echniques such as backgrund learning algrihm using mean and variance f backgrund he mde-based deecin algrihm which uses he mde f he clr hisgram f each pixel fr mdeling backgrund r he saic cmpnens f he scene r he shadw exracin algrihm. n his paper we use he mde-based deecin algrihm fr an experimen in he laer secin. Als each blb is labelled uniquely acrss views using he grund plane mdel hmgraphy by esimaing crrespnding f psiins acrss views s he blbs are racked simulaneusly acrss views. 4. Tracking using Jin rbabiliy Mdels e decuple he racking prblem in hree pars by firs mdeling bjecs appearances by defining a clrbased bjec represenain and hen mdeling D and 3D velciies f bjecs. Each mdel is frmulaed as a prbabilisic mdel and he racking prblem is defined as maximizain f he jin prbabiliy. Jin prbabiliy daa assciain filer JDAF cmbines muliple hypheses/cues fr racking mving b-

3 EEE nernainal rkshp n Visual Surveillance and erfrmance Evaluain f Tracking and Surveillance ETS 3 n cnjuncin wih V Ocber 3 Nice France. jecs [7][8]. JDAF echniques are faul leran and rely n available hypheses fr racking mving bjecs. Hwever he selecin f he muliple hypheses as well as heir weighing is very subjecive and relies n ad-hc mehds. ndeed weighing prperly he varius hypheses is n sraighfrward in JDAF as min clr and appearance shape edge ec. mdels are very disinc and difficul cmbine. Therefre when several disincive hypheses are used hey shuld be equally weighed. Our apprach uses he KF fr mdeling he D and 3D velciy f mving bjecs. The inegrain wih he appearance-based represenain f mving bjecs is perfrmed by a maximisain f jin prbabiliy mdel. 4.. Appearance Mdel Varius mehds have been prpsed slve he racking prblem using clr infrmain. Many f hem use nly ne clr hisgram mdel per bjec prevening frm differeniaing a persn wearing blue jeans wih a red shir frm a persn wearing red pans and a blue shir. Muliple clr mdels and heir relaive lcalizain shuld be cnsidered fr an efficien use f clr in bjec racking. n [] muliple clr mdel apprach was prpsed fr human deecin. The blb f he deeced persn is subdivided in hree regins crrespnding he head rs and legs. n he real life siuains blbs changes are bserved due self-cclusins and incmplee deecin. Furhermre if he bjec is raed he prblem ges mre cmplicaed as i requires segmening he blbs in bdy pars which is n an easy ask. Objec appearance mdels have be cninuus in he sense ha a small lcalized change f he bjec clr shuld creae a small variain in is signaure. The bjec descripin shuld als be invarian D rigid ransfrmain in rder guaranee gd descripin capabiliies. e prpse in his paper an appearance mdel ha is invarian D rigid ransfrmain. This mdel defined by plar disribuin prvides a descripin f bjec s clrs prperies. This D disribuin mdel will be used fr measuring he similariy f racked bjecs wihin and acrss cameras. The clr disribuin mdel is bained by mapping he blb in a plar represenain. Several shape r clr disribuin mdels using a plar represenain have been prpsed [6]Errr! Reference surce n fund.errr! Reference surce n fund.. n [6] he prpsed apprach is fcused n he bjec s shape descripin edge insead f heir appearance clr and i is nly limied represening lcal shape prperies. n Errr! Reference surce n fund. he prpsed mdel measures clr disribuin using a similar plar represenain bu fcuses n characerizing a glbal appearance signaure f he bjec. The mdel is n D rain-invarian and we prpse here use he shape descripin mdel prpsed in Errr! Reference surce n fund. fr guaraneeing invariance D rigid ransfrmain. Given a deeced mving blb we cmpue a reference circle R defined by he smalles circle cnaining he blb. This circle is unifrmly sampled in a se f cnrl pins. Fr each cnrl pin a se f i cncenric circles f varius radii are used fr defining he bins f he appearance mdel. nside each bin a Gaussian clr mdel is cmpued fr mdeling he clr prperies f he verlapping pixels f he deeced blb. Therefre fr a given cnrl pin i we have a unidimensinal disribuinγ i i. The nrmalized cmbinain f he disribuins bained frm each cnrl pin i defines he appearance mdel f he deeced blb: Λ γ i i. Figure. mpuain f he appearance mdel n each mving bjec. An illusrain f he definiin f he appearance mdel is shwn in Figure where we sampled he reference circle wih 8 cnrl pins. The defined mdel is ranslain invarian. Rain invariance is bained by aking a larger number f cnrl pins alng he reference circle. Finally nrmalizing he reference circle uni circle guaranees scale invariance. The appearance mdel prbabiliy is frmulaed as a similariy measure. The prbabiliy is direcly derived frm he Gaussian apprximain f he clr prperies f each bin. The calculain f he clr appearance prbabiliy is defined as fllws: where red greed clr blue red + green 3 + i blue is he prbabiliy likelihd esimain f each clr cmpnen and is defined as: red N µ N r µ µ µ µ r r + µ r N r µ r + r + µ r + where N is he al number f bins angular bins radial bins µ is he mean f he red cmpnen f he r bin i. n Figure 3 several examples are displayed illusrae he prpsed mehd and he cnsisen measure f he similariy wih varius rain angles. The capabiliy disinguish he same bjec wih differen clrs is als illusraed. One can bserve ha he similariy scre is high ver.99 fr he same bjecs a differen rienains while same bjecs wih clr variains are given

4 EEE nernainal rkshp n Visual Surveillance and erfrmance Evaluain f Tracking and Surveillance ETS 3 n cnjuncin wih V Ocber 3 Nice France. a lw similariy scre belw.78. The advanages f he prpsed apprach ver he single clr mdel and her plar crdinae based muliple clr mdels are: he lcalizain f clr prperies and he D rigid ransfrmain invariance prpery. Als lcalized variains due cclusins r change in viewpin generae lcal mdificains f he clr mdel. Finally he similariy beween bjecs is measured by cmparing clr disribuin mdels raher han single scalar values. Ref. Blb Mving Objec lr Descripin R G B Similariy clr apprximaed using a firs rder KF filer. Oher mehds based n Exended Kalman Filer EKF [5] Unscened Kalman Filer UKF [3] ONDENSATON [] and aricle Filer F [4] have been cmmnly used. These appraches are well adaped fr handling nn-linear dynamic sysems bu require a large amun f cmpuain D Min Mdel f we assume ha he deeced regins mve in he image plane a cnsan velciy he dynamics f he min can be apprximaed by a firs rder Newnian dynamics. hen he velciy f he bserved mving bjec des n vary significanly such mdel is accepable. The min mdel f he mving bjec is bained by a firs rder Kalman Filer KF. The use f KF fr predicing and esimaing bjecs rajecries is a well knwn mehd hwever min esimaes are n sufficien fr prviding an accurae racking f mving bjecs wih parial and al cclusins. n his secin we use he KF frmalism derive nly he prbabiliy mdel assciaed a cnsan velciy min in he image plane. This prbabiliy will be hen used by a jin prbabiliy mdel. The sae vecr cnsidered in racking mving bjec in D space is defined by he fllwing vecr: x x y u v x y u v p p p p bm bm bm bm Red Shir Blue Shir Red Shir Figure 3. Example f he invariance rains f he prpsed appearance mdel. 4.. Min Mdel lr similariy f he deeced blbs is a cmmn apprach fr racking mving bjecs. Hwever ambiguiies are frequen and bjec appearances change as he bjecs mve in he scene r he clr infrmain may n be relevan. This laer case ccurs in racking players in spring evens such as sccer. Team members have he same unifrm and appearance mdel is n sufficien fr racking he players. e address his prblem by using D and 3D dynamic mdels represening he min f he deeced mving bjecs. These mdels are where x y and x y are respecively he bm p p bm p-lef and he bm-righ crner f he deeced bunding bx f he mving regin and u p v and u v are he crrespnding D image velciy. bm bm Assuming ha he mving bjec mves in he image space a cnsan velciy he new psiin is bained by he fllwing equain: x + + xp + yp + up + vp + x + y + u + v Fx bm bm bm bm + e x y u v x y u v p p p p bm bm bm bm where F is he sysem evluin marix and p e e e e4 e 5 e6 e 7 e8 is he bm prcessing nise vecr. f we assume ha we are nly able bserve he psiin f he mving bjec frm he image sequence ur measuremen vecr is z i j i j crrespnding he p-lef and bm righ pin f he deeced bunding bx. The measuremen equain is frmulaed as fllws: p p e bm

5 EEE nernainal rkshp n Visual Surveillance and erfrmance Evaluain f Tracking and Surveillance ETS 3 n cnjuncin wih V Ocber 3 Nice France. view f he riginal regisered frames d Zm f he ip crrespnding crwded regin frm he p-dwn view 4 j red bx. z Hx p x ibm The 3D esimain is in he prjecive space and n jbm Euclidean as we d n use camera calibrain paramewhere H is he measuremen marix. The ime updae ers. Hwever since we use Euclidean grund plane and measuremen updae equains are given in he infrmain he 3D psiin f he f can be esimaed fllwing se f equains: up a scale. The main purpse fr acquiring he 3D min mdel f deeced bjecs is fr disambiguaing x + Fx cluered bjecs by using an esimae f heir 3D lca + F FT + Q in. n Figure 4 we shw an example f hw dense and 5 T T ccluded bjecs can be easily segmened by using he K H H H + R esimain f heir 3D psiin and velciy. n he blue x x + K z H x bx depiced in Figure 4 here are 6 sccer players ha are difficul segmen in individual players. The use K H f he p dwn view prvided by he hmgraphy pr where x is he a priri sae esimae a ime x is he a vides a clear segmenain f he players. pseriri sae esimae is he a priri esimae errr The sae vecr cnsidered in racking mving bjec in 3D space is defined by he fllwing vecr: cvariance is he a pseriri esimae errr cvarix f x f y f u f v f 6 ance Q is he prcess nise cvariance and K is he Kalman gain. where we have discarded he z crdinaes as we assume The D min prbabiliy mdel d _ min is calcu- ha his 3D pin lies n he 3D grund plane used as laed by he Gaussian nrmal disribuin f he min rigin. The crdinaes x f y f are cmpued frm esimaes prvided by he KF D Min Mdel n he case f crwded scene when he mving bjecs are ccluding each her he knwledge f sme 3D feaures can reduce he number f ambiguiies. Fr example if we assume ha he mving bjecs mve n he grund we can mdel he 3D min f he f frm he deeced mving humans in he image. Regisering he 3D grund plane he image using a hmgraphy maps bm pins f he deeced bunding bx in 3D psiins. Using again a cnsan 3D velciy dynamic mdel we can bain he 3D esimain f min mdel f he mving bjecs. he deeced mving blb: crrespnds he cenrid f blb s lwes pixels i.e. f minimum y. u f v f crrespnds he velciy f he pin x f y f n he 3D grund plane. Assuming ha he mving bjec mves n he grund plane a cnsan velciy he new psiin is bained by he fllwing equain: x f+ F f x f + e f + x f + y f + u f + v f x f y f u f v f + e e e 3 e 4 7 where F f is he sysem evluin marix f he 3D min mdel and ef is he prcessing nise vecr. f he nly bservain prvided is he cenrid i f j f f he blb s lwes pixels he measuremen vecr is zf i f j f. The measuremen equain is frmulaed as fllws: i f 8 z f Hxf f x j f where H is he measuremen marix. The ime updae and measuremen updae equains are he same as he ne given in equain 5. Figure 4. Using 3D fr disambiguaing cluered bjecs. a The riginal frame b Zm f he ms crwded regin in he riginal frame blue bx c The p-dwn The min prbabiliy mdel 3d _ min is als bained frm Gaussian nrmal disribuin f he min esimaes prvided by he KF Jin rbabiliy Mdel

6 EEE nernainal rkshp n Visual Surveillance and erfrmance Evaluain f Tracking and Surveillance ETS 3 n cnjuncin wih V Ocber 3 Nice France. e frmulae he racking prblem as finding an pimized psiin in D and 3D f he mving bjec by maximizing all prbabiliy mdels. e frmulae he racking prblem as fllws: argmax _ 3 _ min D min D clr 9 The jin prbabiliy is defined by he prduc f he appearance and w min prbabiliies. al argmax R where denes clr bservain is he D psiin n he image space and is he 3D psiin n he grund plane a ime. The calculain f he pimal psiins a each ime sep is frmulaed as fllws: argmax argmax argmax M The pimal psiin a each ime sep depends n he curren bservain as well as n he min esimain a he previus psiins. crrespnds he maximum prbabiliy f all curren and previus bservains. nsequenly a jin prbabiliy f he given psiin a ime is frmulaed as fllws: al Equain can be decupled using Bayes herem and rewrien as fllws: 3 The clr mdel depends nly n he curren bservain. Therefre he firs decupled cmpnen will be rewrien as: 4 The secnd cmpnen in equain 3 is als decupled using Bayes herem: 5 The firs cmpnen in equain 5 is he prbabiliy f he curren psiins given all pas clr bservains and crrespnding pimal D and 3D psiins. The D r 3D min mdels prvided by a firs rder KF rely nly n he previusly esimaed psiin and he infrmain f he pas clr bservain is n prpagaed he curren clr bservain. Therefre he clr bservain mdels can be discarded. Als each min mdel is esimaed by a separae KF he firs erm in equain 5 can hen be rewrien as fllws: 6 where he firs cmpnen is he prbabiliy f he curren D psiin given all pas D pimal psiins and he secnd ne is he prbabiliy f he curren 3D psiin given all pas 3D psiins. The secnd erm in equain 5 is herefre defined by: al 7 Frm equain 4~7 he jin prbabiliy f he curren psiin can be derived as a prduc f he prbabiliy f he curren clr bservain D and 3D psiin esimaed by he KF-based min and he jin prbabiliy f pas pimal psiins. e bain he fllwing equain: _ 3 _ al al min D min D clr al 8 n rder avid he accumulain f prducs f prbabiliies we use he lg f he prbabiliies as jin prbabiliy. ensuring a sable calculain and als we discard ld measuremen frm he esimain prcess. This shrens he memry f he KF and allws variains in speed and clr similariies. 5. niializain f he Tracking mpuing he crrec iniial sae f he mving bjec is a crucial sep. ndeed he clr and min mdels f each mving bjec depend n he firs bservain and herefre a gd segmenain f he mving bjecs is required. Muliple peple r sccer players may ener he field f view a nce; he deeced blb is herefre represening a grup f peple ha will crrup he clr and min mdels prpsed if cnsidered as a single bjec. nsead f segmening he deeced blbs in individual peple r players we have chsen characerize he iniial psiin min and clr prpery f each bjec based n heir empral evluin in he scene. A deeced blb is agged as an uncerain bjec when is heigh and size f he bunding bx d n crrespnd an expeced size. This prevens us frm prpagaing errneus infrmain he nex sep and crruping he JDAF-based racking. The heigh and size f deeced mving bjecs is a srng cue fr generaing such hyphesis. Using he grund plane infrmain prvided by he hmgraphy regisering he w views an esimain f he heigh f he deeced mving player can be derived. One can easily characerize wha is a likelihd f bserving a player f an esimaed heigh. A secnd hyphesis is generaed frm he size f he bunding bx f he racked blb bserved acrss muliple views. Since he cameras are regisered using a hmgraphy ne can

7 EEE nernainal rkshp n Visual Surveillance and erfrmance Evaluain f Tracking and Surveillance ETS 3 n cnjuncin wih V Ocber 3 Nice France. check he validiy acrss views f he size f he esimaed bunding bx. Therefre by cmparing he size f he bunding bx f he racked bjec acrss view anher hyphesis can be generaed. Fr efficiency purpse all he racked bjecs are represened by graph. Each nde crrespnds a deeced mving bjec and each edge crrespnds a pssible mach. Each racked blb agged as uncerain bjec remains as uncerain bjec and he racking decisin is pspned unil all hypheses are cleared. Once he racking decisin is made he infrmain f each racked bjec is reversely prpagaed using he graph represenain and an updae f he bjec psiin and bunding bx is cmpued. 6. Experimenal Resuls e presen in his secin sme resuls bained n ne f he sequence prvided by he VS-ETS 3 websie. The resuls illusrae he racking capabiliy f he prpsed apprach acrss views and shw is efficiency in racking he muliple mving bjecs wih parial and al cclusins. n Figure 5.a he blue and red bx represen bjecs in he verlapped regins frm bh views and he deeced bjecs are labelled wih he same id acrss views. n Figure 5.b ne player is ccluded by a secnd ne see blue bx bu bh f hem are cninuusly racked independenly by inegraing min and clr esimaes acrss views. A mre cmplex case is presened in Figure 5.c. n he blue bx f each frame here are five players generaing a large number f cclusins. The prpsed mehd racks separaely every player and crrec esimain f he bunding bx is generaed fr each player. Finally Figure 5.d and Figure 5.e illusrae he exraced rajecries. n Figure 5.d 3D rajecries f he players are shwn frm a p-dwn view. The crrespnding D rajecries viewed frm each camera are presened in Figure 5.e. 7. nclusin e have presened a nvel apprach fr cninuus racking f muliple bjecs wih a large amun f cclusins acrss muliple sainary cameras. As explained in Secin all available views are pre-regisered using he hmgraphy cmpued frm he grund plane mdel. This allws simulaneusly racking D and 3D players psiins in he field. The mdeling f blbs appearance alng wih is D and 3D min prvides a rbus mehd ha racks separaely every player albei he large amun f cclusins. The prpsed mehd prpagaes backwards he bjecs prperies psiin bunding bx D and 3D velciies fr an accurae segmenain f he deeced blbs in separae bjecs. Several issues have ye be addressed such as reducing he cmplexiy and he delay generaed by he prpagain f uncerain hyphesis curren implemenain runs a frame per secnd. Expanding he jin prbabiliy mdel her crieria such as spai-empral invarian shape descriprs and adaping higher rder esimain mehds fr inrducing nn-lineariy in he measuremen sep will imprve he accuracy f he racking. Acknwledgemens This research was parially funded by he Advanced Research and Develpmen Aciviy f he U.S. Gvernmen under cnrac MDA References [] A. Elgammal and. S. Davis rbabilisic Framewrk fr Segmening eple Under Occlusin n rc. f EEE V. [] A. Mial and. S. Davis MTracker: A Muli-View Apprach Segmening and Tracking eple in a luered Scene Using Regin-Based Sere n rc. f EV. [3] B. Senger. R. S. Mendnca and R. iplla Mdel- Based Hand Tracking Using an Unscened Kalman Filer n rc. BMV Vl. pp [4] B. Senger. R. S. Mendnca and R. iplla Mdel- Based Hand Tracking f an Ariculaed Hand n rc. f EEE VR Vl. pp [5] G. Sein Tracking frm Muliple View ins: Self-calibrain f Space and Time n rc. f EEE VR pp [6] H. Zhang and J. Malik earning a discriminaive classifier using shape cnex disance n rc. f EEE. VR Vl. pp [7] J. Kang. hen and G. Medini ninuus Muli- Views Tracking using Tensr Ving n rc. f EEE MV. [8] J. Kang. hen and G. Medini ninuus Tracking ihin and Acrss amera Sreams n rc. f EEE VR 3. [9]. hen and H. i nference f Human sures by lassificain f 3D Human Bdy Shape n rc. f EEE AMFG 3. [] J. Krumm S. Harris B. Meyers B. Brumi M. Hale and S. Shafer Muli-camera Muli-persn Tracking fr Easy- iving n prc. f EEE rkshp n Visual Surveillance. [] J. Orwell. Remagnin and G.A. Jnes Muli-amera lr Tracking n prc. f he nd EEE rkshp n Visual Surveillance 999. [] M. J. Black and A. D. Jepsn A rbabilisic framewrk fr maching empral rajecries: ndensain-based recgniin f gesures and expressins n rc. f EV Vl. pp [3] Q. ai and J.K. Aggarwal Aumaic Tracking f Human Min in ndr Scenes Acrss Muliple Synchrnized vide Sreams n rc. f EEE V 998. [4] Q. ai and J. K. Aggarwal Tracking Human Min in Srucured Envirnmens Using a Disribued-amera Sysem EEE Trans. n AM Vl. N. pp Nvember 999. [5] R. Rsales and S. Sclarff mprved Tracking f Muliple Humans wih Trajecry redicin and Occlusin Mdeling n rc. f VR Sana Barbara A 998. [6] R. T. llins A. J. ipn H Fujiyshi and T. Kanade Algrihms fr peraive Mulisensr Surveillance n rc. f he EEE Vl 89 pp Oc. [7] T.-J. ham and J. M. Rehg A Muliple Hyphesis Apprach Figure Tracking n rc. f EEE VR Vl. pp F. llins O June 999.

8 EEE nernainal rkshp n Visual Surveillance and erfrmance Evaluain f Tracking and Surveillance ETS 3 n cnjuncin wih V Ocber 3 Nice France. [8] Y. hen Y. Rui and T. S. Huang JDAF Based HMM fr Real-Time nur Tracking n rc. f EEE VR Vl. pp

9 EEE nernainal rkshp n Visual Surveillance and erfrmance Evaluain f Tracking and Surveillance ETS 3 n cnjuncin wih V Ocber 3 Nice France. a b c d Figure 5. Experimen resuls frm VS-ETS 3 esing sequence. a nsisen labeling f he deeced mving bjecs acrss view b Tracking and segmening w players albei heir cclusin c Tracking and segmening muliple players rushing wards he ball d 3D rajecries f racked mving bjecs viewed frm a p-dwn view e D rajecries f racked mving bjecs viewed frm each camera. e

10 EEE nernainal rkshp n Visual Surveillance and erfrmance Evaluain f Tracking and Surveillance ETS 3 n cnjuncin wih V Ocber 3 Nice France.