Facored Reasonng for Monorng Dynamc Team and Goal Formaon Av Pfeffer Harvard Unversy Davd Lawless Charles Rver Analycs Subraa Das Charles Rver Analycs Brenda Ng Harvard Unversy Absrac We sudy he problem of monorng goals, eam srucure and sae of agens, n dynamc sysems where eams and goals change over me. The seng for our sudy s an asymmerc urban warfare envronmen n whch uncoordnaed or loosely coordnaed uns may aemp o aack an mporan arge. The ask s o deec a hrea such as an ambush, as early as possble. We aemp o provde decson-makers wh early warnngs, by smulaneously monorng he posons of uns, he eams o whch hey belong, and he goals of uns. The hope s ha we can deec suaons n whch eams of uns smulaneously make movemens headed owards a arge, and we can deec her goal before hey ge o he arge. By reasonng abou eams, we may be able o deec hreas sooner han f we reasoned abou uns ndvdually. We develop a model n whch he sae space s decomposed no ndvdual uns posons, eam assgnmens and eam goals. When a un belongs o a eam adops he eam s goal. An ndvdual un s movemen depends only on s own goal, bu dfferen uns nerac as hey form eams and adop new goals. We presen an algorhm ha smulaneously racks he posons of uns, he eam srucure and eam goals. Goals are nferred from wo sources: ndvdual uns behavor, whch provdes nformaon abou her goals, and communcaons by uns, whch provdes evdence abou eam formaon. Our algorhm reasons globally abou neracons beween uns and eam formaon, and locally abou ndvdual uns behavor. We show ha our algorhm performs well a he ask, scalng o weny uns. I performs sgnfcanly beer han several alernave algorhms: sandard parcle flerng, sandard facored parcle flerng, and an algorhm ha performs all reasonng locally whn he uns. 1 Inroducon We sudy suaons n whch loosely coordnaed agens dynamcally form eams o acheve goals. The parcular scenaro ha we sudy s an urban warfare 1
envronmen, n whch uns collaborae o aack mporan arges. Our ask s o smulaneously monor he posons of uns, he eam srucure and he goals as hey evolve dynamcally over me. Whaever mehod we use should scale up o a large number of uns and arges. We presen a model for hs scenaro n whch he sae s decomposed no ndvdual un posons, eam membershp and eam goals. Indvdual agens movemen s nfluenced only by her own goals, bu agens nerac by communcang wh each oher. A naural approach o nference n such a model s o use parcle flerng (PF) [Isard and Blake, 1998; Kanazawa e al., 1995; Douce e al., 2001], bu he hgh dmensonaly of he problem makes he number of parcles needed for good monorng very hgh. In parcular, he probably ha a parcle wll conan a good poson esmae for all uns wll be very small, so all parcles wll have very small wegh. Facored PF [Ng e al., 2002] represens he dsrbuon over he sae by a se of local parcles for each un. However suffers from he same problem as PF because he local parcles are always joned ogeher before reasonng abou he dynamcs, and herefore reasonng abou he dynamcs s always done globally for all uns ogeher. We nroduce a new nference prncple: reason locally abou un posons, and globally abou eam srucure and goals. By reasonng separaely abou each un s poson, we are able o effecvely nfer s goal from s behavor, whch we would no have been able o do f we red o reason abou all uns posons ogeher. By reasonng globally abou eam srucure, we can capure neracons beween he uns. We show expermenally ha our global/local approach works very well. I sgnfcanly ouperforms ordnary PF n whch all he reasonng s done globally, and a mehod n whch all reasonng s done locally. Our mehod scales up well o 20 uns, and up o 20 arges. Several researchers have suded mul-agen goal or plan recognon, bu her focus has generally been dfferen from ours. Mos of he work looks a more complex goals and neracons beween uns, bu he goals are sac, and eam srucure s fxed n advance. For example, Inlle and Bobck [1999] recognze Amercan fooball plays, whle Devaney [2003] recognzes mlary accs nvolvng many agens. Hongeng and Nevaa [2001] provde herarchcal plan represenaons, as do Sura and Mahadevan [2004], whose approach s relaed o ours n ha uses dynamc Bayesan neworks and parcle flerng. In conras o all of hese, we allow agens o dynamcally adop goals and form eams as me elapses, bu he plans are smpler. [Lu and Chua, 2003] s more smlar o our work, n ha does focus on dynamc goal formaon. Ther approach s based on HMMs, bu hey only demonsrae for hree uns. Oher work does no consder he ssue of goals, bu does sudy rackng he movemens of neracng agens. Kahn, Balch and Dellaer [2004] s smlar o our work n ha proposes a parcle fler o rack mulple neracng arges. They use an MCMC-based approach o make he rackng problem racable, whereas we apply he global/local prncple. There are a number of dfferences beween her work and ours. They consder only local neracons beween arges ha are close o each oher, whereas n our work wo far-apar arges 2
may nerac by sharng a common goal. Furhermore, because neracons are local n her work s always known whch arges are neracng, and he same arges nerac n all parcles, whereas we have uncerany over whch arges belong o he same eam, and dfferen parcles have dfferen neracon srucure. Fnally, eam membershp n our work s a long-runnng concep, o whch here s no analogue n her work. Our work n hs paper also grealy mproves on our prevous work [Das e al., 2005]. In ha work we suded he rackng of jus wo uns usng facored parcle flerng. As we show n Secon 5, ha mehod does no scale up o a larger number of uns. In hs paper we mprove on he mehod usng he global/local prncple, scalng up reasonably well o 20 uns. 2 The problem The seng s an urban envronmen n whch here are a number of mporan locaons, and some enemy uns ha may wan o aack hem. The goal of he sysem descrbed n hs paper s o deec malcous goals when do he enemy uns have a goal of aackng a locaon? In parcular, we wan o deec hreas, when enemy uns form eams o coordnae an aack on a locaon, such as an ambush. By deecng a hrea early, we can enable frendly uns defendng he locaon o be more prepared, hereby poenally averng a dangerous suaon. The hope s ha we can deec suaons n whch eams of uns smulaneously make movemens headed owards a arge, and we can deec her goal before hey ge o he arge. By reasonng abou eams, we may be able o deec hreas sooner han f we reasoned abou uns ndvdually. The goal of a un can be nferred from wo basc sources of evdence. Frs, we can nfer goals from behavor. Ths s he source of evdence normally used n goal recognon. In our scenaro, he behavor of a un s s movemen. Thus we have o smulaneously rack he locaon of a un and reason abou s goal. Snce a un ha has a goal wll generally move n he drecon of he goal, our belefs abou s movemen wll affec our belefs abou s goal. A he same me, f we srongly beleve he un o be headng owards a parcular goal, ha wll affec our rackng of he un. The second source of evdence allows us o reason drecly abou eam formaon. Uns wll communcae wh each oher o form eams. Thus f we observe one un communcang wh anoher un ha we already beleve nends o aack a locaon, our belef ha he frs un wll have he same goal wll go up. Adequaely reasonng abou jon goals requres combnng hese wo sources of evdence. In our scenaro here s a map conssng of srees and nersecons. In our expermens, we use a map of cenral Baghdad ha has 119 srees and 79 nersecons. Each sree has a forward and a backward drecon. Ths does no mean he srees are one-way srees; a un can ravel n eher he forward or backward drecon. The drecons are used o unquely denfy whch way a un s ravelng a any pon n me. Inersecons on he map 3
have an mporance beween 0 and 1; hose wh an mporance > 0 are poenal arges. Uns move around he map. A each pon n me, we receve a nosy observaon abou he un s poson. We also receve an observaon ndcang wheher or no he un communcaed. However we do no know wh whom he un communcaed. Ths uncerany abou whch uns communcaed wh each oher rases neresng nferenal problems whch wll be addressed n Secon 4.2. 3 The model In he followng presenaon, uns and eams are denfed by negers. Uns are ndexed by and j and eams are ndexed by k. M s he number of uns. Our model uses he followng sae varables: For each un, here s a poson Poson = Sree, Forward, Dsance, Speed, where Sree s he sree he un s on, Forward ndcaes wheher he un s movng n he forward or backward drecon along he sree, Dsance s he dsance he un has raveled along he sree, and Speed s he curren speed of he un. 1 Team s he eam, f any, o whch un belongs. The value s f does no belong o a eam. We assume ha each un belongs o a mos one eam. In some applcaons may be desrable o allow a un o belong o more han one eam and have mulple concurren goals, bu allowng hs would cause an exponenal blowup n he number of possble eam assgnmens of each un. Relaxng hs assumpon s a opc for fuure research. TeamGoal k s he goal of eam k. A goal s any arge locaon. In addon o hese sae varables ha nfluence and are nfluenced by he sae a oher me pons, he model conans he followng ransen varables ha only affec he curren sae: ParComm,j s a flag ndcang wheher or no uns and j communcae, Conen,j descrbes he conen of he communcaon, f any, beween and j. Is possble values are Inve j, whch means ha s nvng j o jon s eam, Inve j, ParGoal[g], whch means ha and j sponaneously adop he new jon goal g, or NoConen, meanng ha he communcaon s no relaed o formng a eam o aack a arge. Acceped,j s a flag ndcang wheher or no he conen, f any, of he communcaon beween and j was acceped. 1 To avod confuson, we emphasze ha here s no a varable for every sree. Insead, here s a Sree node for each un, whose possble values are numbers from 1 o 129, ndcang he deny of he sree he un s on. 4
SngleComm s a flag ndcang wheher un communcaes a all. SngleGoal s he new goal, f any, sponaneously adoped by a un, no as a resul of communcaon. I may be, or any of he possble arges. Goal s he resulng goal, f any, of un, afer any new goal has been adoped or old one mananed. The value s f has no goal. In our observaon model, we assume ha each un s assocaed wh a sngle posonal observaon. By no means do we consder he daa assocaon model o be solved; we acknowledge ha s a hard problem. However, he man goal of hs work s o focus on monorng eam and goal formaon. For hs purpose we assume ha he daa assocaon s performed as a preprocessng sep. I would be neresng o nvesgae smulaneously monorng eam and goal formaon and performng daa assocaon bu ha s beyond he scope of he curren work. A each pon n me, he followng observaons are receved: A nosy observaon ObsPos = X, Y of he coordnaes of un. The posonal observaon s generaed by a Gaussan dsrbuon cenered around he rue poson, wh he x and y componens ndependen. Our framework can easly accomodae more complex models for he posonal observaon. All ha s requred s ha we can deermne he densy of he observaon gven he rue poson. Any model ha sasfes hs propery can be used. An observaon ObsComm ndcang wheher or no un communcaed. When a communcaon does no acually happen, here s a small probably ha a communcaon s erroneously repored. When a communcaon acually happens, s deeced wh hgh probably. Goal Poson 1 Poson... TeamGoal 1 TeamGoal M (a) ObsPos Team Goal (b) Fgure 1: DBN fragmens for (a) movemen model; (b) ndvdual goal model The probablsc model s represened usng a dynamc Bayesan nework (DBN). Raher han presenng he enre model n one go, we presen n fragmens. We use he noaon X o denoe he value of varable X a me. Fgure 1(a) shows he fragmen descrbng he model of ndvdual un posons. The key pon abou hs model s ha an ndvdual un s poson a me 5
depends only on s own goal a me and s own poson a me 1. Also, he posonal observaon a me depends only on he poson a me. The eam srucure and all oher uns goals have no drec nfluence on he ndvdual un s movemen. Of course, hey do have ndrec nfluence by nfluencng he un s goal. The goal affecs he new poson by nfluencng he un s decsons when reaches an nersecon or mgh make a U-urn. The desrably of movng n a ceran drecon depends on he dsance o he goal, as deermned by followng srees on he map. A un headng owards a arge wll prefer o ake he mos drec roue. When a un reaches an nersecon, he probably ha chooses a sree s hgher f he dsance o he arge along ha sree s shorer, bu here s sll some uncerany. A he begnnng of each eraon, a un may choose o make a U-urn. The probably of hs s small unless he un has a goal and he roue o he goal s shorer f makes he U-urn. The dynamcs of movemen are defned as follows: 1. The un may oponally execue a U-urn a he begnnng of he eraon. If he un does no have a goal, hs happens wh very small probably. If he un has adoped a goal, and he roue o he arge s shorer, a U-urn happens wh larger probably. In ha case, makes he U- urn wh probably qd β 1 qd β 1 +d β 2, where d 1 s he dsance o he goal afer makng he U-urn, d 2 s he dsance whou he U-urn, and q s a small consan. A U-urn akes me, and he un begns ravelng n he oppose drecon a low speed. 2. The un hen moves n whchever drecon s headng, begnnng Dsance k 1 along he sree and ravelng a Speed k 1. The poenal dsance along he road a he end of he eraon s compued, wh some Gaussan nose: Dsance k = Dsance k 1 + Speed k 1 + ɛ. 3. If Dsance k s less han he lengh of he sree, he un does no reach an nersecon, and Dsance k becomes Dsance k. A new speed s compued by mxng he begnnng speed wh a arge speed and addng Gaussan nose: Speed k = θspeed k 1 + (1 θ)targe speed + ɛ. 4. If Dsance k s greaer han he lengh of he sree, he un reaches an nersecon. If he un does no have a goal, he nex sree s hen chosen wh unform probably. If he un has a goal, chooses o ake sree j, where d j s he dsance o he goal akng he shores roue begnnng wh sree j. When he un makes a urn, he me elapsed before reachng he urn s compued, and he un proceeds o move for he remanng me n he new drecon, sarng a a slow speed. A new speed s hen compued as n sep 3. wh probably proporonal o d β j The movemen model presened here s que absrac. I could be refned n varous ways. For example, we could nroduce acceleraon no he sae, or we 6
could model he fac ha uns slow down before nersecons and U-urns. We could also nroduce a non-unform dsrbuon over he sree a un chooses a an nersecon when has no goal, makng sragh more lkely. Our nference approach can easly be adaped o a more refned movemen model, as long as manans he propery ha he new poson depends only on he old poson and he un s goal, and as long as s possble o easly sample a new poson gven an old poson. The problem may become more dffcul, f becomes more dffcul o dsngush dangerous behavor from ordnary behavor, bu he prncple wll reman he same: o reason locally abou un s behavor as much as possble, and globally abou un neracons....... ParComm 1, ParComm 1, ParComm,+1 ParComm,m SngleComm ObsComm Fgure 2: DBN fragmen for communcaon observaon Fgure 1(b) shows he DBN fragmen defnng he curren goal of a un. Goal depends on Team and all of he eam goals. I s fully deermned by s parens by he fac ha he goal of s he eam goal of Team. The condonal probably dsrbuon of Goal s a mulplexer, where Team selecs whch paren s value o use. Fgure 2 shows he DBN fragmen defnng a sngle un s communcaon. SngleComm depends on all he ParComm,j and ParComm j, where j. I s fully deermned by s parens: auologcally, a un communcaes f and only f communcaes wh someone else. ObsComm s a nosy observaon of SngleComm. 1 TeamGoal Team TeamGoal Team Team j 1 Team 1 Conen Acceped,j,j SngleGoal Team Fgure 3: DBN fragmen for Team and TeamGoal varables Fgure 3 shows a DBN fragmen for deermnng whch eam a un belongs o, and wha he eam goal s. Frs an explanaon of he noaon: TeamGoal Team means he goal, a me, of he eam o whch belongs a 7
me. TeamGoal 1 means he goal, a me 1, of he eam o whch Team belongs a me. Noe ha hs mgh no be s goal a me 1, f belonged o a dfferen eam a me 1. We make he assumpon ha no un can communcae wh more han one oher un a any pon n me. Ths assumpon s approprae f communcaon s by cellular phone, bu may no be approprae for oher communcaon mehods such as broadcas communcaons. The assumpon grealy smplfes he reasonng process. As a resul of hs assumpon here s a mos one j for whch Conen,j s neresng. (For convenence we wll always refer o he varable as Conen,j even hough mgh acually be he second subscrp.) Tha s he j ha s relevan n decdng he new eam of a un. The eam of a un can change n one of hree ways. Frs, a un may accep an nvaon from anoher un o jon s eam. Ths s ndcaed when Conen,j s Inve j, and Acceped,j s rue. In hs case Team becomes Team 1 j. TeamGoal Team s he same as TeamGoal 1. Ths s because Team belongs o j s eam a me (snce has jus acceped an nvaon from j), so Team s he same as Team j, and he goal of j s eam has no changed. Second, wo uns may sponaneously decde o form a eam and adop a new goal. Ths s ndcaed when Conen,j s ParGoal[g] and Acceped,j s rue. Thrd, a un may, on s own, sponaneously decde o adop a new goal g and form a eam conssng of a sngle un. Ths s ndcaed when SngleGoal = g. In boh he second and hrd cases, Team was prevously an empy eam ha dd no conan any uns. TeamGoal Team becomes g. In all of he above cases, he un may prevously have belonged o anoher eam. If none of hese happen, he eam o whch he un belongs wll say he same as before, and s goal wll say he same. Ths model deermnes he eams of all uns, and he goal of all eams o whch a leas one un belongs. Snce more han one un may belong o a eam, a eam goal may be deermned mulple mes. However, no conradcon can arse. The only way a eam goal can change s f one or wo uns sponaneously creae. Bu n ha case he eam mus have been empy before he uns joned. TeamGoal 1 Poson j 1 TeamGoal 1 1 Team j 1 Team Poson 1 Conen,j Acceped,j SngleGoal, Fgure 4: DBN fragmen for nvaon accepance and sponaneous goal formaon Fgure 4 shows a DBN fragmen for deermnng wheher he conen of a communcaon s acceped and wheher a un sponaneously adops a goal. The fness of a goal s equal o h g /d g where h g s he mporance of arge g, 8
and d g s he dsance of he un o arge g. Wheher or no a communcaon s acceped depends on s conen. If he conen s Inve j, he probably depends on he fness of j s goal o un. If he conen s ParGoal[g], he probably depends on he produc of he fness of g o boh uns. In all cases, he probably of accepance s smaller f he un prevously had a goal, n whch case he probably decreases wh he fness of he old goal. Ths DBN fragmen also defnes he model for SngleGoal. If he conen of any communcaon ha was nvolved n was acceped, canno sponaneously adop a new goal. Oherwse he probably ha wll adop a new goal g s proporonal o he fness of g. The overall probably ha a un wll sponaneously adop a new goal s generally small bu no neglgble. Agan, s smaller f he un already had a goal, he probably s proporonal o he fness of he new goal and nversely proporonal o he fness of he old goal. We do no show he DBN fragmen for deermnng wheher or no wo uns communcae and wha he conen of he communcaon s. The reason s ha all of he ParComm,j and Conen,j varables mus be deermned ogeher. Ths s due o he consran ha a un canno communcae wh more han one oher un. Therefore we defne a probably dsrbuon over all legal complee communcaon confguraons,.e. all complee assgnmens o ParComm and Conen. For each par of uns and j, we defne Score(ParComm,j, Conen,j ). The score of a complee confguraon wll hen be he produc of scores for all possble, j pars. Score(rue, Inve j ) s 0 f j does no have a goal, oherwse ncreases wh he fness of j s goal o. Smlarly, Score(rue, ParGoal[g]) depends on he fness of g o boh uns. Score(rue, NoConen) s a fxed consan. Fnally, we le Score(false, NoConen) be 1. Leng C be he collecon of, j pars ha communcae, hs allows us only o consder communcang uns when deermnng he probably: 4 Inference P (C),j Score(ParComm,j, Conen,j ) =,j C Score(rue, Conen,j) Our ask s o smulaneously rack he un posons, eam srucure and eam goals. A each pon n me we wan a probably dsrbuon over hese varables gven he hsory of poson and communcaon observaons. Dong hs exacly requres mananng a complee jon dsrbuon over all sae varables ha nfluence he nex me sep. Consderng he dscree varables alone, f he number of arges s L and he number of uns s M, hen here are (2 119 (L + 1)) M dscree saes, snce a un may be ravelng forwards or backwards along any of he 119 srees, and s goal may be or any of he arges. Clearly, mananng an explc jon dsrbuon over all hese saes s nfeasble for more han wo uns. Therefore we need o use an approxmae monorng algorhm. The basc framework for our algorhm s parcle flerng (PF) [Douce e al., 2001]. In 9
PF, he jon dsrbuon over he sae varables s approxmaed by a se of samples, or parcles as hey are called. Each parcle conans an assgnmen of values o he sae varables. The probably of any propery of he sae s he fracon of parcles ha have ha propery. In parcular, we are concerned wh he probably ha a hrea exss o a arge hs s smply he fracon of he parcles for whch such a hrea exss. The basc seps of PF are as follows. Le x denoe he sae varables, v he ransen varables, and y he observaons: Begn wh N parcles x 1,1,...,x 1,N. For n = 1 o N: Propagae: Sample values for v,n and ˆx,n from P (v,n, x,n x 1,n ). Condon: w n P (y v,n, ˆx,n ). Resample: For m = 1 o N: Sample x,m from ˆx,1,..., ˆx,N, wh he probably ha ˆx,n s chosen beng proporonal o w n. Propagang he parcles hrough he dynamcs and condonng on he observaons s non-rval for our problem and wll be dscussed n Secon 4.2. The dffculy wh PF for hs problem s ha he varance of he mehod s hgh and he number of parcles requred for a good approxmaon generally grows exponenally wh he dmensonaly of he problem. Therefore hs approach does no scale well wh he number of uns. An observaon s ha he dfferen uns are largely ndependen of each oher. The movemen of he dfferen uns s assumed o be ndependen; hey only nerac wh each oher by communcang and nvng each oher o jon eams or formng new eams ogeher. Therefore we mgh expec ha nsead of mananng parcles ha assgn values o all varables for all uns, we can manan local parcles ha only assgn values o varables belongng o a sngle un. Ths s he dea behnd facored parcle flerng [Ng e al., 2002]. The basc premse of facored parcle flerng s ha all he sae varables n he doman can be dvded no a se of facors. The jon dsrbuon over all sae varables s approxmaed by he produc of margnal dsrbuons over he facors, n he syle of [Boyen and Koller, 1998]. In our applcaon, he choce of facors s obvous: he varables peranng o a sngle un correspond o a facor. We beleve ha n many applcaons he correc facorzaon wll be apparen and can be suppled by hand. We are currenly lookng no ways of facorzng a complex process algorhmcally by auomacally dscoverng weakly neracng componens. In addon o decomposng he sae no facors, facored parcle flerng addonally approxmaes he margnal facor dsrbuons usng a se of facored parcles. Facored PF nroduces wo new seps no he PF process descrbed above. The frs jons facored parcles ogeher o produce global parcles. Ths sep wll be dscussed n more deal n Secon 4.1. The second projecs global parcles back down ono he facors. In beween hese wo seps, 10
all he usual seps of PF are performed. In parcular propagang hrough he dynamcs and condonng on he observaons are done wh global parcles. For hs reason, ordnary facored PF s also no deal for our suaon. The problem s ha n any global parcle, s hghly lkely ha here wll be some uns whose poson s far from he ruh. Therefore, wll ofen be he case ha for all global parcles n he se of parcles, he probably of he posonal observaon wll be exremely low. Even f one un s poson n he parcle s good, oher uns posons may be bad and so he observaon wll no confrm he frs un s poson. As a resul, nference abou uns rue posons based on he global posonal observaons wll be poor. Furhermore, uns posons are mporan ndcaons abou her goals, so he reasonng abou uns goals wll also be poor. One mgh sugges ha snce hs s he case, dong global reasonng a all s a bad dea, and all he reasonng should be done locally. However, ha wll no allow us o reason abou un neracons and communcaon beween uns. Therefore we presen an approach n whch we combne global and local reasonng. We reason globally abou un neracons and eam formaon, and locally abou he posons and ndvdual goals of uns. To avod confuson, we use he noaon P,n,T,n and G,n o refer o un s poson, eam and goal a me n he n-h local parcle belongng o facor, whereas Poson,n wll sand for he vecor of un posons a me n he n-h global parcle. The process s as follows: For each facor, begn wh N facored parcles, T 1,n, G 1,n. P 1,n Jon he facors ogeher o produce N global parcles Poson 1,n, Team 1,n, TeamGoal 1,n, each wh wegh w n. Propagae goals: For n = 1 o N: Sample ParComm,n, Conen,n gven Poson 1,n, Team 1,n, TeamGoal 1,n. Compue SngleComm,n from ParComm,n. w n w n M =1 P (ObsComm SngleComm,n ). (*) Sample Acceped,n gven Conen,n and Poson 1,n, Team 1,n, TeamGoal 1,n. Sample SngleGoal,n gven Acceped,n and Poson 1,n, Team 1,n, TeamGoal 1,n. Compue Team,n and TeamGoal,n from Conen,n, Acceped,n, SngleGoal,n as well as Team 1,n, TeamGoal 1,n. Projec: For = 1 o M: For n = 1 o N: Projec Team,n, TeamGoal,n o oban T,n, G,n. w n = M w n. 11
Propagae posons: For n = 1 o N: Sample P,n gven P 1,n, G,n. w n w np (ObsPos P,n ). Resample locally n each facor. The weghs for communcaon observaon n (*) could also be compued locally, bu n fac does no make a dfference because n Secon 4.2 we show how we avod compung hese weghs alogeher. 4.1 Jonng facored parcles As dscussed earler, he facored nference approaches nroduce a jon sep and a projecon sep no he PF framework. The projecon sep s sraghforward. A global sae unquely defnes a local sae for each un. However he jon sep s more dffcul. The jon consss of all global parcles, such ha he projecons of he global parcle ono each of he facors s conssen wh a local parcle for ha facor. Ths jon s oo large o sore or compue, so nsead we generae N samples from he resul of he jon. Snce dfferen uns may belong o he same eam, and herefore mus have he same goal, we mus make sure ha he parcles chosen from dfferen facors are conssen wh each oher. In [Ng e al., 2002], an mporance samplng process was presened for samplng from he jon. The process samples a parcle from each facor n urn. When samples a parcle from a facor, forces he varables ha overlap wh oher facors o have he same value as prevously sampled. Thus only a subse of he avalable parcles are consdered for samplng. To compensae for hs, he mehod nroduces an mporance wegh equal o he fracon of parcles n he new facor ha are conssen. The resul of he process s a global parcle wh an mporance wegh. Our mehod s smlar, bu here s a ws. [Ng e al., 2002] assumed ha he overlappng varables beween dfferen facors are fxed and known n advance. Ths assumpon s no vald here. Insead, he overlap s deermned by he ndvdual parcles chosen by samplng. Wheher or no he facors for uns 1 and 2 overlap depends on wheher he uns are on he same eam n he chosen parcles. Ths means ha we canno predefne a jon process n whch we know n advance whch varables o consran a each pon n he process. We canno opmze he jon process o ask a se of predeermned queres. We have o be flexble and deermne on he fly exacly how he dfferen facors jon ogeher. We defne a parcle-dependen process for producng a sngle weghed sample from he jon as follows. Noaon: P [V ], T [V ] and G[V ] denoe he values of P,T and G n parcle V. Se TeamGoal k = for all k. Le w = 1. For = 1 o M: Le V =. For n = 1 o N: 12
If TeamGoal T n = or G n = TeamGoal T n Add P n, T n, Gn o V. Sample a facored parcle V from V. Poson = P [V ]. Team = T [V ]. If T [V ] TeamGoal T [V] = G[V ]. w w V N. Reurn ( Poson, Team, TeamGoal, w). Unforunaely, he me complexy of he sample-jon s O(N 2 M). We need o perform he mporance samplng process once for each of he N parcles produced. The process goes hrough all M uns, and for each un needs o go hrough a process of decdng whch of he N facored parcles are conssen wh he prevously sampled facored parcles. One hng ha mgh be done s o precompue, for each facor, all ses of parcles conssen wh each possble assgnmen of goals n prevous facors, so ha when we need o sample from he facor, we know mmedaely whch parcles o use. However, hs mus be done for all possble prevous eam goal assgnmens, whch s exponenal n he number of uns. In comparson, he cos of PF s O(MN log N). Ths s an advanage of PF, bu PF may requre many more parcles. 4.2 Samplng communcaon confguraons A key sep n he samplng process s o sample a communcaon confguraon, gven he observaon abou whch uns have communcaed. Of course, we canno possbly enumerae all possble communcaon confguraons, so we canno compue he probably of any sngle confguraon. We canno drecly generae a sample confguraon, because each par of uns communcaon depends on all he oher pars. Furhermore, even f we could generae a confguraon, he probably of he observed communcaon would be very low for all bu a small fracon of confguraons. We could use Markov chan Mone Carlo echnques o sample a parcular confguraon, gven he observaons. However each sample would be que cosly o generae. Insead, we use he echnque of evdence reversal [Kanazawa e al., 1995], ogeher wh approxmang assumpons abou he model, o generae samples drecly gven he evdence. The frs assumpon s ha every me a un communcaes, he communcaon sensor wll deec. 2 I s sll possble for he sensor o repor a communcaon when here s none. Makng hs assumpon allows us o resrc aenon o uns ha we have observed communcang. Furhermore, he only pars we need o consder are hose where we have observed boh uns 2 Ths does no conradc he saemen n Secon 3 ha a communcaon s observed wh hgh probably, bu no all he me. The assumpon made here s an approxmaon o he ruh, made for he purpose of makng he samplng of communcaon confguraons racable/ 13
communcang. The second assumpon s ha he fness of goals does no deermne wheher or no uns communcae n he frs place, only he conen of he communcaon and wheher or no s acceped. As we sad earler, he goal fness shows up n mulple places n he model. Therefore gnorng n one place does no mean ha we are gnorng alogeher. We emphasze ha hese assumpons are made only by he reasonng process, no n he model self. Now, gven a se of observed communcang uns, we wan o sample a se of parwse communcaons. Each assgnmen wll mach some pars of uns ha have been observed o communcae, whle ohers wll be lef unmached. Under he assumpons, any parwse assgnmen wh he same number of machng pars mus have he same probably. How do we ensure hs? We nroduce a parameer p, whch nuvely represens he probably ha a un acually communcaes wh one parcular oher un when here are uns o communcae wh. Thus he oal probably ha a un communcaes wh any oher un s p. Le q = 1 p ; hus q s he probably ha a un does no communcae wh any oher un when here are uns o communcae wh. Now suppose we sar wh one un, and do no mach wh any oher un. Ths happens wh probably q. We hen go o anoher un, and mach wh a hrd un. Ths happens wh probably p 1. Tha gves us a confguraon of hree uns wh one unmached and wo mached. Now suppose ha nsead we mach he frs un wh a second un, whch happens wh probably p, hen go o a hrd un and do no mach, whch happens wh probably q 2. Tha also gves us a confguraon of hree uns wh one unmached and wo mached. No maer wha confguraon we choose for he remanng uns, eher of hese sars wll produce he same number of machng pars n he oal confguraon. Thus boh hese sars mus have he same probably. Ths mples ha p q 2 = q p 1, for >= 2. Therefore p = qp 1 q 2. Subsung n he defnon of q, we have p = (1 p )p 1 1 2p 2 Solvng yelds he recurrence relaonshp p = p 1 1 ( 2)p 2 + p 1 for 2 To sar he recurrence, we need a base case p 1, whch s a free parameer. p 1 can be undersood as he probably ha one un communcaes wh anoher un when here are no oher uns o communcae wh. We do no need p 0 o derve p 2 because p 2 s mulpled by 2 n he formula. The values of he p are precompued before any monorng begns. The process for samplng a confguraon s now easy. A he begnnng, each un s marked as unprocessed. We sar by consderng a un. Le he number of oher uns be m. Wh probably q m we do no mach he un beng consdered o any un. Oherwse we choose anoher un o mach o 14
unformly a random; hus each oher un has probably p m of beng chosen. We hen mark he un as beng processed. If we mached o anoher un, we also mark he oher un as beng processed. We hen repea he process unl all uns have been processed. The followng pseudocode descrbes he process: Pars For each un : Mark as unprocessed. Repea unl all uns have been processed: If here s only one unprocessed un : Mark as processed. Else: Choose an unprocessed un unformly a random. Le he number of oher unprocessed uns be m. Wh probably q m : // s no mached o any oher un Mark as processed. Oherwse: // s mached o some oher un Choose an unprocessed un j dfferen from unformly a random. Pars Pars (, j). Mark and j as processed. Reurn Pars. 5 Expermenal resuls We esed our algorhm on smulaed daa generaed from he model, and compared s performance o ordnary PF, facored PF, and an algorhm ha performs all he reasonng locally and never jons he facored parcles. Each run of he sysem lased 100 me seps. A hrea, whch was defned o be four uns sharng a common goal, was consdered o be successfully deeced f was dscovered whn 12 me seps of s developmen. Ths was enough me for each un o reach wo nersecons on average. If he hrea was no deeced whn ha me, he resul was a false negave. If a hrea was repored when none was presen, he resul was a false posve. For each expermen, excep expermen (d), we ran 500 runs and couned he number of rue posves (TP), false posves (FP), and false negaves (FN). Expermen (d) used 100 runs. T P T P +F P Our mercs are precson, whch s,.e he fracon of hreas repored T P by he algorhm ha were really hreas, and recall, whch s T P +F N,.e. he fracon of real hreas caugh by he algorhm. In each expermen we vared he hreshold of probably requred for an algorhm o repor a hrea, hereby radng off precson for recall. In all expermens, we adjused he number of parcles allocaed o each algorhm so ha hey all had approxmaely he same runnng me. The frs column of Fgure 6(f) shows he number of parcles used for mos of our expermens; he second column shows he number used 15
1 0.9 0.8 Our mehod Ordnary PF All local Facored PF Random guessng 1 0.9 0.8 Our mehod Ordnary PF All local Facored PF 0.7 0.7 0.6 0.6 Recall 0.5 Recall 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Precson (a) 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Precson (b) 1 0.9 0.8 Our mehod Ordnary PF All local Facored PF 1 0.9 0.8 Our mehod Ordnary PF All local Facored PF 0.7 0.7 0.6 0.6 Recall 0.5 Recall 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Precson (c) 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Precson (d) Fgure 5: Comparson of mehods: (a) 10 uns, 6 arges; (b) 20 uns, 6 arges; (c) 10 uns, 20 arges; (d) 10 uns, 6 arges wh more parcles. for expermen (d). Fgure 5(a) shows he precson-recall curves for each mehod for expermens wh en uns and sx arge locaons. The graph shows he recall ha could be acheved for dfferen levels of precson. Also shown for reference s he performance of random guessng. Whle all mehods do beer han random guessng, our mehod does bes, geng much hgher precson whle sll achevng hgh recall. A one pon acheves 56% precson wh 87% recall (sandard error 3.7%). A sysem wh hs level of performance would be very useful n pracce. Ineresngly, facored PF performs very poorly, ndcang ha s no smply he facorng ha leads o he good performance of our mehod, bu reasonng locally abou un posons. Also, he relavely poor performance of he mehod ha does all he reasonng locally shows he mporance of reasonng globally abou un neracons. 16
1 0.9 All evdence No communcaon evdence No poson evdence 0.8 0.7 Recall 0.6 0.5 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Precson (a) (a) (d) Our mehod 100 300 Ordnary PF 190 1550 Facored PF 115 300 All local 120 880 Tme per eraon 0.25 2 (b) Fgure 6: (a) Comparson of performance of our mehod wh all evdence and whou dfferen sources of evdence; (b) number of parcles used and runnng me per eraon n seconds n he dfferen expermens. Fgure 5(b) shows how he algorhms scale up o a suaon wh 20 uns. Agan our mehod does bes, a one pon achevng 57% precson wh 76% recall (sandard error 4.0%). Fgure 5(c) shows he performance when he number of arges s ncreased o 20. Ths s a much harder ask, because some arges are close o each oher and s dffcul o denfy a un s goals. Neverheless, our mehod s able o acheve reasonably good performance, a one pon geng 55% precson wh 51% recall (sandard error 4.5%). As we dscussed, one of he dsadvanages of our mehod s ha s quadrac n he number of parcles. Ths mgh lead us o beleve ha f we allocaed more parcles o he dfferen algorhms, he gap beween hem would close. Fgure 5(d) shows ha hs s no he case. The performance of our algorhm mproves sgnfcanly wh more parcles. Srangely, ordnary PF does somewha worse, alhough hs may be nose resulng from he fac ha only 100 runs were used for hs expermen. The sandard errors for hs curve range from 8 o 10%, ndcang ha here s a far amoun of nose, bu he rend s sll clear. A he very leas, we can say ha beng able o use many more parcles does no seem o provde an advanage o PF. Fgure 6(a) assesses he relave mporance of each of he wo sources of evdence. We see ha evdence from posonal observaons s more mporan, bu akng communcaons no accoun s also useful. Surprsngly, he mehod ha does no ake no accoun communcaon evdence performs beer han he all local mehod, perhaps because sll consders poenal un neracons. 17
6 Concluson We have presened a new model of dynamc eam and goal formaon and an algorhm for dynamcally monorng he posons of uns, he eam srucure and goals, and appled hem o an asymmerc urban warfare doman. Our mehod, based on he prncple of reasonng locally abou ndvdual uns acons and globally abou un neracons, has been shown o be successful. Ths prncple s a general one, and can be appled o any suaon n whch uns operae ndvdually bu nerac wh each oher. We have shown ha our mehod scales up o reasonably large suaons, nvolvng up o 20 uns or 20 arges. An mporan nex sep s o ry o furher scale up our mehod, o suaons nvolvng possbly hundreds of uns. Anoher nex sep s o exend he mehod o suaons n whch he number of uns changes, and uns can spl or combne dynamcally. We would also lke o allow dfferen uns o have dfferen roles on a eam. Our basc prncple should sll hold n boh cases. I would also be good o fnd a way o approxmaely compue he jon n me ha s less han quadrac n he number of parcles. Acknowledgemens Ths work has been performed under several conracs funded by ONR and OSD, wh specal hanks o Dr. Wendy Marnez. References [Boyen and Koller, 1998] X. Boyen and D. Koller. Tracable nference for complex sochasc processes. In Uncerany n Arfcal Inellgence (UAI), 1998. [Das e al., 2005] S. Das, D. Lawless, B. Ng, and A. Pfeffer. Facored parcle flerng for daa fuson and suaon assessmen n urban envronmens. In Inernaonal Conference on Informaon Fuson, 2005. [Devaney, 2003] M. Devaney. Plan recognon n large-scale mul-agen accal domans. PhD hess, College of Compung, Georga Insue of Technology, 2003. [Douce e al., 2001] A. Douce, N. de Freas, and N. Gordon (eds). Sequenal Mone Carle Mehods n Pracce. Sprnger-Verlag, 2001. [Hongeng and Nevaa, 2001] S. Hongeng and R. Nevaa. Mul-agen even recognon. In Inernaonal Conference on Compuer Vson, 2001. [Inlle and Bobck, 1999] S.S. Inlle and A.F. Bobck. A framework for recognzng mulagen acon from vsual evdence. In Naonal Conference on Arfcal Inellgence (AAAI), 1999. 18
[Isard and Blake, 1998] M. Isard and A. Blake. Condensaon condonal densy propagaon for vsual rackng. Inernaonal Journal of Compuer Vson, 29:5 28, 1998. [Kahn e al., 2004] Z. Kahn, T. Balch, and F. Dellaer. An MCMC-based parcle fler for rackng mulple neracng arges. In European Conference on Compuer Vson (ECCV), 2004. [Kanazawa e al., 1995] K. Kanazawa, D. Koller, and S. Russell. Sochasc smulaon algorhms for dynamc probablsc neworks. In Uncerany n Arfcal Inellgence (UAI), 1995. [Lu and Chua, 2003] X. Lu and C.-S. Chua. Mul-agen acvy recognon usng observaon decomposed hdden markov model. In Inernaonal Conference on Compuer Vson, 2003. [Ng e al., 2002] B. Ng, L. Peshkn, and A. Pfeffer. Facored parcles for scalable monorng. In Uncerany n Arfcal Inellgence (UAI), 2002. [Sura and Mahadevan, 2004] S. Sura and S. Mahadevan. Probablsc plan recognon n mulagen sysems. In Inernaonal Conference on Auomaed Plannng and Schedulng, 2004. 19